diff --git a/docs/Navigation.md b/docs/Navigation.md index cb0aa190..9ef24d70 100644 --- a/docs/Navigation.md +++ b/docs/Navigation.md @@ -30,8 +30,10 @@ search: - Administration - [Overview](administration/index.md) - [Users & Roles](administration/users-and-roles.md) + - [Parental Controls](administration/parental-controls.md) - [Invitations & Registration](administration/invitations-and-registration.md) - [Authentication](administration/authentication.md) + - [Email](administration/email.md) - OIDC Setup - [Overview](administration/oidc/index.md) - [Authelia](administration/oidc/authelia.md) @@ -42,8 +44,10 @@ search: - [VoidAuth](administration/oidc/voidauth.md) - [Scanning & Watcher](administration/scanning-and-watcher.md) - [Scheduled Tasks](administration/scheduled-tasks.md) + - [Library Conversion](administration/library-conversion.md) - [Server Stats](administration/server-stats.md) - [Observability](administration/observability.md) + - [Audit Log](administration/audit-log.md) - [Firmware Management](administration/firmware-management.md) - Using RomM - [Overview](using/index.md) @@ -55,6 +59,7 @@ search: - [Downloads](using/downloads.md) - [Uploads](using/uploads.md) - In-Browser Play + - [EasyRPG](using/in-browser-play/easyrpg.md) - [EmulatorJS](using/in-browser-play/emulatorjs.md) - [`js-dos`](using/in-browser-play/js-dos.md) - [MS-DOS](using/in-browser-play/ms-dos.md) @@ -64,11 +69,13 @@ search: - [Migrating to webstation](using/emulator-streaming-migration.md) - [Jukebox](using/jukebox.md) - [Saves & States](using/saves-and-states.md) + - [Devices](using/devices.md) - [RetroAchievements](using/retroachievements.md) - [Walkthroughs](using/walkthroughs.md) - [ROM Patcher](using/rom-patcher.md) - [Netplay](using/netplay.md) - [Account & Profile](using/account-and-profile.md) + - [Notifications](using/notifications.md) - [Languages](using/languages.md) - Platforms & Players - [Overview](platforms/index.md) @@ -78,6 +85,7 @@ search: - [Overview](ecosystem/index.md) - [First-Party Apps](ecosystem/first-party-apps.md) - [Feed Clients](ecosystem/feed-clients.md) + - [RetroArch Cloud Sync](ecosystem/retroarch-cloud-sync.md) - [Igir Collection Manager](ecosystem/igir.md) - API & Development - [Overview](developers/index.md) diff --git a/docs/administration/audit-log.md b/docs/administration/audit-log.md new file mode 100644 index 00000000..f7c93795 --- /dev/null +++ b/docs/administration/audit-log.md @@ -0,0 +1,79 @@ +--- +title: Audit Log +description: A record of who downloaded, played, changed and signed in to what +--- + +# Audit Log + +The audit log records who did what on the server: downloads and player launches, play sessions, uploads and edits, collection changes, scans and tasks, and security events like sign-ins, failed sign-ins and permission changes. Admins read it in the Events tab of the logs settings page, filtered by user, category and date, with a search over names and IP addresses. + +Each event keeps the actor, the action, its target, when it happened, the client's IP address and, for a request made with a device-bound token, the [device](../using/devices.md) it came from. Names are copied into the event, so it still reads after the user or the game it names is deleted. Recording is best effort, so a request still succeeds when its event can't be written. + +## What's recorded + +Events fall into five categories: + +| Category | Actions | +| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `consumption` | `rom.download`, `rom.bulk_download`, `rom.player_load`, `rom.play` | +| `library` | `rom.upload`, `rom.create`, `rom.edit`, `rom.match`, `rom.unmatch`, `rom.delete`, `rom.file_delete`, `platform.create`, `platform.edit`, `platform.delete`, `firmware.upload`, `firmware.delete`, `config.update` | +| `collections` | `collection.create`, `collection.edit`, `collection.delete`, `collection.add_roms`, `collection.remove_roms`, `smart_collection.create`, `smart_collection.edit`, `smart_collection.delete` | +| `operations` | `scan.start`, `scan.finish`, `scan.stop`, `task.run` | +| `security` | `auth.login`, `auth.login_failed`, `auth.password_reset_request`, `auth.password_reset`, `user.create`, `user.register`, `user.edit`, `user.delete`, `user.permissions_edit`, `permission_group.create`, `permission_group.edit`, `permission_group.delete`, `visibility.hide`, `visibility.unhide`, `client_token.create`, `client_token.regenerate`, `client_token.revoke`, `device.approve` | + +Some actions and actors need more detail: + +- **`rom.download` and `rom.player_load`** both come from the ROM content endpoint. When a player fetches the file to run it, the client passes `purpose=play` on `GET /api/roms/{id}/content/{file_name}` and the fetch is recorded as a player load. Any other fetch is a download (`purpose=download`, the default). A repeat of the same download by the same caller within 10 minutes counts as one event, so resumed and ranged downloads don't flood the log. +- **`rom.play`** is a [play session](../using/saves-and-states.md) reported by a player or a companion app, recorded at the time the session started. +- **Actors** are a user, an anonymous visitor (a [kiosk](authentication.md#kiosk-mode) guest, a download with endpoint auth turned off, or a failed sign-in for a username that doesn't exist), or the system for scheduled tasks and the filesystem watcher. + +## Who can read it + +| Caller | Sees | +| ------------------------------------ | --------------------------- | +| An admin with the `users.read` scope | Everyone's events | +| Any other signed-in user (`me.read`) | Only the events they caused | + +Only admins get the Events tab in the UI, but other users can still read their own history through the API. Either way, events about a platform or game the caller [can't see](users-and-roles.md) are left out, so a hidden or age-restricted game doesn't leak through the log. + +## Client IP addresses + +Each event records the client address as the web server resolved it. Behind a reverse proxy, that's the address from `X-Forwarded-For`, which is only trusted when the proxy's own address is in `FORWARDED_ALLOW_IPS`. The default trusts loopback and the private ranges, so a proxy on a public address has to be added, or every event is logged as coming from the proxy (see [Reverse Proxy](../install/reverse-proxy.md)). + +## Retention + +Events are kept for 90 days by default, and a scheduled cleanup deletes older ones daily at `30 4 * * *`, in batches so the table stays writable while it runs. + +```yaml +environment: + - AUDIT_LOG_RETENTION_DAYS=365 # 0 keeps events forever +``` + +With `AUDIT_LOG_RETENTION_DAYS=0` the cleanup task is turned off and the table only grows, so keep an eye on the database's size on a busy instance (see [Scheduled Tasks](scheduled-tasks.md)). + +## API + +`GET /api/audit-events` returns events newest first, `50` per page by default and at most `200`. + +| Query param | Description | +| ------------- | ----------------------------------------------------------------------------------------------------------- | +| `limit` | Page size, `1` to `200` | +| `offset` | Events to skip | +| `actor_id` | Only these users' events, repeatable. Ignored unless the caller sees everyone. | +| `action` | Only these actions, repeatable, such as `action=rom.download&action=rom.player_load` | +| `category` | Only these categories, repeatable: `consumption`, `library`, `collections`, `operations`, `security` | +| `target_type` | `rom`, `platform`, `firmware`, `collection`, `smart_collection`, `user`, `client_token`, `device` and so on | +| `target_id` | The target's id, used with `target_type` | +| `since` | ISO 8601 timestamp, inclusive | +| `until` | ISO 8601 timestamp, exclusive | +| `search` | Substring match on the actor's name, the target's name or the IP address | +| `max_id` | Pins later pages to the events the first page saw | + +The response is a page of `items` with `total`, `limit`, `offset` and `max_id`. Pass the first page's `max_id` back on later pages, so events recorded while you page through don't shift the results. + +```sh +curl -G https://romm.example.com/api/audit-events \ + -H "Authorization: Bearer $ROMM_TOKEN" \ + --data-urlencode "category=security" \ + --data-urlencode "since=2026-10-01T00:00:00Z" +``` diff --git a/docs/administration/authentication.md b/docs/administration/authentication.md index 377ecbb8..88792b72 100644 --- a/docs/administration/authentication.md +++ b/docs/administration/authentication.md @@ -43,9 +43,16 @@ environment: !!! warning "Keep a way in" Before setting `DISABLE_USERPASS_LOGIN=true`, confirm that at least one Admin account can sign in via OIDC. If OIDC breaks and you've already disabled local login, your only way in is editing the container env. -### Admin-triggered password reset +### Password reset -Until email-based self-serve reset lands, admins set passwords manually for any user. The next login on that account will use the new password, but existing sessions remain valid until they expire. +A user who forgot their password can ask for a reset link from the sign-in page (`POST /api/forgot-password`), which works once within 10 minutes. Where it goes depends on the server: + +- With [email](email.md) set up, `ROMM_BASE_URL` pointing at a real host, and an email address on the account, the link goes to that address, at most once a minute per user. +- Otherwise, or when the email can't be sent, the link is written to the container log for an admin to pass on. + +The response is the same whether or not the username exists, so the form can't be used to find out which accounts exist. Requests for an existing account and completed resets are both recorded in the [audit log](audit-log.md). + +Admins can also set any user's password directly, and the new password applies from the next login while existing sessions stay valid until they expire. ## OIDC diff --git a/docs/administration/email.md b/docs/administration/email.md new file mode 100644 index 00000000..b7fe67f5 --- /dev/null +++ b/docs/administration/email.md @@ -0,0 +1,56 @@ +--- +title: Email +description: Send notification emails and password reset links over SMTP +--- + +# Email + +RomM can send email through an SMTP server you provide, once both `SMTP_HOST` and `SMTP_FROM` are set. It's used for: + +- Password reset links (see [Password reset](authentication.md#password-reset)) +- Email [notification channels](../using/notifications.md#email) +- Confirmation codes that prove an address belongs to the user who added it as a channel + +## Configuration + +| Variable | Default | Description | +| --------------- | ---------- | ---------------------------------------------------------- | +| `SMTP_HOST` | | SMTP server host | +| `SMTP_PORT` | `587` | SMTP server port | +| `SMTP_USERNAME` | | Login for the SMTP server, left empty when it needs none | +| `SMTP_PASSWORD` | | Password for the SMTP server | +| `SMTP_FROM` | | Sender address, such as `romm@example.com` | +| `SMTP_SECURITY` | `starttls` | How the connection is secured: `starttls`, `tls` or `none` | + +`SMTP_SECURITY` takes one of three modes: + +- `starttls` connects in plain text and upgrades with STARTTLS before logging in, which is what port `587` expects. +- `tls` uses implicit TLS from the first byte, usually on port `465`. +- `none` sends everything, including the SMTP password, unencrypted, so keep it for a relay on the same host or a trusted network. + +Any other value leaves email off rather than falling back to plain text. Certificates are checked against the system CAs. + +```yaml +environment: + - SMTP_HOST=smtp.example.com + - SMTP_PORT=587 + - SMTP_SECURITY=starttls + - SMTP_USERNAME=romm@example.com + - SMTP_PASSWORD=app-password-here + - SMTP_FROM=romm@example.com +``` + +For a provider that only offers implicit TLS, set `SMTP_PORT=465` and `SMTP_SECURITY=tls`. Many mail providers refuse an account's normal password over SMTP and want an app password or an SMTP-specific credential instead. + +Set [`ROMM_BASE_URL`](../reference/environment-variables.md) to your instance's public URL, such as `https://romm.example.com`. Reset links are only emailed when it points at a real host, not `localhost`, a loopback address or the default `0.0.0.0`, and notification emails use it to link back into RomM. + +## Testing it + +`GET /api/heartbeat` reports the email status under `NOTIFICATIONS`: + +- `EMAIL_ENABLED` is `true` once the `SMTP_*` settings are complete. +- `EMAILS_RESET_LINKS` is `true` when reset links will be emailed, which also needs `ROMM_BASE_URL`. + +To check that mail actually goes out, add an email [notification channel](../using/notifications.md#email) for your own address. The confirmation code is the first message sent there, and if the SMTP server refuses that message, the request returns the server's error right away. A confirmed channel can send a test notification on demand too. + +If a reset link can't be emailed, it's written to the container log instead, so check `docker logs romm` for `Could not email the reset link` along with the SMTP server's error. diff --git a/docs/administration/library-conversion.md b/docs/administration/library-conversion.md new file mode 100644 index 00000000..26df0dac --- /dev/null +++ b/docs/administration/library-conversion.md @@ -0,0 +1,113 @@ +--- +title: Library Conversion +description: Convert your library to compressed formats with rom-converto +--- + +# Library Conversion + +Disc and cartridge images can be converted between formats with [rom-converto](https://github.com/DevYukine/rom-converto), in two ways: + +- **Library conversion** rewrites the files in your library to one storage format per platform, such as CHD for PlayStation or RVZ for GameCube, to save disk space. +- **Download conversion** leaves the library alone and converts a single download to a format the client asks for, for a handheld or emulator that can't read the stored one. + +## Enabling rom-converto + +Set `ROM_CONVERTO_ENABLED=true` and restart the container. The CLI is probed on first use and its version logged. If the probe fails, a warning is logged and conversion stays off until the next restart. + +| Variable | Default | Description | +| ------------------------------ | ------- | ------------------------------------------------------------------- | +| `ROM_CONVERTO_ENABLED` | `false` | Enable the Convert library task and `?format=` download conversions | +| `ROM_CONVERTO_TIMEOUT` | `600` | Seconds each rom-converto operation may run | +| `ROM_CONVERTO_MAX_CONCURRENCY` | `2` | Concurrent conversions per web or task worker | + +With it enabled, scans also use the CLI to read title IDs and serials from the files of supported platforms (3DS, DS, PSP, PS Vita, PlayStation, PS2, PS3, GameCube, Wii, Wii U, Switch, Switch 2, Xbox and Xbox 360), which you can turn off with [`converto.scan_metadata`](#configuration). + +## Configuration + +Admins set these options on the **Conversion settings** page, which writes them to the `converto` section of [`config.yml`](../reference/configuration-file.md#converto). You can also edit that section by hand: + +```yaml +converto: + download_conversion_enabled: true + platform_formats: + psx: chd + ps2: chd + ngc: rvz + wii: rvz + switch: nsz +``` + +| Key | Default | Description | +| ----------------------------- | ------- | ----------------------------------------------------------------------------------------------------------- | +| `download_conversion_enabled` | `false` | Offer converted downloads through `?format=` | +| `platform_formats` | `{}` | Library format per platform slug, used by the Convert library task. A platform left out is never converted. | +| `cache_max_size_gb` | `20` | Size cap of the converted download cache, in GB. `0` leaves it unbounded. | +| `cache_ttl_hours` | `24` | Hours a converted download stays cached after it was last served (at least `1`) | +| `scan_metadata` | `true` | Read title IDs with rom-converto during scans | + +`cache_ttl_hours` and `scan_metadata` are only set in `config.yml`, and the settings page keeps whatever value they have there. An invalid value in this section stops RomM from starting, with an `Invalid config.yml: converto...` log line. For `platform_formats`, that's a platform rom-converto can't store or a format that isn't one of its library formats, and the log line lists the valid options. + +## Library formats + +Only lossless conversions are offered as library formats, so a converted game holds the same data as the original: + +| Platform | Slug | Library formats | +| --------------------------------------- | ------------------------------- | -------------------------- | +| Nintendo 3DS | `3ds` | `z3ds`, `cia`, `cci` | +| PlayStation Portable | `psp` | `chd`, `cso`, `zso`, `iso` | +| PlayStation 2 | `ps2` | `chd`, `cso`, `zso`, `iso` | +| PlayStation, Saturn, Sega CD, Dreamcast | `psx`, `saturn`, `segacd`, `dc` | `chd` | +| GameCube | `ngc` | `rvz`, `iso` | +| Wii | `wii` | `rvz`, `iso`, `wbfs` | +| Switch, Switch 2 | `switch`, `switch-2` | `nsz`, `xcz`, `nsp`, `xci` | +| Xbox 360 | `xbox360` | `zar` | + +Each format accepts its own source files: CHD is made from `.cue` sheets or `.iso` images (and for PSP and PS2 also from `.cso`, `.zso` and `.dax`), RVZ from GameCube `.iso`, `.gcm`, `.gcz` and NKit images or Wii `.iso`, `.wbfs`, `.wia`, `.gcz` and NKit images, and NSZ from `.nsp`. A file with no route to the chosen format is counted as unsupported and left alone. The `iso`, `cia`, `cci`, `nsp`, `xci` and `wbfs` targets are uncompressed, so converting to one of them can grow your library rather than shrink it. + +## The Convert library task + +**Convert library** is a manual [task](scheduled-tasks.md) that converts every game on a platform with a library format, in place: + +- Only matched games are converted, and unidentified ones are skipped because a converted file no longer hash-matches a DAT, so match them first. +- Games keep their identity: the database entry is updated to point at the new file, so saves, states, collections, notes and play history carry over. In a multi-file game, the `.m3u` playlists beside the files are rewritten to the new names. +- Originals are deleted once their conversion succeeds and the game points at the new file. A conversion that fails, or whose output name is already taken, keeps the original. +- A `.cue` sheet converts with its `.bin` tracks, which are deleted along with it. A cue that shares tracks with another cue, or references a file outside its folder, is refused. + +Because it deletes files, the task is flagged as **destructive**, and the UI asks you to type a confirmation before it runs. It's also single-instance, so asking to run it again while it's queued or running returns `409 Conflict`. A full run can take hours on a large library, bounded per file by `ROM_CONVERTO_TIMEOUT`. When it finishes, the task reports how many files it converted, skipped as already converted, unmatched or unsupported, and failed, and how many bytes it saved. + +**Back up your library before the first run!** + +## Download conversion + +With `download_conversion_enabled` on, a client can ask for a single-file download in another format by adding `?format=` to the download URL, with a comma-separated list of formats it can read in order of preference: + +```text +GET /api/roms/{id}/content/{file_name}?format=zso,iso +``` + +The response is one of: + +| Response | When | +| ---------------------------- | --------------------------------------------------------------------------------------------- | +| `200` with the stored file | The stored file is already in one of the listed formats | +| `200` with a converted copy | A cached conversion exists, or a new one finishes within about 15 seconds | +| `202` with `Retry-After: 30` | A conversion is running. Repeat the request after the delay. | +| `406 Not Acceptable` | No listed format can be produced, the request covers more than one file, or conversion is off | + +`HEAD` on the same URL reports the same outcome without starting a conversion. A conversion started by a download keeps running when the client stops waiting, and its result lands in the cache for the next request. + +Any format rom-converto can reach from the stored file is allowed here, lossy ones included. `DetailedRomSchema.download_formats` lists the formats the caller can request for a game, which is empty for multi-file games, when conversion is off, and for callers who can't start a conversion. Only signed-in users can start one, so visitors in [kiosk mode](authentication.md) or on an unauthenticated download endpoint (`DISABLE_DOWNLOAD_ENDPOINT_AUTH`) get a cached copy or a `406`. The web UI offers the same list as "Download as" on a game page. + +### The conversion cache + +Converted downloads are cached in `/romm/cache/converts`. A copy expires `cache_ttl_hours` after it was last served, and when the cache would grow past `cache_max_size_gb`, the least recently served copies are evicted first. The **Scheduled conversion cache cleanup** task removes expired copies every day at 04:00. A conversion that failed is remembered, so it isn't retried on every request until its cache entry expires. + +## API + +| Method | Path | Description | +| ------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | +| `PUT` | `/api/config/converto_settings` | Replace `download_conversion_enabled`, `cache_max_size_gb` and `platform_formats` (needs `platforms.write`) | +| `GET` | `/api/config` | `CONVERTO` holds the current settings, and `CONVERTO_LIBRARY_TARGETS` the library formats per platform | +| `GET` | `/api/heartbeat` | `CONVERTO` reports whether rom-converto is enabled and available | +| `POST` | `/api/tasks/run/convert_library` | Run the Convert library task (needs `tasks.run`) | +| `GET`, `HEAD` | `/api/roms/{id}/content/{file_name}?format=` | Download in a requested format | diff --git a/docs/administration/observability.md b/docs/administration/observability.md index 8040dd43..ae72b79f 100644 --- a/docs/administration/observability.md +++ b/docs/administration/observability.md @@ -54,6 +54,10 @@ environment: This also disables the `GET /api/logs` endpoint that backs it. The container logs above are unaffected. +### Audit log + +The [audit log](audit-log.md) records what users did, such as who downloaded or played what, who changed the library, and who signed in or failed to. It's stored in the database rather than the container log, so it survives restarts, and admins can read it even with `DISABLE_LOGS_VIEWER=true`. + ## `/api/heartbeat` This single-request endpoint returns health and config information. It works when not logged in, though some fields only appear for authenticated callers. diff --git a/docs/administration/oidc/index.md b/docs/administration/oidc/index.md index 8e3a4f4d..9f2f7131 100644 --- a/docs/administration/oidc/index.md +++ b/docs/administration/oidc/index.md @@ -20,6 +20,8 @@ OpenID Connect (OIDC) lets users sign in through an external identity provider: 5. The code is exchanged for an ID token and the user's email, username and role claims are read. Claims the ID token leaves out are fetched from the provider's UserInfo endpoint. 6. The matching local user is logged in (see [Account matching](#account-matching)), or a new one is created on the fly unless you've [turned off registration](#auto-provisioning). +If the provider returns an error, or the code exchange or ID token fails validation, RomM logs the reason on an `OIDC callback rejected` line and sends the browser to `/login?bypass_autologin=true`, where the bypass flag keeps [autologin](#autologin) from looping straight back to the provider. A sign-in RomM refuses after that, such as for a disabled account or with registration turned off, returns an error instead of redirecting. + ## Provider guides Pick your provider and follow the step-by-step instructions. Every guide ends with the same set of app-side env vars and differs only in how you register the app and where you find the client ID and secret. @@ -50,6 +52,19 @@ environment: `OIDC_REDIRECT_URI` must exactly match what you register at the provider (same scheme, host, path, no trailing slash). +## Private certificate authority + +If your provider's HTTPS certificate comes from a private CA, such as a homelab step-ca or an internal Active Directory CA, it can't be verified out of the box. Mount the CA certificate into the container and point `OIDC_TLS_CACERTFILE` at it: + +```yaml +environment: + - OIDC_TLS_CACERTFILE=/romm/config/ca/homelab-root.pem +volumes: + - ./ca:/romm/config/ca:ro +``` + +The path can be a single bundle file or a directory, in which case every file in it is read, and PEM, DER and PKCS#7 (`.p7b`) certificates all work. These certificates are trusted alongside the system CAs, so public providers keep working. A path that doesn't exist, or a file with no certificate in it, is logged and skipped. + ## Auto-provisioning By default, the first successful OIDC login for an email that has no matching account creates a local account automatically. To require accounts to exist beforehand (so only pre-provisioned users can sign in via OIDC), turn registration off: diff --git a/docs/administration/parental-controls.md b/docs/administration/parental-controls.md new file mode 100644 index 00000000..4de33f84 --- /dev/null +++ b/docs/administration/parental-controls.md @@ -0,0 +1,69 @@ +--- +title: Parental Controls +description: Age limits on permission groups and users +--- + +# Parental Controls + +Parental controls hide games above an age from the members of a [permission group](users-and-roles.md#permission-groups), or from a single user. You set them in the group and user dialogs under Administration. + +Two settings make up a rule: + +- **Age limit**: members only see games rated for this age or younger, from `0` to `21`, with no limit as the default. +- **Hide unrated games**: also hides every game that no known rating covers, even with no age limit set. Without it, an unrated game stays visible whatever the limit, because there's nothing to judge it by. + +## Who a rule applies to + +- **Groups** carry a limit and a hide-unrated switch for every member. A user with no group follows the server's default group, its age settings included. +- **Users** can replace either setting of their group. A user's own value wins, and an unset value inherits the group's, so you can give one member a stricter or looser limit than the rest of the group. +- **Admins** are never limited, whatever their group says, because they bypass permission groups entirely. + +## Where it applies + +An age rule hides a game the same way as a [hidden ROM](users-and-roles.md#hidden-entities), so it stays out of everything a member can reach: the gallery, search, collections (smart and virtual ones included), server stats, recommendations, the Jukebox, feeds for [feed clients](../ecosystem/feed-clients.md), netplay rooms, device installs and sync, and every per-ROM route such as downloads, manuals and screenshots. A direct request for one answers as if the game didn't exist, usually with a `404`. + +## How a game's age is worked out + +Each game stores one minimum age, the **strictest** age any of its ratings sets, which is recomputed whenever that metadata changes. Ratings are read from: + +| Source | What RomM reads | +| ----------------------------- | ------------------------------------------------------------------- | +| IGDB | Every age rating, from any board | +| ScreenScraper | Every classification, from any board | +| LaunchBox | The ESRB rating | +| Steam | The required age, when the store sets one | +| Manual metadata (Edit dialog) | The game's own age rating list, which **replaces** all of the above | + +As on the game page, the manual list overrides the providers entirely, so you can loosen a rating as well as tighten it. Manual entries take the form `BOARD:RATING`, such as `ESRB:T` or `PEGI:12`. + +A game counts as unrated when it has no ratings at all, or only ones that can't be read (an unknown board, or an unrecognised value). + +### Rating boards + +Boards that rate with letters map to the youngest age each rating admits: + +| Board | Ratings | +| --------------- | ------------------------------------------------------------------------- | +| ESRB | `EC` 3, `E` 6, `KA` (Kids to Adults) 6, `E10` 10, `T` 13, `M` 17, `AO` 18 | +| CERO | `A` 0, `B` 12, `C` 15, `D` 17, `Z` 18 | +| ACB (also OFLC) | `G` 0, `PG` 8, `M` 15, `MA15` 15, `R18` 18, `X18` 18, `RC` 18 | +| BBFC | `U` 0, `PG` 8, `12` 12, `12A` 12, `15` 15, `18` 18, `R18` 18 | + +Boards that rate with numbers use the number as the age, with `L` or `ALL` meaning any age: PEGI, USK, GRAC, ClassInd (also DJCTQ), ScreenScraper's own `SS` classification, ELSPA and JV. + +ESRB's `E` maps to 6 rather than to "everyone" because it's treated as the pre-1998 Kids to Adults floor, so a 3-year limit hides `E` games but shows `EC` ones. + +## Existing libraries + +The upgrade to 5.4 computes the minimum age for every game already in the database, so limits work straight away. A game picks up new ratings on its next metadata refresh, and its age follows. + +## API + +The permission group schemas (`PermissionGroupSchema`, `PermissionGroupCreate` and `PermissionGroupUpdate`) and the user permission schemas (`UserPermissionsSchema` and `UserPermissionsUpdate`) carry two fields: + +| Field | On a group | On a user | +| ------------------- | ----------------------------------- | ------------------------------------------------- | +| `age_limit` | `0` to `21`, or `null` for no limit | `0` to `21`, or `null` to inherit the group's | +| `hide_unrated_roms` | `true` or `false` | `true`, `false`, or `null` to inherit the group's | + +The update payloads (`PUT /api/permissions/groups/{id}` and `PUT /api/permissions/users/{user_id}`) apply both fields only when `set_age_settings` is `true`, and then write them exactly as given, so an update that leaves `set_age_settings` out never touches its age rule. Both routes require `users.write` and an admin caller. diff --git a/docs/administration/scheduled-tasks.md b/docs/administration/scheduled-tasks.md index b1f0f2ec..813b4cd8 100644 --- a/docs/administration/scheduled-tasks.md +++ b/docs/administration/scheduled-tasks.md @@ -35,6 +35,13 @@ Unlike other tasks, **build recommendations index** ships enabled, because the [ The housekeeping tasks (netplay cleanup, upload tmp cleanup, ZIP cache cleanup, sync session cleanup) also ship enabled. Each has an `ENABLE_SCHEDULED_CLEANUP_*` var to turn it off and a matching `SCHEDULED_CLEANUP_*_CRON` to move it. With upload tmp cleanup off, abandoned chunked uploads stay in `tmp/uploads` under `ROMM_TMP_PATH` (or the resources folder when that's unset) until you delete them. Check the [env var reference](../reference/environment-variables.md) for the full list. +A few tasks follow other settings instead of an `ENABLE_*` var: + +- **Scheduled audit log cleanup** removes [events](audit-log.md) older than `AUDIT_LOG_RETENTION_DAYS` (90 by default) every day at 04:30, and doesn't run at all when that var is `0`, which keeps events forever. +- **Scheduled streaming session reaper** runs every minute, but only while [emulator streaming](../using/emulator-streaming.md) is enabled, and stops sessions whose player stopped sending heartbeats. +- **Scheduled conversion cache cleanup** removes expired [converted downloads](library-conversion.md#the-conversion-cache) every day at 04:00. +- **Convert library** is manual only, and can only run with `ROM_CONVERTO_ENABLED=true` (see [Library Conversion](library-conversion.md)). + ## Triggering a task manually ### From the Administration page @@ -48,6 +55,24 @@ POST /api/tasks/run/{task_name} Authorization: Bearer ``` +The call answers `503` when no task worker is running to pick the job up. Some tasks, such as Convert library, are **single-instance**: asking to run one while it's already queued or running returns `409 Conflict` instead of queueing a second copy. + +Library scans have their own endpoint, `POST /api/tasks/scan`, for clients that authenticate with a token rather than a session cookie. Its optional JSON body takes the same options as the `scan` socket event, and an empty body queues a quick scan of the whole library with every enabled metadata source: + +```http +POST /api/tasks/scan +Authorization: Bearer +Content-Type: application/json + +{"type": "quick", "platforms": [12], "apis": ["igdb", "ss"]} +``` + +The body also accepts `platform_fs_slugs`, `roms_ids` and `launchbox_remote_enabled`, and rejects unknown keys with `422`. It answers `202` with the queued job, `409` while another scan is in flight, and `503` when no scan worker is running. + +### Destructive tasks + +A task that deletes or replaces files in your library is flagged `destructive` in the task list (`GET /api/tasks`), and the UI asks you to type a confirmation before running it. Convert library is one, since it deletes each original once its converted copy is in place. The API doesn't ask for a confirmation, so a script that calls one runs it straight away. + ## Monitoring tasks - **Live**: Administration → Tasks page shows every task's current status (queued, running, idle, failed). diff --git a/docs/administration/users-and-roles.md b/docs/administration/users-and-roles.md index 417ad6c2..d8b1a5ba 100644 --- a/docs/administration/users-and-roles.md +++ b/docs/administration/users-and-roles.md @@ -38,6 +38,17 @@ Rules of the model: - **`own_only`** narrows a grant to entities the user owns. For example, `assets.write` with `own_only` lets a user manage their own saves/states/screenshots but not anyone else's. - **Default group**: exactly one group is marked as the server-wide default and is applied automatically to every new User (invite sign-up, OIDC first login, admin-created accounts without an explicit group). +### Seeded groups + +Two groups are seeded, which are also used by an upgrade from a release before permission groups to keep existing accounts working: + +| Group | Grants | +| ---------- | --------------------------------------------------------------------------------------- | +| **Viewer** | Read the library, and manage only your own collections, assets and devices | +| **Editor** | Viewer access plus library-wide create, edit and delete of ROMs, platforms and firmware | + +You can rename them, edit their grants or delete them like any other group, and the UI warns you first. In the API, `PermissionGroupSchema.system_key` marks them as `viewer` or `editor` whatever they're called, and is `null` on groups you created. + ### Per-user overrides On top of the group, you can add or revoke individual capabilities for one user without creating a whole new group: @@ -51,6 +62,10 @@ Use overrides for one-offs ("this one user can also delete ROMs"), and use group Beyond allow/deny, you can hide specific platforms or ROMs from a user or from an entire group. A hidden entity simply doesn't appear for that principal, regardless of read grants. Firmware can't be hidden directly, because its visibility cascades from the platform it belongs to. +### Age limits + +A group can also hide games rated above an age from its members. Each user can replace the group's limit with their own, and admins are never limited (see [Parental Controls](parental-controls.md)). + ## Creating users Three paths: diff --git a/docs/developers/api-authentication.md b/docs/developers/api-authentication.md index 39e116a0..30b70662 100644 --- a/docs/developers/api-authentication.md +++ b/docs/developers/api-authentication.md @@ -113,6 +113,17 @@ Users signing in through an OIDC provider get a regular RomM session, same as us OIDC provider setup lives in [Administration → OIDC](../administration/oidc/index.md). +## Browser clients on another origin (CORS) + +A browser app on another domain, port or scheme can only call the API if that origin is listed in `ROMM_CORS_ALLOWED_ORIGINS`. The list is empty by default, which denies every cross-origin request: + +```yaml +environment: + - ROMM_CORS_ALLOWED_ORIGINS=https://dashboard.example.com +``` + +A listed origin gets `Access-Control-Allow-Credentials: true`, so the browser can send the session cookie with `credentials: "include"`. A `*` entry answers any origin but turns that header off for every origin, listed ones included, so it only suits requests that carry no cookie, such as ones authenticated with a bearer token in the `Authorization` header. Native apps, scripts and server-side code don't send an `Origin` header, so CORS doesn't apply to them. + ## Which scopes do I need? Every endpoint in the [API Reference](api-reference.md) lists its required scopes. The short version: @@ -129,6 +140,8 @@ Every endpoint in the [API Reference](api-reference.md) lists its required scope | `403 Forbidden` | Authenticated but the identity lacks a required scope. | | `404 Not Found` | The resource doesn't exist, or, for privacy, the identity can't see it. | +An unusable credential never fails the request on its own. A malformed `Authorization` header, a Basic header that doesn't decode, wrong Basic credentials, or an invalid or expired JWT all leave the request unauthenticated, and it then gets the route's usual answer: `401` on a route that needs a login, or the normal response on one that doesn't. + When debugging a 403, check: 1. The **user's role** in Administration → Users. diff --git a/docs/developers/api-reference.md b/docs/developers/api-reference.md index 0bc7d11a..a5b5dc8a 100644 --- a/docs/developers/api-reference.md +++ b/docs/developers/api-reference.md @@ -18,6 +18,26 @@ The raw spec: For code generation, see [Consuming OpenAPI](openapi.md). +## Starting a scan + +Clients that authenticate with a token rather than a session cookie can queue a library scan with `POST /api/tasks/scan`, which needs the `tasks.run` scope and takes the same `ScanPayload` body as the `scan` socket event (see [WebSockets → Scans](websockets.md#scans)). Leave the body out for a quick scan of the whole library: + +```bash +curl -X POST https://demo.romm.app/api/tasks/scan \ + -H "Authorization: Bearer rmm_..." \ + -H "Content-Type: application/json" \ + -d '{"type": "unmatched", "platforms": [12], "apis": ["igdb", "ss"]}' +``` + +| Status | Meaning | +| ------ | ------------------------------------------------------------------------------------------- | +| `202` | Queued. The response's `task_id` can be followed on `GET /api/tasks/{task_id}` | +| `409` | A library scan is already queued or running. A scan limited to `roms_ids` is still accepted | +| `422` | The body has an unknown key or an invalid value | +| `503` | No scan worker is running, so the scan can't be queued | + +`POST /api/tasks/run/{task_name}` likewise returns `409` when a single-instance task such as Convert library is already queued or running. In `GET /api/tasks`, destructive tasks such as Convert library (which replaces the original files) carry `destructive: true`, so a client can ask for confirmation before running one. + ## WebSockets Alongside REST, two socket.io endpoints cover live-update and coordination use cases (see [WebSockets](websockets.md)). diff --git a/docs/developers/client-api-tokens.md b/docs/developers/client-api-tokens.md index 62b60a29..a1e422c6 100644 --- a/docs/developers/client-api-tokens.md +++ b/docs/developers/client-api-tokens.md @@ -80,6 +80,16 @@ The token's owner generates the code from a device already signed into RomM (usu The companion app stores the token and uses it on every subsequent API call. RomM treats it like any other token, and pairing only affects how the token reached the device. +## Device-bound tokens + +A token issued through the device authorization flow under `/api/auth/device/` (where the device starts the flow and the user approves its code) is bound to the [device](device-sync-protocol.md#registering-a-device) it registers. A bound token identifies its device on every call, so sync endpoints can leave out `device_id`, and some calls accept nothing else: + +- Claiming and reporting [installs on devices](device-sync-protocol.md#installs-on-devices) only works with a token bound to that device. +- The [`/devices` socket](device-sync-protocol.md#the-devices-socket) only admits bound tokens that hold `devices.read`. +- It can read its own device's play sessions without `devices.read`. + +It can't manage installs for the user's other devices, and deleting the device closes the sockets its tokens opened. + ## Scoping tokens properly A token can only hold scopes the owning user _also_ holds, and a user's scopes come from their [permission group](../administration/users-and-roles.md#permission-groups) plus any overrides (Admins hold everything). For example, a token can't carry `users.write` unless its owner is an Admin. Default to read-only, and only grant write scopes the app actually needs. diff --git a/docs/developers/development-setup.md b/docs/developers/development-setup.md index a8d754dc..768e7728 100644 --- a/docs/developers/development-setup.md +++ b/docs/developers/development-setup.md @@ -159,6 +159,16 @@ ln -s ../romm_mock/assets assets/romm/assets npm run dev ``` +#### Run the frontend against a remote RomM + +Set `DEV_PROXY_TARGET` in the repo-root `.env` to another instance's origin, such as a home server with a full library: + +```sh +DEV_PROXY_TARGET=https://romm.example.com +``` + +Vite then proxies `/api`, `/ws`, `/netplay`, `/openapi.json`, the EasyRPG player's game files and `/assets/romm` to that instance instead of the local backend, which you don't need to run. Leave it empty to keep the default `http://127.0.0.1:${DEV_PORT}`. The remote's TLS certificate must be valid. Sign in with a username and password, because OIDC doesn't work through the proxy: the identity provider redirects to the remote's `OIDC_REDIRECT_URI`, so the session never reaches localhost. + ## Setting up the linter We use [Trunk](https://trunk.io) for linting, which combines multiple linters and formatters with sensible defaults and a single configuration file. You'll need to install the Trunk CLI to use it. diff --git a/docs/developers/device-sync-protocol.md b/docs/developers/device-sync-protocol.md index 01ac8de5..71d74d2a 100644 --- a/docs/developers/device-sync-protocol.md +++ b/docs/developers/device-sync-protocol.md @@ -23,17 +23,21 @@ The sync endpoints accept either: Required scopes: -| Endpoint | Scope | -| ------------------------------------------------------- | ------------------------------ | -| `POST /api/devices` | `devices.write` | -| `POST /api/sync/negotiate` | `assets.read` + `devices.read` | -| `POST /api/sync/sessions/{id}/complete` | `devices.write` | -| `GET /api/sync/sessions`, `GET /api/sync/sessions/{id}` | `devices.read` | -| `POST /api/saves`, `PUT /api/saves/{id}` | `assets.write` | -| `GET /api/saves/{id}/content` | `assets.read` | -| `POST /api/saves/{id}/downloaded` | `devices.write` | -| `POST /api/play-sessions` | `roms.user.write` | -| `POST /api/sync/devices/{device_id}/push-pull` | `devices.write` | +| Endpoint | Scope | +| ------------------------------------------------------- | ------------------------------------------------------ | +| `POST /api/devices` | `devices.write` | +| `POST /api/sync/negotiate` | `assets.read` + `devices.read` | +| `POST /api/sync/sessions/{id}/complete` | `devices.write` | +| `GET /api/sync/sessions`, `GET /api/sync/sessions/{id}` | `devices.read` | +| `POST /api/saves`, `PUT /api/saves/{id}` | `assets.write` | +| `GET /api/saves/{id}/content` | `assets.read` | +| `POST /api/saves/{id}/downloaded` | `devices.write` | +| `POST /api/play-sessions` | `roms.user.write` | +| `GET /api/play-sessions` | `roms.user.read` (see [Play sessions](#play-sessions)) | +| `POST /api/sync/devices/{device_id}/push-pull` | `devices.write` | +| `PUT /api/devices/{device_id}` | `devices.write` | +| `POST /api/devices/{device_id}/installs/claim` | `devices.write` + `roms.read`, device-bound token | +| `PUT /api/devices/{device_id}/installs/{request_id}` | `devices.write`, device-bound token | Passing `device_id` to the save endpoints also needs `devices.write` on upload and update, and `devices.read` on download. Without it the call returns `403`. @@ -57,21 +61,21 @@ Content-Type: application/json } ``` -| Field | Notes | -| ----------------- | -------------------------------------------------------------------------------------------- | -| `name` | Display name. | -| `platform` | Device platform or OS. | -| `client` | Short slug for the app (for example `grout`). The activity feed shows it as the device type. | -| `client_version` | App version. | -| `ip_address` | Device IP address. | -| `mac_address` | Used to match an existing device. | -| `hostname` | Used to match an existing device. | -| `sync_mode` | `api`, `file_transfer` or `push_pull`. | -| `sync_config` | Mode-specific settings. | -| `capabilities` | Flags the device reports about itself, for example `{ "remote_install": true }`. | -| `allow_existing` | Default `true`. Return a matching existing device. `false` returns `409` if one exists. | -| `allow_duplicate` | Default `false`. `true` always creates a new device and turns off `allow_existing`. | -| `reset_syncs` | Default `false`. | +| Field | Notes | +| ----------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| `name` | Display name. | +| `platform` | Device platform or OS. | +| `client` | Short slug for the app (for example `grout`). The activity feed shows it as the device type. | +| `client_version` | App version. | +| `ip_address` | Device IP address. | +| `mac_address` | Used to match an existing device. | +| `hostname` | Used to match an existing device. | +| `sync_mode` | `api`, `file_transfer` or `push_pull`. | +| `sync_config` | Mode-specific settings. | +| `capabilities` | Boolean flags the device reports about itself, for example `{ "remote_install": true }`. At most 32 keys of 1 to 64 characters. | +| `allow_existing` | Default `true`. Return a matching existing device. `false` returns `409` if one exists. | +| `allow_duplicate` | Default `false`. `true` always creates a new device and turns off `allow_existing`. | +| `reset_syncs` | Default `false`. | Registration is idempotent. The server looks for one of the user's devices with the same `mac_address`, or failing that the same `hostname` and `platform`, and returns it (`200`) instead of creating a second one (`201`), unless `allow_duplicate` is set. With `allow_existing: false`, a match returns `409` with `error: "device_exists"` and the existing `device_id`. Sending `capabilities` for an existing device updates them. @@ -87,6 +91,8 @@ Response: `device_id` is a string (a UUID). Cache it for subsequent calls. +`PUT /api/devices/{device_id}` updates a device with the same fields, plus `sync_enabled`. A device with `sync_enabled: false` gets `400` from negotiate until sync is turned back on from the user's [devices list](../using/devices.md). `DELETE /api/devices/{device_id}` removes a device, closes the sockets its tokens opened and drops its queued installs, but keeps the saves it uploaded. + ## Sync negotiation The device sends the saves it has and RomM returns what to do: @@ -108,7 +114,9 @@ Content-Type: application/json "file_size_bytes": 8192 } ], - "rom_ids": [1234] + "rom_ids": [1234], + "restore_unlisted": false, + "emulators": null } ``` @@ -116,8 +124,10 @@ Content-Type: application/json - An unknown device returns `404`. A device with sync turned off returns `400`. - `content_hash` is an MD5 hex digest of the file. - `rom_ids` is an optional, read-only scope. Downloads are offered only for these ROMs, plus any ROM a save was sent for. Leaving a ROM out never deletes anything. The number of IDs per request is capped. +- `restore_unlisted` (default `false`) offers every current server save the device didn't list as a `download`, even one it already synced. Normally a save the device synced before and no longer lists is taken as deleted there and left alone. Set it for a client that never deletes saves itself, such as a browser whose storage can be evicted, so a missing save reads as lost and comes back. +- `emulators` (default `null`) limits pairing to server saves written by one of these emulators, so a save another emulator wrote into the same slot is neither paired nor offered. RomM's browser players send their own emulator here. - Saves are paired on **(`rom_id`, `slot`)**. Send a stable slot such as `autosave`. A `null` slot marks an archival or manual save: it is never paired, so it always comes back as `upload`. -- Every negotiate opens a new session, so one device can have several open at once (for example, two games running). A session nobody completes is marked failed by a scheduled cleanup after 24 hours. +- Every negotiate opens a new session that lasts one launch, so one device can have several open at once (for example, two games running). A session nobody completes is marked failed by the scheduled sync session cleanup (`ENABLE_SCHEDULED_CLEANUP_SYNC_SESSIONS`, hourly at `23 * * * *` by default, set by `SCHEDULED_CLEANUP_SYNC_SESSIONS_CRON`) once it's 24 hours old. Response: @@ -224,18 +234,103 @@ Content-Type: application/json - The response is `{ session, play_session_ingest }`. - An unknown session returns `404`. A session that was cancelled or failed on purpose returns `400`. One that the cleanup expired can still be completed, so its counts and playtime aren't lost. +## Play sessions + +`GET /api/play-sessions` lists the caller's sessions, filtered by `rom_id`, `device_id`, `start_after` and `end_before`. Reading another device's sessions, or every device's at once, needs `devices.read`. A device-bound token without that scope can still read its own device's sessions by passing its own `device_id`. The server never fills in `device_id` from the token, so leaving it out lists every device and needs `devices.read`. + +Play session responses have no `sync_session_id` field. + +## Paging + +`GET /api/play-sessions` and `GET /api/sync/sessions` share these page parameters: + +| Parameter | Default | Range | +| --------- | ------- | ----------- | +| `limit` | `50` | 1 to 10000 | +| `offset` | `0` | 0 and above | + +On play sessions, `limit` is ignored when `start_after` or `end_before` is set, so a time window always comes back whole. + ## Other endpoints -- `GET /api/sync/sessions` and `GET /api/sync/sessions/{id}`: list and inspect sync sessions. +- `GET /api/sync/sessions` (filtered by `device_id`) and `GET /api/sync/sessions/{id}`: list and inspect sync sessions. - `POST /api/sync/devices/{device_id}/push-pull`: push/pull sync for a device. See the [API Reference](api-reference.md) for their full schemas. +## Installs on devices + +Users can send a game to one of their devices from the web UI (see [Devices](../using/devices.md#install-on-device)), and the device downloads it the next time it's online. A device opts in by registering with the `remote_install` capability: + +```json +{ "capabilities": { "remote_install": true } } +``` + +Only devices with that flag are offered as install targets. Every install endpoint answers `404` when the server sets `DEVICE_INSTALL_ENABLED=false`, and `GET /api/heartbeat` reports it under `DEVICE_INSTALL` (`ENABLED` and `EXCLUDED_PLATFORM_SLUGS`). + +### Request lifecycle + +An install request is a JSON object: + +```json +{ + "id": "b6f1d3c2-6a0e-4d55-9f3e-7f6b1c2d9e10", + "user_id": 1, + "device_id": "3f1c2b9e-8a4d-4c7e-9f21-6d0b5a7e1c34", + "rom_id": 1234, + "file_ids": [5678, 5679], + "status": "pending", + "reason": null, + "created_at": "2026-10-05T09:00:00Z", + "updated_at": "2026-10-05T09:00:00Z" +} +``` + +`file_ids` lists the ROM's game, update and DLC files on disk, which is what the device should download, for example with `GET /api/roms/{rom_id}/content/{file_name}?file_ids=5678,5679`. + +| Status | Meaning | +| ------------------- | --------------------------------------------------------- | +| `pending` | Queued, waiting for the device to claim it | +| `taken` | Claimed by the device, which is downloading it | +| `done` | The device installed it | +| `already_installed` | The device already had it | +| `failed` | The device couldn't install it, with an optional `reason` | +| `cancelled` | The user cancelled it before it finished | + +`done`, `already_installed`, `failed` and `cancelled` end a request, and an ended request is gone from every list. A request nobody touches expires `DEVICE_INSTALL_REQUEST_TTL_DAYS` days (2 by default) after its last change. + +### Endpoints + +| Method and path | Called by | Description | +| ------------------------------------------------------- | --------- | ------------------------------------------------------------------------------------------------------ | +| `POST /api/devices/{device_id}/installs` | Web UI | Queue `{ "rom_id": 1234 }`. `201` with a new request, or `200` with the live one for the same pair | +| `GET /api/devices/{device_id}/installs` | Either | The device's `pending` and `taken` requests, oldest first | +| `POST /api/devices/{device_id}/installs/claim` | Device | Take every `pending` request and return all the device holds `taken`, oldest first | +| `PUT /api/devices/{device_id}/installs/{request_id}` | Device | Report `{ "status": "done" \| "already_installed" \| "failed", "reason": "..." }` on a `taken` request | +| `DELETE /api/devices/{device_id}/installs/{request_id}` | Either | Cancel a `pending` or `taken` request | +| `GET /api/devices/online` | Web UI | IDs of the caller's devices with an open `/devices` socket | +| `GET /api/roms/{id}/installs` | Web UI | The caller's open requests for a ROM, across devices | + +Queueing needs `devices.write` and `roms.read`. A ROM the user can't see returns `404`, and a device without the `remote_install` capability, a ROM on a platform in `DEVICE_INSTALL_EXCLUDED_PLATFORM_SLUGS` or a ROM with no installable file returns `400`. Claiming and reporting need a token bound to that same device (see [Client API Tokens](client-api-tokens.md#device-bound-tokens)), and any other caller gets `403`. A report or cancel on a request in the wrong status returns `409`, and `reason` is capped at 500 characters. Each report also sends the user a notification. + +### The `/devices` socket + +A device learns about requests over this Socket.IO namespace. Connect with a device-bound client token holding `devices.read`, passed either in the handshake's `auth` payload as `{ "token": "rmm_..." }` or as an `Authorization: Bearer` header. Any other credential is refused, and so is every connection while installs are disabled. + +The server sends two events, each with a payload of `{ "id": "", "rom_id": 1234 }`: + +| Event | Meaning | +| ------------------- | --------------------------------------------------------------------- | +| `install:queued` | A request is waiting. Claim it with `POST .../installs/claim` | +| `install:cancelled` | The user cancelled a request. Stop downloading it if it's in progress | + +The open socket also marks the device as online until the server drops it, which happens when the token expires or is revoked, or when the device is deleted. Events sent while the device is offline aren't replayed, so claim on every connect to pick up whatever queued in the meantime. + ## Rate limits and polling - Sync once per session, not per save - Don't poll `/api/sync/negotiate` tightly -- Nothing tells a device to start a sync. The `/devices` Socket.IO namespace only carries install requests, so the device decides when to negotiate +- Nothing tells a device to start a sync. The [`/devices` Socket.IO namespace](#the-devices-socket) only carries install requests, so the device decides when to negotiate ## See also diff --git a/docs/developers/websockets.md b/docs/developers/websockets.md index d6334539..1fb7e19d 100644 --- a/docs/developers/websockets.md +++ b/docs/developers/websockets.md @@ -7,29 +7,172 @@ description: Endpoints for live updates RomM uses socket.io for real-time communication over two endpoints: -| Endpoint | Purpose | -| -------------------- | ----------------------------------------------------------------------------- | -| `/ws/socket.io` | Live updates (scan progress, task status, ROM mutations, admin notifications) | -| `/netplay/socket.io` | Netplay session coordination (room discovery, join/leave, lifecycle) | +| Endpoint | Purpose | +| -------------------- | ---------------------------------------------------------------------------------------------------------------------- | +| `/ws/socket.io` | Live updates (scans, notifications, sync, activity, installs, streaming, log streaming), plus the `/devices` namespace | +| `/netplay/socket.io` | Netplay session coordination (room discovery, join/leave, WebRTC signalling) | ## Authentication -socket.io connections inherit the HTTP session, so if your `Cookie` header carries a session cookie, the WebSocket connection authenticates as that user. Programmatic clients pass the credential via the handshake `auth` field: +The default namespace on `/ws/socket.io` and the netplay endpoint authenticate with the browser's session cookie, and the handshake binds the socket to the login session in `romm_session`. A socket without a valid session still connects, but it joins no per-user room, so it never receives events addressed to a user, and the events that act on a user's behalf (`scan`, `activity:*`) are rejected or ignored. + +Signing out, or revoking a session, disconnects every socket that session opened, on both endpoints and on every worker, and the client has to sign in again before it reconnects. + +### The `/devices` namespace + +Companion apps that hold a [device-bound Client API Token](client-api-tokens.md) connect to this namespace instead, passing the token in the handshake's `auth` payload or in an `Authorization: Bearer` header: ```javascript -const socket = io("https://demo.romm.app", { +const socket = io("https://demo.romm.app/devices", { path: "/ws/socket.io", - auth: { - token: "rmm_...", - }, + transports: ["websocket"], + auth: { token: "rmm_..." }, }); ``` -A failed handshake closes the connection with an error event. +The handshake is refused with `unauthorized` unless the token is live, bound to a device, and holds `devices.read` (as does its owner), and with `disabled` when `DEVICE_INSTALL_ENABLED=false`. While connected, the device counts as online for `GET /api/devices/online`, until the socket closes because the token expired or was revoked, or the device was deleted. + +## Events + +### Scans + +Client to server, both requiring the `tasks.run` scope: + +| Event | Payload | +| ----------- | ------------------------------------------------------------------ | +| `scan` | A `ScanPayload` object, the same body `POST /api/tasks/scan` takes | +| `scan:stop` | None. Cancels queued scans and stops the running one | + +`ScanPayload` rejects unknown keys, so a misspelt option fails instead of silently scanning the whole library. Every field is optional: + +```json +{ + "type": "quick", + "platforms": [12, 34], + "platform_fs_slugs": [], + "roms_ids": [], + "apis": ["igdb", "ss", "hasheous"], + "launchbox_remote_enabled": true +} +``` + +| Field | Default | Meaning | +| -------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `type` | `quick` | One of `new_platforms`, `quick`, `update`, `unmatched`, `complete`, `hashes` | +| `platforms` | `[]` (all) | Platform ids to scan | +| `platform_fs_slugs` | `[]` | Platform folder names to scan | +| `roms_ids` | `[]` | Scan only these ROMs. Such a scan may queue while a library scan runs, and goes ahead of any that's waiting | +| `apis` | every enabled provider | Metadata providers to use, by source key: `igdb`, `moby`, `ss`, `ra`, `launchbox`, `hasheous`, `tgdb`, `sgdb`, `flashpoint`, `hltb`, `demozoo`, `pouet`, `csdb`, `steam`, `gamelist`, `libretro`, `playmatch` | +| `launchbox_remote_enabled` | `true` | Let LaunchBox use its remote API as well as the local metadata store | + +Server to client, broadcast to every connected socket: + +| Event | Payload | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------ | +| `scan:scanning_platform` | The platform being scanned | +| `scan:scanning_rom` | The ROM just scanned, as a `SimpleRomSchema` without its file list, plus `is_new` (`true` when this scan added it) | +| `scan:update_stats` | Running totals for the scan | +| `scan:done` | Final stats | +| `scan:done_ko` | An error message string | + +`scan:done_ko` is also sent, only to the socket that asked, when a `scan` event couldn't start. That happens when the caller lacks `tasks.run`, when the payload fails validation (the message lists each invalid field), when no scan worker is running, or when a library scan is already queued or running. + +### Notifications + +Sent to every open tab of the user they belong to (see [Notifications](../using/notifications.md)): + +| Event | Payload | +| ------------------------- | -------------------------------------------------------------- | +| `notifications:new` | The new notification, as `GET /api/notifications` returns it | +| `notifications:read` | `{"ids": [...]}`, or `{"ids": null}` when all were marked read | +| `notifications:dismissed` | `{"ids": [...]}`, or `{"ids": null}` when all were dismissed | + +### Activity + +The "now playing" feed. Clients report their own sessions, and the acting user always comes from the socket's session, never from the payload: + +| Event | Direction | Payload | +| -------------------- | ---------------- | -------------------------------------------------------------------------------------- | +| `activity:start` | Client to server | `{"rom_id": 123, "device_id": "..."}` | +| `activity:heartbeat` | Client to server | Same as `activity:start`, to keep the session alive | +| `activity:stop` | Client to server | Same as `activity:start` | +| `activity:update` | Server to client | An activity entry, sent only to users who can see the ROM | +| `activity:clear` | Server to client | `{"user_id", "device_id", "rom_id"}`, sent to the same audience | +| `activity:refresh` | Server to client | `{}`. Refetch `GET /api/activity`, because the session's audience couldn't be resolved | + +`activity:update` reaches only the `user:{id}` rooms of users allowed to see the ROM, which takes [hidden entities](../administration/users-and-roles.md#hidden-entities) and [age limits](../administration/parental-controls.md) into account. When that audience can't be resolved, `activity:refresh` goes to everyone instead, coalesced across workers so a burst of failures produces one refresh. + +### Device sync + +Sent to the user's tabs while a device syncs (see [Device Sync Protocol](device-sync-protocol.md)). Every payload carries `device_id` and `session_id`: + +| Event | Extra fields | +| ---------------- | ------------------------------------------------------------ | +| `sync:started` | `sync_mode` | +| `sync:progress` | `operations_completed`, `operations_planned`, `current_file` | +| `sync:completed` | `operations_completed`, `operations_failed` | +| `sync:conflict` | `file_name`, `rom_id`, `rom_name`, `reason` | +| `sync:error` | `error` | + +### Device installs + +| Event | Sent to | Payload | +| ------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------- | +| `install:updated` | The owner's tabs, on the default namespace | The full install request: `id`, `user_id`, `device_id`, `rom_id`, `file_ids`, `status`, `reason`, `created_at`, `updated_at` | +| `install:queued` | The device, on `/devices` | `{"id", "rom_id"}`. A request is waiting, so claim it with `POST /api/devices/{device_id}/installs/claim` | +| `install:cancelled` | The device, on `/devices` | `{"id", "rom_id"}`. Drop the request, even if the download already started | + +`status` is one of `pending`, `taken`, `done`, `already_installed`, `failed` or `cancelled`. + +### Permissions and logs + +| Event | Sent to | Payload | +| --------------------- | ------------ | ------------------------------------------------------------------------------ | +| `permissions:changed` | Every socket | `{"user_id": 123}`. The named user should refetch `GET /api/permissions/me` | +| `logs:entry` | Admins only | One backend log line, for the live log viewer (off with `DISABLE_LOGS_VIEWER`) | + +### Emulator streaming + +Sent to the user's tabs while an [emulator streaming](../using/emulator-streaming.md) session runs. Each payload names the `platform`, the `container`, and the `claimed_at` timestamp of the claim it belongs to, so a tab can ignore events meant for an older claim of the same container: + +| Event | Extra fields | +| ------------------------- | ---------------------------------------------------------------------------------------------------- | +| `streaming:launch-phase` | `phase`, the broker's extraction phase | +| `streaming:launch-ready` | `host`, `resume`, and the RetroArch `core` and `core_tier` the session booted | +| `streaming:launch-failed` | `detail`, plus `refusals` and `refusals_truncated` when the broker refused an imported save or state | +| `streaming:session-ended` | `ended_by`, `reason`, `ended_at`, `rom_id`, `rom_name` and `desktop` | + +### Netplay + +Netplay runs on its own endpoint, `/netplay/socket.io`, so a client there can never address the per-user or admin rooms of the main endpoint. RomM's player connects to it over the WebSocket transport only, because long polling breaks when gunicorn runs more than one worker. + +| Event | Who may send it | +| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------ | +| `open-room` | A signed-in user with `roms.read` who can see the ROM named by `extra.game_id` (a ROM id) | +| `join-room` | For a room with a password, anyone who sends the right one. For a room without, the same rule as `open-room` | +| `leave-room`, `webrtc-signal`, `data-message`, `snapshot`, `input` | Players already in the room | + +The room password travels as a top-level `password` field beside `extra`, which is where EmulatorJS sends it: + +```json +{ + "extra": { + "sessionid": "a1b2c3", + "userid": "player-1", + "room_name": "Friday co-op", + "game_id": "123", + "player_name": "alice" + }, + "maxPlayers": 2, + "password": "hunter2" +} +``` + +Both events answer through the socket.io acknowledgement with an error string, such as `Not authorized to open a room for this game` or `Incorrect password`, when they fail. Over REST, `GET /api/netplay/list?game_id=` lists a game's open rooms, requiring `roms.read` and returning `404` for a ROM the caller can't see. ## Reverse-proxy requirements -Every reverse-proxy setup must forward the WebSocket upgrade. The recipes in [Reverse Proxy](../install/reverse-proxy.md) all keep WebSockets on by default. +Your proxy must forward the WebSocket upgrade, which all the recipes in [Reverse Proxy](../install/reverse-proxy.md) do by default. The main UI falls back to HTTP long polling when the upgrade fails, but polling only works with a single gunicorn worker (`WEB_SERVER_CONCURRENCY=1`), since with the default of 4 a request can land on a worker that never saw the handshake. Netplay has no fallback at all and stops working outright. Common breakages: diff --git a/docs/ecosystem/index.md b/docs/ecosystem/index.md index 260877f9..dcfd8e1e 100644 --- a/docs/ecosystem/index.md +++ b/docs/ecosystem/index.md @@ -25,6 +25,10 @@ RomM exposes several URL feed endpoints for external homebrew/custom firmware ap - **[fpkgi](feed-clients.md#fpkgi)**: PS4/PS5 installer - **[Kekatsu](feed-clients.md#kekatsu)**: Nintendo DS multiboot loader +## RetroArch + +RomM is a WebDAV target for RetroArch's built-in Cloud Sync, so saves and states sync between RetroArch and your library with no companion app (see [RetroArch Cloud Sync](retroarch-cloud-sync.md)). + ## Community apps Individuals in the community, not the team, maintain these apps, so support quality varies. diff --git a/docs/ecosystem/retroarch-cloud-sync.md b/docs/ecosystem/retroarch-cloud-sync.md new file mode 100644 index 00000000..62728d7b --- /dev/null +++ b/docs/ecosystem/retroarch-cloud-sync.md @@ -0,0 +1,140 @@ +--- +title: RetroArch Cloud Sync +description: Use RomM as the WebDAV target for RetroArch's built-in Cloud Sync +--- + +# RetroArch Cloud Sync + +RetroArch has a built-in Cloud Sync feature that mirrors its saves, states and other folders to a WebDAV server. RomM can be that server, so RetroArch syncs directly into your library without a companion app. + +Each save and state is attached to the ROM it belongs to, next to the ones from the web player and other [devices](../using/devices.md). + +## Requirements + +- A RetroArch build with Cloud Sync and the WebDAV driver (desktop, Android and most handheld builds have it) +- A RomM account with a password, since RetroArch signs in with HTTP Basic auth and an account that only signs in through [OIDC](../administration/oidc/index.md) has none to send +- Your ROM files named the same on both sides, because that's how RomM tells which game a save belongs to (see [How files match games](#how-files-match-games)) + +## Configure RetroArch + +Point Cloud Sync at RomM's WebDAV path and sign in with your RomM username and password: + +```text +https://romm.example.com/api/sync/retroarch/ +``` + +The trailing slash matters, because RetroArch appends its own paths to the URL as written. + +You can set it in the Cloud Sync settings or directly in `retroarch.cfg`: + +```ini +cloud_sync_enable = "true" +cloud_sync_driver = "webdav" +webdav_url = "https://romm.example.com/api/sync/retroarch/" +webdav_username = "your-romm-username" +webdav_password = "your-romm-password" + +# What to sync +cloud_sync_sync_saves = "true" +cloud_sync_sync_configs = "true" +cloud_sync_sync_thumbs = "false" +cloud_sync_sync_system = "false" +``` + +We also recommend turning on RetroArch's options to sort saves and states into per-core folders (`sort_savefiles_enable` and `sort_savestates_enable`). RomM stores each save and state with the emulator that wrote it, and the per-core folder is how that emulator survives the round trip (see [Cores and folders](#cores-and-folders)). + +## What syncs + +| RetroArch folder | Where it goes in RomM | +| ---------------- | ----------------------------------------------------------------------------------------------- | +| `saves/` | A save on the matching ROM, for your user | +| `states/` | A state on the matching ROM, for your user, with RetroArch's `.png` thumbnail as its screenshot | +| `config/` | Stored as-is, per user | +| `thumbnails/` | Stored as-is, per user | +| `system/` | Stored as-is, per user | + +The `config`, `thumbnails` and `system` folders belong to no game, so RomM keeps them as opaque files under `/romm/retroarch_sync/users//` and hands them back unchanged to every RetroArch install signed in as that user. To RomM those installs are all one device, because RetroArch's requests don't say which install they come from. The first sync registers a single device named **RetroArch**, which shows up with the user's other [devices](../using/devices.md). RomM also ignores the `manifest.server` file RetroArch uploads and builds its own from the database on every sync, so RetroArch always compares against what RomM actually holds. + +### Saves + +RetroArch sees the saves stored without a slot, which covers everything it uploaded itself and saves you uploaded by hand. Saves written into a [slot](../using/saves-and-states.md#save-slots), such as the browser player's `autosave` history or the ones other device-sync apps upload, are left out of the manifest because they're RomM's versioned history and not files a core would load. + +### States + +RetroArch only loads states from its numbered slots (`.state`, `.state1` and so on, plus `.state.auto`). RomM groups every state for a game by core and by the slot its file name ends in, and offers the newest one in each slot under the name RetroArch would give it. + +A state made in RomM's web player has a label and a timestamp in its name instead of a slot number, but it still ends in `.state`, so RomM offers it as that game's slot 0 when it's the newest state there, and RetroArch loads it like any other. + +### Deletes + +Deleting a save or state in RetroArch deletes it in RomM too. In non-destructive mode RetroArch moves a deleted file into a `deleted/` folder instead, and RomM treats that move as a delete as well, since keeping the row would push the file straight back on the next sync. + +## How files match games + +A synced file names no platform or ROM, so RomM matches it by file name. The save or state's name, minus its extension (and minus the `.state`, `.stateN` or `.state.auto` suffix for a state), has to equal a ROM's file name without its extension. RetroArch names saves after the content file it loaded, so this works as long as the ROM is named the same on the device and in RomM. + +- `saves/Snes9x/Super Mario World (USA).srm` matches the ROM file `Super Mario World (USA).sfc` +- `states/Snes9x/Super Mario World (USA).state2` matches the same ROM, in slot 2 + +The match prefers the exact spelling and falls back to a case-insensitive one. Only ROMs you can see count, and when two visible ROMs on different platforms share a name, the one with the lowest ID wins. + +An upload that matches no ROM is refused with `409`, and a `Cloud sync upload ... matches no ROM in the library` warning is logged. RetroArch reports the sync as failed for that file and tries again next time. + +### Cores and folders + +With saves and states sorted by core, RetroArch puts them in folders named after the core, such as `saves/Snes9x/`. RomM maps the common RetroArch folder names to the core IDs its web player uses, so a state from the web player's `snes9x` core shows up in RetroArch's `Snes9x` folder and the other way around. A folder name that isn't recognized is stored unchanged as the emulator name. + +## PSP saves + +PPSSPP keeps each game's save as a folder of files under `PSP/SAVEDATA//`, and those files only make sense together. RomM bundles each folder into one zipped save on the matching ROM, named `PSP-.zip`, and unpacks it again when RetroArch downloads it. Files under `PSP/SYSTEM/` are PPSSPP's own caches and are ignored. + +The ROM for a save folder is found in two ways: + +1. **From `SYNC_RETROARCH_PSP_SERIAL_MAP`**, if the folder's serial is listed there +2. **From the title in the folder's `PARAM.SFO`**, compared against your ROMs' names with punctuation and case ignored + +The serial is the folder name with any trailing `DATA` dropped, so `ULUS10041DATA00` has the serial `ULUS10041`. Until one of those finds a ROM, the folder's files are held in the cache and retried when the rest of the folder arrives. + +When a game's title doesn't match the ROM's name, as with a translated title or a ROM renamed by hand, map its serial to the ROM's file name without the extension: + +```yaml +environment: + SYNC_RETROARCH_PSP_SERIAL_MAP: '{"ULUS10041": "Lumines (USA)", "NPJH50465": "Persona 3 Portable (Japan) (En)"}' +``` + +Whenever a folder can't be matched, the serial to add is logged. + +## Browsing with other WebDAV clients + +The same path works read-only in generic WebDAV clients, such as a file manager that mounts WebDAV shares: + +- `roms/` lists your platforms and their ROM files. Opening a file redirects to the normal [download endpoint](../using/downloads.md), so range requests and multi-file ZIPs behave the same way. +- `saves/` and `states/` list the same files RetroArch sees. + +Reading anything needs the `assets.read` permission, and browsing `roms/` also needs `roms.read` and shows only the platforms and games you can see. With [kiosk mode](../administration/authentication.md#kiosk-mode) on, anonymous visitors can browse `roms/` without signing in, but `saves/` and `states/` still ask for credentials. Changes (`PUT`, `DELETE`, `MOVE`, `MKCOL`) need `assets.write`. `LOCK` and `UNLOCK` always succeed, because some clients refuse to mount a share without a lock. + +## Reverse proxy + +WebDAV is served under `/api/sync/retroarch/`, so a proxy that already forwards `/api` covers it, as long as it lets these methods through: + +```text +OPTIONS, PROPFIND, GET, HEAD, PUT, DELETE, MKCOL, MOVE, LOCK, UNLOCK +``` + +Most proxies forward any method, but some web application firewalls and CDN rules block the WebDAV ones. The proxy also has to pass the `Authorization` header through and accept upload bodies as large as your biggest save or state (see [Reverse Proxy](../install/reverse-proxy.md)). + +## Troubleshooting + +- **RetroArch says the sync failed right away**: check the URL ends in `/api/sync/retroarch/`, with the trailing slash, and that the username and password sign in to the RomM web UI. +- **A save never shows up in RomM**: its file name matches no ROM you can see. Look for a `matches no ROM in the library` warning in the logs, then rename the ROM or the save so they agree. +- **A web player state doesn't show up in RetroArch**: only the newest state per slot is offered, so a newer state in the same slot hides it. RetroArch also has to be sorting states by core for the state to land in the folder the core reads from. +- **A browser save doesn't show up in RetroArch**: saves in a slot aren't offered to RetroArch (see [Saves](#saves)). +- **A PSP save doesn't sync**: the folder's title didn't match a ROM. Add the serial from the log message to `SYNC_RETROARCH_PSP_SERIAL_MAP` and restart RomM. +- **Errors only on `PROPFIND`, `MOVE` or `MKCOL` requests**: something in front of RomM blocks WebDAV methods (see [Reverse proxy](#reverse-proxy)). +- **Large states fail to upload**: raise your proxy's body size limit. + +## See also + +- [Saves & States](../using/saves-and-states.md): how they're stored +- [Devices](../using/devices.md): the RetroArch device and the others you sync from +- [Device Sync Protocol](../developers/device-sync-protocol.md): the API that companion apps sync over instead diff --git a/docs/getting-started/metadata-providers.md b/docs/getting-started/metadata-providers.md index 16541eb7..dff17613 100644 --- a/docs/getting-started/metadata-providers.md +++ b/docs/getting-started/metadata-providers.md @@ -97,6 +97,8 @@ To enable LaunchBox, set `LAUNCHBOX_API_ENABLED=true` and `ENABLE_SCHEDULED_UPDA You must run a LaunchBox metadata update (either manually, or scheduled via cron) to generate a local `.xml` file with Launchbox metadata before using it as a metadata provider. Manual LaunchBox updates can be run from the System/Administration/Tasks Page. The server will parse the local `.xml` file when trying to match a ROM and fetch metadata from this source. +Besides the front cover, LaunchBox supplies the box back, the spine and the 3D box for games that have them, picking the image for the ROM's region first. These are stored under the same media types ScreenScraper uses (`box2d_back`, `box2d_side` and `box3d`), so a face one provider lacks can come from the other, and like ScreenScraper's they're only downloaded when listed in [`scan.media`](#boxart-styles-and-media-types). + ### Hasheous [Hasheous](https://hasheous.org/) is a free, open-source metadata provider that matches games by file hash (MD5, SHA1 and CRC). It proxies IGDB data for titles, descriptions, cover art and screenshots, and adds region and language tags from the matched dump. @@ -135,6 +137,8 @@ Matching is hash-only: RomM computes each ROM's RetroAchievements hash with RAHa After that, each user needs to set their own username in their profile and sync it with RetroAchievements. A new `Achievements` tab will appear in the `Personal` tab in the game details. +A game matched by its RetroAchievements hash counts as **verified**, the same as a Hasheous match against a No-Intro, Redump or other DAT, so it shows in the `verified` gallery filter and [smart collection](../using/smart-collections.md) rule. A later hash scan that no longer finds the hash on RetroAchievements clears the flag. A game matched by an `(ra-xxxx)` filename tag or by hand is only verified when its file's hash is also one RetroAchievements lists for that game. + To avoid unnecessary API calls, a cached file with the RA database is stored locally. Refresh time for that cache file can be changed with the environment variable `REFRESH_RETROACHIEVEMENTS_CACHE_DAYS`. @@ -322,6 +326,20 @@ Scans parse custom metadata tags in the filename that match specific patterns an Filenames will not be renamed to add tags, as they are a non-standard formatting system and could create conflicts with other software. +### Regions, languages and tags from hash matches + +When ScreenScraper or Hasheous matches a game by its hash, the game takes the region and languages of the exact dump that matched, so a correctly dumped `.chd` with no tags in its name still gets them. ScreenScraper's flags on that dump also become tags: a translation, a hack, a beta or a demo gets the same `Translation`, `Hack`, `Beta` or `Demo` tag a filename would give it. A scan that skips one of these sources keeps the tags it gave before. + +### Fan translations + +Filenames with a fan translation tag get the `Translation` tag, and the language the tag names is added to the game's languages, since that's the language you'll play it in. The common spellings are all recognised, and a superseded GoodTools translation (`[T-Eng]`) is tagged the same way: + +| Set convention | Example | Language added | +| -------------- | ------------------------------- | -------------- | +| GoodTools | `[T+Eng1.1_RPGe]` | English | +| TOSEC | `[tr fr]` | French | +| Plain | `(Translation)`, `(Translated)` | none | + ## Boxart styles and media types The gallery renders one piece of cover art per game card. Which one it shows is controlled by the **Boxart style** picker, a per-user setting under **Settings → User Interface**. The options are: diff --git a/docs/install/image-variants.md b/docs/install/image-variants.md index c5ded6c8..7e7bb088 100644 --- a/docs/install/image-variants.md +++ b/docs/install/image-variants.md @@ -12,6 +12,8 @@ RomM publishes two production image variants. They're interchangeable at the con | **Full** (default) | `rommapp/romm:latest` `rommapp/romm:5.0.0` | ~400MB | You want in-browser play with every runtime baked in (most users pick this). | | **Slim** | `rommapp/romm:slim` `rommapp/romm:5.0.0-slim` | ~100MB | Headless use, or you're fine fetching EmulatorJS and `js-dos` from a CDN at runtime. | +The Ruffle, PICO-8 and [EasyRPG](../using/in-browser-play/easyrpg.md) players ship with the full image only and, unlike EmulatorJS and `js-dos`, have no CDN fallback, so slim can't play Flash, PICO-8 or RPG Maker 2000/2003 games in the browser. Both variants include the `rom-converto` CLI used by [library conversion](../administration/library-conversion.md). + Both variants are published on Docker Hub (`docker.io/rommapp/romm`) and GitHub Container Registry (`ghcr.io/rommapp/romm`). The GHCR images track the same tags and are a good choice if you run into Docker Hub's rate limits. ## Switching variants diff --git a/docs/install/reverse-proxy.md b/docs/install/reverse-proxy.md index ceb743ee..65589693 100644 --- a/docs/install/reverse-proxy.md +++ b/docs/install/reverse-proxy.md @@ -243,26 +243,36 @@ What each block buys you: ## Set `FORWARDED_ALLOW_IPS` for your proxy -Inside the container, a bundled nginx sits in front of gunicorn, and by default gunicorn only trusts that nginx. When you put your own reverse proxy in front, the bundled nginx appends your proxy's address to `X-Forwarded-For` before passing the request on. Gunicorn reads the header from right to left and takes the first address it doesn't trust as the client. That address is your proxy, so every request looks like it came from the proxy instead of the real visitor. +Inside the container, a bundled nginx sits in front of gunicorn and appends the address each request came from to `X-Forwarded-For`. Gunicorn reads that header from right to left, skips every address listed in `FORWARDED_ALLOW_IPS`, and takes the first one it doesn't trust as the client. By default it trusts loopback and the private ranges: -The client IP matters beyond the logs. RomM uses it to rate-limit metadata heartbeats, device pairing and client tokens, and as part of the fingerprint that identifies a browser as a web device. When every visitor shares the proxy's address, they all share one rate-limit bucket, and a single user hitting the limit blocks everyone else until the window resets. Every browser behind the proxy also ends up registered as the same device. +```text +127.0.0.1,::1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10,fc00::/7 +``` + +That covers a reverse proxy on the same host, on a Docker network, on your LAN or on a Tailscale network (`100.64.0.0/10`), and a Cloudflare Tunnel because `cloudflared` connects from inside your network, so most setups need no change. + +A proxy that reaches RomM from a public address, such as a VPS that forwards to your home server or a CDN that connects straight to your public IP, isn't trusted. Every visitor then looks like the proxy, which affects more than the logs: + +- The [audit log](../administration/audit-log.md) records the proxy's address for every sign-in, failed sign-in and download. It also caps how many failed sign-ins it records per address, so one noisy client can crowd out everyone else's. +- Metadata heartbeats, device pairing and client token requests are rate limited per address, so every visitor shares one bucket, and a single user hitting the limit blocks everyone else until the window resets. +- The address is part of the fingerprint that identifies a browser as a web device, so every browser behind the proxy ends up registered as the same device. -Append your proxy's address to the default rather than replacing it: +Append the proxy's public address (or its published ranges, for a CDN) to the default rather than replacing it: ```yaml environment: - - FORWARDED_ALLOW_IPS=127.0.0.1,172.18.0.0/16 + - FORWARDED_ALLOW_IPS=127.0.0.1,::1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10,fc00::/7,203.0.113.10 ``` -The value is a comma-separated list of addresses and CIDR ranges. If your proxy runs in another container, use a range, because the container gets a new address each time it's recreated. To find the address, check the last entry of `X-Forwarded-For` on a real request. Your proxy's access log shows it, and so does the bundled nginx log (`docker logs romm`). +The value is a comma-separated list of addresses and CIDR ranges. To find the address to add, check the last entry of `X-Forwarded-For` on a real request in the bundled nginx log (`docker logs romm`). !!! warning "Keep `127.0.0.1`, and don't reach for `*`" `127.0.0.1` is the bundled nginx, the only hop gunicorn talks to directly, so leave it in the list. `*` trusts every hop, which makes gunicorn fall back to the leftmost entry of the header, and that one is whatever the client sent. -!!! note "Why listing your proxy is still safe" - Addresses to the left of the first untrusted hop are never read, so adding your proxy to the list does not hand clients a way to spoof the address they are seen as. +!!! note "What trusting a range means" + When every address in the header is trusted, gunicorn takes the leftmost one, which the client wrote itself. With the default list, a client connecting from your own private network can therefore choose the address recorded for it. Clients connecting from the internet through a trusted proxy can't, because their real address is the first untrusted hop. If you don't trust your LAN, narrow the list to your proxy's own address. ## Set `ROMM_BASE_URL` behind HTTPS @@ -286,4 +296,10 @@ environment: ``` - `ROMM_SESSION_SECURE_COOKIE` marks the session and CSRF cookies `Secure` so browsers only send them over HTTPS. Leave it `false` if you still reach the instance over plain HTTP, or logins will silently fail. -- `ROMM_CORS_ALLOWED_ORIGINS` is a comma-separated list of origins allowed to call the API from a browser. It's empty by default, which blocks every other origin, and most deployments can leave it that way because the UI and the API share an origin. Add an entry only for another browser app that calls this instance. `*` allows any origin, but browsers won't hand a wildcard response to a request that carries credentials, so it only works for endpoints that don't need a login. +- `ROMM_CORS_ALLOWED_ORIGINS` is a comma-separated list of origins allowed to call the API from a browser. It's empty by default, which denies every other origin, and most deployments can leave it that way because the UI and the API share an origin. Add an entry only for another browser app that calls this instance, written as scheme, host and port with no trailing slash. `*` answers any origin, but then no response carries `Access-Control-Allow-Credentials`, even for origins also listed by name, so browsers won't send the session cookie and only endpoints that don't need a login work. + +Native apps and scripts don't send an `Origin` header, so CORS doesn't affect them. Before 5.4, an empty list allowed every origin. + +## RetroArch Cloud Sync (WebDAV) + +[RetroArch Cloud Sync](../ecosystem/retroarch-cloud-sync.md) talks WebDAV to `/api/sync/retroarch/`, sending `OPTIONS`, `PROPFIND`, `MOVE`, `MKCOL`, `LOCK` and `UNLOCK` besides `GET`, `HEAD`, `PUT` and `DELETE`. Caddy, nginx, Traefik and NPM pass any method through by default, but a proxy, WAF or security rule that filters HTTP methods has to allow these on that path. The client signs in with HTTP Basic, so the proxy must also forward the `Authorization` header and RomM's `401` challenge untouched. diff --git a/docs/platforms/index.md b/docs/platforms/index.md index 0dc38982..b9cc5403 100644 --- a/docs/platforms/index.md +++ b/docs/platforms/index.md @@ -10,6 +10,6 @@ description: Everything about the supported platforms ## Related sections -- **[In-Browser Play](../using/in-browser-play/emulatorjs.md)**: the end-user side of EmulatorJS, `js-dos`, and Ruffle +- **[In-Browser Play](../using/in-browser-play/emulatorjs.md)**: the end-user side of EmulatorJS, `js-dos`, Ruffle, PICO-8 and EasyRPG - **[Metadata Providers](../getting-started/metadata-providers.md)** - **[Folder Structure](../getting-started/folder-structure.md)** diff --git a/docs/reference/configuration-file.md b/docs/reference/configuration-file.md index 518c130e..7cd868a6 100644 --- a/docs/reference/configuration-file.md +++ b/docs/reference/configuration-file.md @@ -610,8 +610,56 @@ See [Emulator Streaming → Memory cards](../using/emulator-streaming.md#memory- --- +## `converto` + +Configure [library and download conversion](../administration/library-conversion.md) with rom-converto, which only takes effect with `ROM_CONVERTO_ENABLED=true`. Admins can also edit `download_conversion_enabled`, `platform_formats` and `cache_max_size_gb` from the Conversion settings page, which writes them back to this file. + +```yaml +converto: + download_conversion_enabled: true + scan_metadata: true + cache_ttl_hours: 24 + cache_max_size_gb: 20 + platform_formats: + psx: chd + ngc: rvz +``` + +### `converto.download_conversion_enabled` + +**Default:** `false` + +Let clients ask for a single-file download in another format with `?format=` on the download URL. + +### `converto.platform_formats` + +**Default:** `{}` + +Map of [platform slug](../platforms/supported-platforms.md) to the format the Convert library task stores that platform's games in. Platforms left out are never converted. Each platform only accepts its own lossless library formats (see [Library formats](../administration/library-conversion.md#library-formats)), and an invalid entry stops RomM from starting, with a log line listing the valid options. + +### `converto.cache_ttl_hours` + +**Default:** `24` (minimum `1`) + +Hours a converted download stays cached after it was last served. + +### `converto.cache_max_size_gb` + +**Default:** `20` (`0` for no limit) + +Size cap of the converted download cache, past which the least recently served copies are evicted first. + +### `converto.scan_metadata` + +**Default:** `true` + +Read title IDs and serials with rom-converto while scanning the platforms it supports. + +--- + ## Related - [Folder Structure](../getting-started/folder-structure.md): how the filesystem shape interacts with `config.yml` - [Metadata Providers](../getting-started/metadata-providers.md): per-provider detail for the `scan.priority.*` slugs - [Emulator Streaming](../using/emulator-streaming.md): the full setup guide for the `streaming` block +- [Library Conversion](../administration/library-conversion.md): what the `converto` block drives diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index f0dbb45f..2f002ab1 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -11,10 +11,16 @@ These are the terms the docs, UI, and API use consistently. Foundational concept **Admin**: highest user role. Full scope set, including user management and task execution (see [Users & Roles](../administration/users-and-roles.md)). +**Age limit**: a parental control set on a permission group or a single user. Members see only games rated for that age or younger, and admins are never limited (see [Parental Controls](../administration/parental-controls.md)). + **API token**: see Client API Token. +**Apprise**: the notification library behind the admin-only channels for services such as Discord, Telegram, ntfy or Matrix (see [Notifications](../using/notifications.md)). + **Asset**: user-uploaded content attached to a ROM, such as save files, emulator states, and screenshots. Assets live under `/romm/assets` (separate from the library) and are owned per-user. Saves and states can sync to registered devices, and are not the same as a Resource (see [Saves & States](../using/saves-and-states.md)). +**Audit log**: the record of who did what, such as downloads, launches, edits, scans and sign-ins, kept for `AUDIT_LOG_RETENTION_DAYS` (see [Audit Log](../administration/audit-log.md)). + **Autologin**: OIDC feature that bypasses the login page and redirects straight to the IdP. Set via `OIDC_AUTOLOGIN=true`. **Basic auth**: HTTP auth with username+password in a header. Supported for API calls. @@ -27,6 +33,10 @@ These are the terms the docs, UI, and API use consistently. Foundational concept **Device**: a registered endpoint that syncs with RomM, such as a handheld running Grout, an Android phone running Argosy, or a SteamDeck running DeckRommSync. Devices pull saves and states, and some push them back after a session (see [Device Sync Protocol](../developers/device-sync-protocol.md)). +**Device-bound token**: a Client API Token tied to one registered device. Only a bound token can claim that device's install requests or connect to the `/devices` socket namespace (see [Client API Tokens](../developers/client-api-tokens.md)). + +**EasyRPG**: the bundled in-browser player for RPG Maker 2000 and 2003 games, full image only (see [In-Browser Play → EasyRPG](../using/in-browser-play/easyrpg.md)). + **EmulatorJS**: the bundled in-browser retro emulator. Handles NES, SNES, N64, PSX, Saturn, and 20+ more cores (see [In-Browser Play → EmulatorJS](../using/in-browser-play/emulatorjs.md)). **Facet**: one axis a [recommendation](../using/recommendations.md) is explained by, such as a shared franchise, genre, theme or developer. A facet counts for as much as it is rare in your own library. @@ -35,12 +45,14 @@ These are the terms the docs, UI, and API use consistently. Foundational concept **Firmware**: BIOS or system firmware required for certain emulators (PS1, GBA, Saturn, etc.). Lives wherever the [`filesystem.structure.firmware`](configuration-file.md#filesystemstructure) template points, `bios/{platform}` by default. Uploaded via the UI and managed by admins and users with the `firmware.write` scope (see [Firmware Management](../administration/firmware-management.md)). -**Full image**: the default container variant, bundling all four browser runtimes (EmulatorJS, Ruffle, `js-dos`, FAKE-08). `rommapp/romm:X.Y.Z` (see [Image Variants](../install/image-variants.md)). +**Full image**: the default container variant, bundling all five browser runtimes (EmulatorJS, EasyRPG, Ruffle, `js-dos`, FAKE-08). `rommapp/romm:X.Y.Z` (see [Image Variants](../install/image-variants.md)). **Game Data tab**: the ROM detail page tab for saves, states, and screenshots. User-specific. **gamelist.xml**: ES-DE/Batocera-compatible metadata format. Importable as a metadata source and exportable. +**Install request**: a game queued for download on one of your devices, which the device picks up the next time it's online (see [Devices](../using/devices.md)). + **Invite link**: single-use URL that lets a new user register with a pre-assigned role (see [Invitations & Registration](../administration/invitations-and-registration.md)). **Jukebox**: the library-wide soundtrack player, reading the audio files in each game's `soundtrack/` folder (see [Jukebox](../using/jukebox.md)). @@ -57,6 +69,8 @@ These are the terms the docs, UI, and API use consistently. Foundational concept **Netplay**: EmulatorJS's multiplayer mode. Two or more players share a session across the internet. Open rooms are tracked and brokered via WebSocket. Needs STUN/TURN (ICE servers) configured in `config.yml` for reliable NAT traversal (see [Netplay](../using/netplay.md)). +**Notification channel**: a per-user destination that notifications are forwarded to, such as a webhook, an email address or an Apprise service, filtered by level and topic (see [Notifications](../using/notifications.md)). + **OIDC**: OpenID Connect. Supported SSO protocol for external auth (see [OIDC Setup](../administration/oidc/index.md)). **OpenTelemetry (OTEL)**: opt-in traces/metrics/logs export. `OTEL_ENABLED=true` (see [Observability](../administration/observability.md)). @@ -79,6 +93,8 @@ These are the terms the docs, UI, and API use consistently. Foundational concept **Role**: an account's top-level type. Two roles: User (access governed by a permission group) and Admin, which has full access and bypasses groups (see [Users & Roles](../administration/users-and-roles.md)). +**rom-converto**: the conversion tool behind the Convert library task and `?format=` downloads, shipped in both images (see [Library Conversion](../administration/library-conversion.md)). + **Ruffle**: the bundled in-browser Flash/Shockwave emulator (see [In-Browser Play → Ruffle](../using/in-browser-play/ruffle.md)). **Scan**: the process of walking the library, hashing files, calling metadata providers, and updating the DB. Scans run in six modes (New Platforms, Quick, Unmatched, Update, Hashes, Complete) and can be triggered manually, on a cron, or by the filesystem watcher (see [Scanning & Watcher](../administration/scanning-and-watcher.md)). @@ -91,7 +107,7 @@ These are the terms the docs, UI, and API use consistently. Foundational concept **Smart Collection**: rule-based auto-populating collection (see [Smart Collections](../using/smart-collections.md)). -**socket.io**: the WebSocket protocol. Two endpoints: `/ws` and `/netplay` (see [WebSockets](../developers/websockets.md)). +**socket.io**: the WebSocket protocol. Two endpoints, `/ws` (with its `/devices` namespace for companion apps) and `/netplay` (see [WebSockets](../developers/websockets.md)). **Task**: a unit of background work (scan, metadata sync, cleanup, device sync). Runs through RQ. Can be scheduled (cron), watcher-triggered, or manual (see [Scheduled Tasks](../administration/scheduled-tasks.md)). @@ -99,6 +115,8 @@ These are the terms the docs, UI, and API use consistently. Foundational concept **Title id**: the platform-native identifier read out of a ROM's own binary during a scan, on the platforms that have one. Identifies a game where RomM doesn't hash, and says where the game writes its saves (see [Scanning & Watcher](../administration/scanning-and-watcher.md#title-ids-read-from-the-binary)). +**Trusted proxy**: a proxy allowed to report the client's address in `X-Forwarded-For`, listed in `FORWARDED_ALLOW_IPS` (see [Reverse Proxy](../install/reverse-proxy.md)). + **User**: an account. Its role is either User (access from a permission group plus per-user overrides) or Admin (full access). Can be created by the Setup Wizard, an admin, an invite link, or OIDC auto-provisioning (see [Users & Roles](../administration/users-and-roles.md)). **Valkey**: open-source Redis fork, drop-in compatible (see [Redis or Valkey](../install/redis-or-valkey.md)). diff --git a/docs/resources/snippets/env-vars.md b/docs/resources/snippets/env-vars.md index 0afa19fa..ed2b59fb 100644 --- a/docs/resources/snippets/env-vars.md +++ b/docs/resources/snippets/env-vars.md @@ -38,42 +38,53 @@ ### Authentication -| Variable | Default | Required | Description | -| ------------------------------------ | --------- | :------: | -------------------------------------------------------------------- | -| `ROMM_AUTH_SECRET_KEY` | | `✓` | App secret, generate with `openssl rand -hex 32` | -| `OAUTH_ACCESS_TOKEN_EXPIRE_SECONDS` | `1800` | | Access token lifetime in seconds | -| `OAUTH_REFRESH_TOKEN_EXPIRE_SECONDS` | `604800` | | Refresh token lifetime in seconds | -| `SESSION_MAX_AGE_SECONDS` | `1209600` | | Maximum age of a session in seconds | -| `INVITE_TOKEN_EXPIRY_SECONDS` | `600` | | Invite token lifetime in seconds | -| `DISABLE_DOWNLOAD_ENDPOINT_AUTH` | `false` | | Disable auth on the download endpoint for WebRcade/Tinfoil | -| `DISABLE_CSRF_PROTECTION` | `false` | | Disable CSRF protection (not recommended) | -| `DISABLE_USERPASS_LOGIN` | `false` | | Disable username/password login when using OIDC | -| `DISABLE_SETUP_WIZARD` | `false` | | Skip the first-boot setup wizard | -| `DISABLE_LOGS_VIEWER` | `false` | | Disable the backend logs viewer | -| `ROMM_CORS_ALLOWED_ORIGINS` | | | Comma-separated list of allowed CORS origins (empty allows all) | -| `ROMM_SESSION_SECURE_COOKIE` | `false` | | Mark session and CSRF cookies Secure (enable when served over HTTPS) | +| Variable | Default | Required | Description | +| ------------------------------------ | --------- | :------: | -------------------------------------------------------------------------------------------------------------------- | +| `ROMM_AUTH_SECRET_KEY` | | `✓` | App secret, generate with `openssl rand -hex 32` | +| `OAUTH_ACCESS_TOKEN_EXPIRE_SECONDS` | `1800` | | Access token lifetime in seconds | +| `OAUTH_REFRESH_TOKEN_EXPIRE_SECONDS` | `604800` | | Refresh token lifetime in seconds | +| `SESSION_MAX_AGE_SECONDS` | `1209600` | | Maximum age of a session in seconds | +| `INVITE_TOKEN_EXPIRY_SECONDS` | `600` | | Invite token lifetime in seconds | +| `DISABLE_DOWNLOAD_ENDPOINT_AUTH` | `false` | | Disable auth on the download endpoint for WebRcade/Tinfoil | +| `DISABLE_CSRF_PROTECTION` | `false` | | Disable CSRF protection (not recommended) | +| `DISABLE_USERPASS_LOGIN` | `false` | | Disable username/password login when using OIDC | +| `DISABLE_SETUP_WIZARD` | `false` | | Skip the first-boot setup wizard | +| `DISABLE_LOGS_VIEWER` | `false` | | Disable the backend logs viewer | +| `ROMM_CORS_ALLOWED_ORIGINS` | | | Comma-separated list of allowed CORS origins (empty allows none; a wildcard allows any origin but never credentials) | +| `ROMM_SESSION_SECURE_COOKIE` | `false` | | Mark session and CSRF cookies Secure (enable when served over HTTPS) | + +### Email, for notification channels and password reset links (off unless SMTP_HOST and SMTP_FROM are set) + +| Variable | Default | Required | Description | +| --------------- | ---------- | :------: | ---------------------------------------------------------------------------------------- | +| `SMTP_HOST` | | | SMTP server host | +| `SMTP_PORT` | `587` | | SMTP server port | +| `SMTP_USERNAME` | | | Login for the SMTP server, if it needs one | +| `SMTP_PASSWORD` | | | Password for the SMTP server | +| `SMTP_FROM` | | | Sender address, such as romm@example.com | +| `SMTP_SECURITY` | `starttls` | | starttls, tls (implicit TLS, usually port 465) or none; any other value leaves email off | ### OpenID Connect -| Variable | Default | Required | Description | -| ----------------------------- | -------------------- | :------: | ---------------------------------------------------------------------------------- | -| `OIDC_ENABLED` | `false` | | Enable OpenID Connect authentication | -| `OIDC_AUTOLOGIN` | `false` | | Skip the OIDC button on the login page and auto-redirect | -| `OIDC_ALLOW_REGISTRATION` | `true` | | Allow new accounts to be created automatically on first OIDC login | -| `OIDC_PROVIDER` | | | Name of the OIDC provider in use | -| `OIDC_CLIENT_ID` | | | Client ID for OIDC authentication | -| `OIDC_CLIENT_SECRET` | | | Client secret for OIDC authentication | -| `OIDC_REDIRECT_URI` | | | Absolute redirect URI for OIDC authentication | -| `OIDC_SERVER_APPLICATION_URL` | | | Absolute URL of the OIDC server application | -| `OIDC_SERVER_METADATA_URL` | | | URL to the OIDC provider metadata endpoint | -| `OIDC_CLAIM_ROLES` | | | OIDC claim containing user roles | -| `OIDC_ROLE_VIEWER` | | | Role value mapping to viewer permissions | -| `OIDC_ROLE_EDITOR` | | | Role value mapping to editor permissions | -| `OIDC_ROLE_ADMIN` | | | Role value mapping to admin permissions | -| `OIDC_TLS_CACERTFILE` | | | CA bundle file (PEM, DER or `.p7b`) or directory, trusted alongside the system CAs | -| `OIDC_USERNAME_ATTRIBUTE` | `preferred_username` | | Attribute on OIDC user info used as the username | -| `OIDC_RP_INITIATED_LOGOUT` | `false` | | Enable RP-initiated logout flow | -| `OIDC_END_SESSION_ENDPOINT` | | | OIDC end-session endpoint override URL | +| Variable | Default | Required | Description | +| ----------------------------- | -------------------- | :------: | -------------------------------------------------------------------------------- | +| `OIDC_ENABLED` | `false` | | Enable OpenID Connect authentication | +| `OIDC_AUTOLOGIN` | `false` | | Skip the OIDC button on the login page and auto-redirect | +| `OIDC_ALLOW_REGISTRATION` | `true` | | Allow new accounts to be created automatically on first OIDC login | +| `OIDC_PROVIDER` | | | Name of the OIDC provider in use | +| `OIDC_CLIENT_ID` | | | Client ID for OIDC authentication | +| `OIDC_CLIENT_SECRET` | | | Client secret for OIDC authentication | +| `OIDC_REDIRECT_URI` | | | Absolute redirect URI for OIDC authentication | +| `OIDC_SERVER_APPLICATION_URL` | | | Absolute URL of the OIDC server application | +| `OIDC_SERVER_METADATA_URL` | | | URL to the OIDC provider metadata endpoint | +| `OIDC_CLAIM_ROLES` | | | OIDC claim containing user roles | +| `OIDC_ROLE_VIEWER` | | | Role value mapping to viewer permissions | +| `OIDC_ROLE_EDITOR` | | | Role value mapping to editor permissions | +| `OIDC_ROLE_ADMIN` | | | Role value mapping to admin permissions | +| `OIDC_TLS_CACERTFILE` | | | CA bundle file (PEM, DER or .p7b) or directory, trusted alongside the system CAs | +| `OIDC_USERNAME_ATTRIBUTE` | `preferred_username` | | Attribute on OIDC user info used as the username | +| `OIDC_RP_INITIATED_LOGOUT` | `false` | | Enable RP-initiated logout flow | +| `OIDC_END_SESSION_ENDPOINT` | | | OIDC end-session endpoint override URL | ### Metadata Providers @@ -108,49 +119,81 @@ ### Scans & Tasks -| Variable | Default | Required | Description | -| -------------------------------------------------- | ------------ | :------: | ----------------------------------------------------------------------------------------- | -| `SCAN_TIMEOUT` | `14400` | | Timeout for background scan/rescan tasks in seconds | -| `SCAN_WORKERS` | `4` | | How many ROMs a scan processes at once | -| `TASK_TIMEOUT` | `300` | | Timeout for other background tasks in seconds | -| `TASK_RESULT_TTL` | `86400` | | How long to keep task results in Valkey in seconds | -| `SEVEN_ZIP_TIMEOUT` | `180` | | Timeout for 7-Zip operations in seconds | -| `ENABLE_RESCAN_ON_FILESYSTEM_CHANGE` | `false` | | Re-scan the library automatically when the filesystem changes | -| `RESCAN_ON_FILESYSTEM_CHANGE_DELAY` | `5` | | Delay in minutes before re-scanning after a filesystem change | -| `ENABLE_SCHEDULED_RESCAN` | `false` | | Enable scheduled library re-scans | -| `SCHEDULED_RESCAN_CRON` | `0 3 * * *` | | Cron expression for scheduled re-scans | -| `ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB` | `false` | | Enable scheduled Switch TitleDB index updates | -| `SCHEDULED_UPDATE_SWITCH_TITLEDB_CRON` | `0 4 * * *` | | Cron expression for scheduled Switch TitleDB updates | -| `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA` | `false` | | Enable scheduled LaunchBox metadata updates | -| `SCHEDULED_UPDATE_LAUNCHBOX_METADATA_CRON` | `0 4 * * *` | | Cron expression for scheduled LaunchBox metadata updates | -| `ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP` | `false` | | Enable scheduled conversion of images to WebP | -| `SCHEDULED_CONVERT_IMAGES_TO_WEBP_CRON` | `0 4 * * *` | | Cron expression for scheduled WebP conversion | -| `ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES` | `false` | | Enable scheduled cleanup of orphaned resources (covers, screenshots) left by deleted ROMs | -| `SCHEDULED_CLEANUP_ORPHANED_RESOURCES_CRON` | `0 5 * * *` | | Cron expression for scheduled orphaned resource cleanup | -| `ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC` | `false` | | Enable scheduled RetroAchievements progress sync | -| `SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC_CRON` | `0 4 * * *` | | Cron expression for scheduled RetroAchievements sync | -| `ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS` | `true` | | Enable the scheduled rebuild of the recommendations index | -| `SCHEDULED_BUILD_RECOMMENDATIONS_CRON` | `30 5 * * *` | | Cron expression for the recommendations index rebuild | +| Variable | Default | Required | Description | +| -------------------------------------------------- | -------------- | :------: | ---------------------------------------------------------------------------------------------------------------------- | +| `SCAN_TIMEOUT` | `14400` | | Timeout for background scan/rescan tasks in seconds | +| `SCAN_WORKERS` | `4` | | How many ROMs a scan processes at once | +| `TASK_TIMEOUT` | `300` | | Timeout for other background tasks in seconds | +| `TASK_RESULT_TTL` | `86400` | | How long to keep task results in Valkey in seconds | +| `SEVEN_ZIP_TIMEOUT` | `180` | | Timeout for 7-Zip operations in seconds | +| `ENABLE_RESCAN_ON_FILESYSTEM_CHANGE` | `false` | | Re-scan the library automatically when the filesystem changes | +| `RESCAN_ON_FILESYSTEM_CHANGE_DELAY` | `5` | | Delay in minutes before re-scanning after a filesystem change | +| `ENABLE_SCHEDULED_RESCAN` | `false` | | Enable scheduled library re-scans | +| `SCHEDULED_RESCAN_CRON` | `0 3 * * *` | | Cron expression for scheduled re-scans | +| `ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB` | `false` | | Enable scheduled Switch TitleDB index updates | +| `SCHEDULED_UPDATE_SWITCH_TITLEDB_CRON` | `0 4 * * *` | | Cron expression for scheduled Switch TitleDB updates | +| `ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA` | `false` | | Enable scheduled LaunchBox metadata updates | +| `SCHEDULED_UPDATE_LAUNCHBOX_METADATA_CRON` | `0 4 * * *` | | Cron expression for scheduled LaunchBox metadata updates | +| `ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP` | `false` | | Enable scheduled conversion of images to WebP | +| `SCHEDULED_CONVERT_IMAGES_TO_WEBP_CRON` | `0 4 * * *` | | Cron expression for scheduled WebP conversion | +| `ENABLE_SCHEDULED_CLEANUP_NETPLAY` | `true` | | Enable scheduled cleanup of empty netplay rooms | +| `SCHEDULED_CLEANUP_NETPLAY_CRON` | `*/30 * * * *` | | Cron expression for scheduled netplay room cleanup | +| `ENABLE_SCHEDULED_CLEANUP_UPLOAD_TMP` | `true` | | Enable scheduled cleanup of abandoned chunked uploads; when disabled, expired temp directories must be removed by hand | +| `SCHEDULED_CLEANUP_UPLOAD_TMP_CRON` | `0 * * * *` | | Cron expression for scheduled upload temp cleanup | +| `ENABLE_SCHEDULED_CLEANUP_ZIP_CACHE` | `true` | | Enable scheduled cleanup of stale cached ZIP files | +| `SCHEDULED_CLEANUP_ZIP_CACHE_CRON` | `0 4 * * *` | | Cron expression for scheduled ZIP cache cleanup | +| `ENABLE_SCHEDULED_CLEANUP_SYNC_SESSIONS` | `true` | | Enable scheduled failing of sync sessions no client completed | +| `SCHEDULED_CLEANUP_SYNC_SESSIONS_CRON` | `23 * * * *` | | Cron expression for scheduled sync session cleanup | +| `ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES` | `false` | | Enable scheduled cleanup of orphaned resources (covers, screenshots) left by deleted ROMs | +| `SCHEDULED_CLEANUP_ORPHANED_RESOURCES_CRON` | `0 5 * * *` | | Cron expression for scheduled orphaned resource cleanup | +| `ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC` | `false` | | Enable scheduled RetroAchievements progress sync | +| `SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC_CRON` | `0 4 * * *` | | Cron expression for scheduled RetroAchievements sync | +| `ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS` | `true` | | Enable the scheduled rebuild of the recommendations index | +| `SCHEDULED_BUILD_RECOMMENDATIONS_CRON` | `30 5 * * *` | | Cron expression for the recommendations index rebuild | + +### Audit log + +| Variable | Default | Required | Description | +| -------------------------- | ------- | :------: | --------------------------------------------------------------------------------------------------- | +| `AUDIT_LOG_RETENTION_DAYS` | `90` | | Days to keep audit log events (plays, downloads, library and account changes); 0 keeps them forever | ### Sync -| Variable | Default | Required | Description | -| ---------------------------- | -------------- | :------: | ------------------------------------------------------------------------- | -| `ENABLE_SYNC_FOLDER_WATCHER` | `false` | | Watch the sync folder and trigger scans on change | -| `SYNC_FOLDER_SCAN_DELAY` | `2` | | Delay in minutes before scanning after a sync folder change | -| `ENABLE_SYNC_PUSH_PULL` | `false` | | Enable scheduled sync push/pull | -| `SYNC_PUSH_PULL_CRON` | `*/30 * * * *` | | Cron expression for scheduled sync push/pull | -| `SYNC_SSH_KEYS_PATH` | | | Path to SSH keys for sync remotes (defaults to $ROMM_BASE_PATH/sync/keys) | -| `SYNC_SSH_KNOWN_HOSTS_PATH` | | | Path to SSH known_hosts (defaults to $ROMM_BASE_PATH/sync/known_hosts) | +| Variable | Default | Required | Description | +| ------------------------------- | -------------- | :------: | --------------------------------------------------------------------------------------------------------- | +| `ENABLE_SYNC_FOLDER_WATCHER` | `false` | | Watch the sync folder and trigger scans on change | +| `SYNC_FOLDER_SCAN_DELAY` | `2` | | Delay in minutes before scanning after a sync folder change | +| `ENABLE_SYNC_PUSH_PULL` | `false` | | Enable scheduled sync push/pull | +| `SYNC_PUSH_PULL_CRON` | `*/30 * * * *` | | Cron expression for scheduled sync push/pull | +| `SYNC_SSH_KEYS_PATH` | | | Path to SSH keys for sync remotes (defaults to $ROMM_BASE_PATH/sync/keys) | +| `SYNC_SSH_KNOWN_HOSTS_PATH` | | | Path to SSH known_hosts (defaults to $ROMM_BASE_PATH/sync/known_hosts) | +| `SYNC_RETROARCH_PSP_SERIAL_MAP` | | | JSON map of PSP serial to extensionless ROM file name, for RetroArch PSP saves whose title matches no ROM | + +### Device install + +| Variable | Default | Required | Description | +| ---------------------------------------- | ------------------------------ | :------: | ---------------------------------------------------------------------------------------------- | +| `DEVICE_INSTALL_ENABLED` | `true` | | Let users push a ROM to one of their registered devices for install | +| `DEVICE_INSTALL_REQUEST_TTL_DAYS` | `2` | | Days an unfinished install request waits for its device after its last change; 0 waits forever | +| `DEVICE_INSTALL_EXCLUDED_PLATFORM_SLUGS` | `win,win3x,win9x,windows-apps` | | Comma-separated platform slugs that can never be pushed to a device | ### Emulation -| Variable | Default | Required | Description | -| --------------------- | ------- | :------: | -------------------------------------------------------- | -| `DISABLE_EMULATOR_JS` | `false` | | Disable in-browser play via EmulatorJS | -| `DISABLE_RUFFLE_RS` | `false` | | Disable in-browser Flash playback via RuffleRS | -| `DISABLE_JSDOS` | `false` | | Disable in-browser Win3.x and Win9.x playback via js-dos | -| `DISABLE_PICO8` | `false` | | Disable in-browser PICO-8 playback via FAKE-08 | +| Variable | Default | Required | Description | +| --------------------- | ------- | :------: | ----------------------------------------------------------- | +| `DISABLE_EMULATOR_JS` | `false` | | Disable in-browser play via EmulatorJS | +| `DISABLE_RUFFLE_RS` | `false` | | Disable in-browser Flash playback via RuffleRS | +| `DISABLE_JSDOS` | `false` | | Disable in-browser Win3.x and Win9.x playback via js-dos | +| `DISABLE_PICO8` | `false` | | Disable in-browser PICO-8 playback via FAKE-08 | +| `DISABLE_EASYRPG` | `false` | | Disable in-browser RPG Maker 2000/2003 playback via EasyRPG | + +### ROM conversion + +| Variable | Default | Required | Description | +| ------------------------------ | ------- | :------: | ------------------------------------------------------------------------------------ | +| `ROM_CONVERTO_ENABLED` | `false` | | Enable rom-converto for the "Convert library" task and ?format= download conversions | +| `ROM_CONVERTO_TIMEOUT` | `600` | | Timeout for each rom-converto operation in seconds | +| `ROM_CONVERTO_MAX_CONCURRENCY` | `2` | | Concurrent conversions per web or task worker | ### Integrations @@ -177,16 +220,17 @@ ### Web Server -| Variable | Default | Required | Description | -| ---------------------------------- | ------- | :------: | ---------------------------------------------------------------------------------------- | -| `WEB_SERVER_CONCURRENCY` | `4` | | Number of API worker processes, raise for several concurrent users | -| `WEB_SERVER_TIMEOUT` | `300` | | Timeout for web server requests in seconds | -| `WEB_SERVER_KEEPALIVE` | `65` | | Keep-Alive connection wait time in seconds, keep above nginx's 60s upstream idle timeout | -| `WEB_SERVER_MAX_REQUESTS` | `1000` | | Maximum requests a worker processes before restarting | -| `WEB_SERVER_MAX_REQUESTS_JITTER` | `100` | | Random jitter added to max requests value | -| `WEB_SERVER_WORKER_CONNECTIONS` | `1000` | | Maximum simultaneous clients per worker process | -| `WEB_SERVER_GUNICORN_WAIT_SECONDS` | `30` | | Seconds to wait for Gunicorn to start before giving up | -| `IPV4_ONLY` | `false` | | Bind only to IPv4 | +| Variable | Default | Required | Description | +| ---------------------------------- | ------------------------------------------------------------------------------ | :------: | -------------------------------------------------------------------------------------------------------------------------- | +| `WEB_SERVER_CONCURRENCY` | `4` | | Number of API worker processes, raise for several concurrent users | +| `WEB_SERVER_TIMEOUT` | `300` | | Timeout for web server requests in seconds | +| `WEB_SERVER_KEEPALIVE` | `65` | | Keep-Alive connection wait time in seconds, keep above nginx's 60s upstream idle timeout | +| `WEB_SERVER_MAX_REQUESTS` | `1000` | | Maximum requests a worker processes before restarting | +| `WEB_SERVER_MAX_REQUESTS_JITTER` | `100` | | Random jitter added to max requests value | +| `WEB_SERVER_WORKER_CONNECTIONS` | `1000` | | Maximum simultaneous clients per worker process | +| `WEB_SERVER_GUNICORN_WAIT_SECONDS` | `30` | | Seconds to wait for Gunicorn to start before giving up | +| `FORWARDED_ALLOW_IPS` | `127.0.0.1,::1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10,fc00::/7` | | Proxies trusted to report the client's address (X-Forwarded-For); add a reverse proxy's public IP here, or \* to trust any | +| `IPV4_ONLY` | `false` | | Bind only to IPv4 | ### Proxy @@ -204,18 +248,19 @@ ### Development -| Variable | Default | Required | Description | -| ------------------------------ | ----------- | :------: | -------------------------------------------------- | -| `DEV_MODE` | `false` | | Enable development mode (debugging, hot-reloading) | -| `DEV_HOST` | `127.0.0.1` | | Host for the development server | -| `DEV_PORT` | `5000` | | Port for the development server | -| `DEV_HTTPS` | `false` | | Enable HTTPS in the development server | -| `DEV_SQL_ECHO` | `false` | | Log all SQL queries in development mode | -| `POSTGRES_DB` | `authentik` | | Postgres database name for the Authentik dev stack | -| `POSTGRES_USER` | `authentik` | | Postgres user for the Authentik dev stack | -| `POSTGRES_PASSWORD` | `authentik` | | Postgres password for the Authentik dev stack | -| `AUTHENTIK_SECRET_KEY` | | | Authentik secret key | -| `AUTHENTIK_BOOTSTRAP_PASSWORD` | | | Initial Authentik admin bootstrap password | +| Variable | Default | Required | Description | +| ------------------------------ | ----------- | :------: | ------------------------------------------------------------------- | +| `DEV_MODE` | `false` | | Enable development mode (debugging, hot-reloading) | +| `DEV_HOST` | `127.0.0.1` | | Host for the development server | +| `DEV_PORT` | `5000` | | Port for the development server | +| `DEV_PROXY_TARGET` | | | Optional RomM origin (e.g. https://romm.example.com) for vite proxy | +| `DEV_HTTPS` | `false` | | Enable HTTPS in the development server | +| `DEV_SQL_ECHO` | `false` | | Log all SQL queries in development mode | +| `POSTGRES_DB` | `authentik` | | Postgres database name for the Authentik dev stack | +| `POSTGRES_USER` | `authentik` | | Postgres user for the Authentik dev stack | +| `POSTGRES_PASSWORD` | `authentik` | | Postgres password for the Authentik dev stack | +| `AUTHENTIK_SECRET_KEY` | | | Authentik secret key | +| `AUTHENTIK_BOOTSTRAP_PASSWORD` | | | Initial Authentik admin bootstrap password | ### Emulator Streaming diff --git a/docs/resources/snippets/scheduled-tasks.md b/docs/resources/snippets/scheduled-tasks.md index b920895a..775b52c5 100644 --- a/docs/resources/snippets/scheduled-tasks.md +++ b/docs/resources/snippets/scheduled-tasks.md @@ -7,15 +7,20 @@ | Scheduled Switch TitleDB update | Scheduled | `0 4 * * *` | `ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB` | `SCHEDULED_UPDATE_SWITCH_TITLEDB_CRON` | Updates the Nintendo Switch TitleDB file. | | Build recommendations index | Scheduled | `30 5 * * *` | `ENABLE_SCHEDULED_BUILD_RECOMMENDATIONS` | `SCHEDULED_BUILD_RECOMMENDATIONS_CRON` | Rebuilds the similar-games index from library metadata, play history and collections. | | Convert images to WebP | Scheduled | `0 4 * * *` | `ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP` | `SCHEDULED_CONVERT_IMAGES_TO_WEBP_CRON` | Convert existing image files (PNG, JPG, BMP, TIFF, GIF) to WebP format for better performance. | -| Scheduled ZIP cache cleanup | Scheduled | `0 4 * * *` | `-` | `-` | Removes stale cached ZIP files based on tiered TTL. Always on, not configurable. | +| Scheduled ZIP cache cleanup | Scheduled | `0 4 * * *` | `ENABLE_SCHEDULED_CLEANUP_ZIP_CACHE` | `SCHEDULED_CLEANUP_ZIP_CACHE_CRON` | Removes stale cached ZIP files based on tiered TTL. | +| Scheduled conversion cache cleanup | Scheduled | `0 4 * * *` | `-` | `-` | Removes stale converted download files based on TTL. Always on, not configurable. | | Cleanup orphaned resources | Scheduled | `0 5 * * *` | `ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES` | `SCHEDULED_CLEANUP_ORPHANED_RESOURCES_CRON` | Clean up orphaned resources in the ROMs directory. | -| Scheduled netplay cleanup | Scheduled | `*/30 * * * *` | `-` | `-` | Cleans up empty netplay rooms. Always on, not configurable. | -| Scheduled upload tmp cleanup | Scheduled | `0 * * * *` | `-` | `-` | Cleans up orphaned chunked-upload temp directories. Always on, not configurable. | +| Scheduled netplay cleanup | Scheduled | `*/30 * * * *` | `ENABLE_SCHEDULED_CLEANUP_NETPLAY` | `SCHEDULED_CLEANUP_NETPLAY_CRON` | Cleans up empty netplay rooms. | +| Scheduled upload tmp cleanup | Scheduled | `0 * * * *` | `ENABLE_SCHEDULED_CLEANUP_UPLOAD_TMP` | `SCHEDULED_CLEANUP_UPLOAD_TMP_CRON` | Cleans up orphaned chunked-upload temp directories. | +| Scheduled streaming session reaper | Scheduled | `* * * * *` | `-` | `-` | Stops streaming sessions whose player stopped sending heartbeats. Runs only while emulator streaming is enabled. | +| Scheduled sync session cleanup | Scheduled | `23 * * * *` | `ENABLE_SCHEDULED_CLEANUP_SYNC_SESSIONS` | `SCHEDULED_CLEANUP_SYNC_SESSIONS_CRON` | Fails sync sessions no client ever completed. | +| Scheduled audit log cleanup | Scheduled | `30 4 * * *` | `AUDIT_LOG_RETENTION_DAYS` | `-` | Removes audit log events older than `AUDIT_LOG_RETENTION_DAYS` days. Runs while `AUDIT_LOG_RETENTION_DAYS > 0`. | | Scheduled RetroAchievements progress sync | Scheduled | `0 4 * * *` | `ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC` | `SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC_CRON` | Updates RetroAchievements progress for all users. | | Push-Pull Sync | Scheduled | `*/30 * * * *` | `ENABLE_SYNC_PUSH_PULL` | `SYNC_PUSH_PULL_CRON` | Sync saves with devices via SSH/SFTP. | | Cleanup missing ROMs | Manual | `-` | `-` | `-` | Delete all ROMs flagged as missing from the filesystem from the database. | | Cleanup missing firmware | Manual | `-` | `-` | `-` | Delete all firmware flagged as missing from the filesystem from the database. | | Sync Folder Scan | Manual | `-` | `ENABLE_SYNC_FOLDER_WATCHER` | `-` | Scan device sync folders for new save files. | | Recompute save content hashes | Manual | `-` | `-` | `-` | Re-scan every save row and rewrite content_hash with the current compute_content_hash algorithm. One-time recovery after the zip-hash dispatch fix. | +| Convert library | Manual | `-` | `ROM_CONVERTO_ENABLED` | `-` | Convert each matched ROM to its platform's library format, replacing the original files. | | Filesystem watcher | Watcher | `-` | `ENABLE_RESCAN_ON_FILESYSTEM_CHANGE` | `RESCAN_ON_FILESYSTEM_CHANGE_DELAY` | Watch the library folder and trigger a rescan on changes. | | Sync folder watcher | Watcher | `-` | `ENABLE_SYNC_FOLDER_WATCHER` | `SYNC_FOLDER_SCAN_DELAY` | Watch the sync folder and trigger a scan on changes. | diff --git a/docs/resources/snippets/supported-platforms.md b/docs/resources/snippets/supported-platforms.md index 428e5d10..f4cbf6e7 100644 --- a/docs/resources/snippets/supported-platforms.md +++ b/docs/resources/snippets/supported-platforms.md @@ -286,7 +286,7 @@ | Pandora | `pandora` | mobygames logo | | PC Booter | `pc-booter` | mobygames logo | | PC Engine SuperGrafx | `supergrafx` | igdb logo screenscraper logo mobygames logo launchbox logo retroachivements logo libretro logo | -| PC Win9X | `win9x` | screenscraper logo | +| PC Win9X | `win9x` | screenscraper logo | | PC-50X Family | `pc-50x-family` | igdb logo | | PC-6001 | `pc-6001` | mobygames logo | | PC-8000 | `pc-8000` | mobygames logo | diff --git a/docs/troubleshooting/authentication.md b/docs/troubleshooting/authentication.md index 7bada4cc..55349fef 100644 --- a/docs/troubleshooting/authentication.md +++ b/docs/troubleshooting/authentication.md @@ -5,14 +5,23 @@ description: Fix login, session, CSRF, and OIDC issues # Authentication Troubleshooting -## `403 Forbidden` on API calls +## `401 Unauthorized` or `403 Forbidden` on API calls -When auth is enabled (almost always), any endpoint that requires a session returns `403` if: +- **`401 Unauthorized`**: the request carries no usable credential, because you're signed out, your session expired or was revoked, or the `Authorization` header is malformed, has the wrong password, or holds an expired token. Sign in again, or refresh or replace the token. +- **`403 Forbidden`**: you're signed in, but your account or token lacks the scope the endpoint needs. Check the user's permission group and the token's scopes (see [API Authentication → Errors](../developers/api-authentication.md#errors)). -- You're not authenticated. -- Your session is in a broken state (expired, signed with an old secret, missing CSRF). +If a browser session seems broken in a way signing in again doesn't fix (for example a session signed with an old `ROMM_AUTH_SECRET_KEY`), [clear cookies](https://support.google.com/accounts/answer/32050) for the host and sign in again. -Fix: [clear cookies](https://support.google.com/accounts/answer/32050) for the host and sign in again. +## Browser app on another origin is blocked by CORS + +The browser console shows `blocked by CORS policy` or `No 'Access-Control-Allow-Origin' header`. An empty `ROMM_CORS_ALLOWED_ORIGINS` denies every cross-origin request, so list the app's origin: + +```yaml +environment: + - ROMM_CORS_ALLOWED_ORIGINS=https://dashboard.example.com +``` + +Write it as scheme, host and port, exactly as the browser shows it in the `Origin` request header, with no trailing slash. A `*` entry won't fix an app that signs in with the session cookie, because a `*` anywhere in the list turns credentials off for every origin, listed ones included (see [Reverse Proxy → Cookies and CORS](../install/reverse-proxy.md#harden-cookies-and-cors-behind-https)). ## `Forbidden (403) CSRF verification failed` @@ -33,6 +42,8 @@ Your reverse proxy is stripping the WebSocket upgrade, and live updates (scan pr - **Caddy**: WebSockets work out of the box with `reverse_proxy`. - **Cloudflare**: enable **WebSockets** under Network settings. +The main UI falls back to HTTP long polling when the upgrade fails, but that only works with a single gunicorn worker (`WEB_SERVER_CONCURRENCY=1`), so with the default of 4 live updates break too. [Netplay](../using/netplay.md) connects over WebSockets only and fails outright. + ## `Error: Could not get twitch auth token: check client_id and client_secret` IGDB creds are wrong or revoked on the Twitch side. @@ -123,6 +134,22 @@ If the new provider user really is the account's owner, clear the stored link in UPDATE users SET oidc_issuer = NULL, oidc_sub = NULL WHERE username = 'alice'; ``` +### Signing in at the provider lands back on the RomM login page + +You end up on `/login?bypass_autologin=true` instead of signed in when RomM rejects the provider's callback, for example because the provider returned an error, the authorization code was already used or expired, or the ID token failed validation. The `bypass_autologin` flag keeps `OIDC_AUTOLOGIN` from sending you straight back into the same failure. + +The container log names the reason on an `OIDC callback rejected` line: + +```sh +docker logs romm 2>&1 | grep -i "OIDC callback rejected" +``` + +Common causes are clock drift (see below), a client secret that changed at the provider, or a provider that expects a different redirect URI. + +### `certificate verify failed` when RomM talks to the provider + +Your provider's certificate is signed by a private CA that the container doesn't trust. Mount the CA certificate and point `OIDC_TLS_CACERTFILE` at it (see [OIDC Setup → Private certificate authority](../administration/oidc/index.md#private-certificate-authority)). It's trusted alongside the system CAs, so public providers keep working. + ### `OAuthException: expired token` on callback Your host and the IdP have significant clock drift, so run NTP on both. @@ -137,7 +164,7 @@ This usually happens because something else in the chain (a CSRF check, a cookie 2. Sign in as a local admin. 3. Disable `OIDC_AUTOLOGIN`, restart, and debug the IdP config with autologin off. -If `bypass_autologin` doesn't work in your version, shell into the container and unset `OIDC_AUTOLOGIN` in the env, or edit your compose and restart. +A callback the provider rejects, or whose code exchange or ID token fails validation, already redirects to `/login?bypass_autologin=true`, so a loop usually means the callback succeeds but the session doesn't stick, which points at cookies or the proxy. If `bypass_autologin` doesn't work in your version, shell into the container and unset `OIDC_AUTOLOGIN` in the env, or edit your compose and restart. ## Still stuck? diff --git a/docs/troubleshooting/in-browser-play.md b/docs/troubleshooting/in-browser-play.md index a65ff30f..bfbc4dbc 100644 --- a/docs/troubleshooting/in-browser-play.md +++ b/docs/troubleshooting/in-browser-play.md @@ -1,13 +1,13 @@ --- title: In-Browser Play Troubleshooting -description: Diagnose EmulatorJS and Ruffle issues +description: Diagnose EmulatorJS, Ruffle and EasyRPG issues --- # In-Browser Play Troubleshooting ## EmulatorJS won't load at all -- **On the slim image without internet?** The slim image fetches EmulatorJS cores from a CDN at runtime rather than bundling them, so without outbound network the browser can't load games. Either switch to the full image (cores bundled) or open outbound access (see [Image Variants](../install/image-variants.md)). Ruffle and PICO-8 are full-image only. +- **On the slim image without internet?** The slim image fetches EmulatorJS cores from a CDN at runtime rather than bundling them, so without outbound network the browser can't load games. Either switch to the full image (cores bundled) or open outbound access (see [Image Variants](../install/image-variants.md)). Ruffle, PICO-8 and EasyRPG are full-image only. - Check the **browser console** and look for 404s on `/assets/emulatorjs/...`, which indicate the EmulatorJS bundle didn't install correctly in the container. Check `docker logs romm` for entrypoint install-step failures. - **Browser compatibility**: EmulatorJS uses SharedArrayBuffer, which needs a modern Chrome/Firefox/Safari and an HTTPS-served instance (cross-origin isolation requires HTTPS). If you're still on plain HTTP, set up TLS first (see [Reverse Proxy](../install/reverse-proxy.md)). @@ -57,3 +57,8 @@ See the [MS-DOS](../using/in-browser-play/ms-dos.md) page for DOS-specific notes - **Consider a native app.** [Argosy Launcher](../ecosystem/first-party-apps.md#argosy-launcher) on Android uses native emulators, which are orders of magnitude more efficient. For Netplay-specific issues: [Netplay Troubleshooting](netplay.md). + +## RPG Maker game won't start in EasyRPG + +- **No browser play offered?** The game has to be an extracted folder on the `rpg-maker` platform with `RPG_RT.ldb` at its top, since the web player can't read archives. `DISABLE_EASYRPG=true` also removes browser play, and so does the slim image, which doesn't bundle the player (see [EasyRPG](../using/in-browser-play/easyrpg.md)). +- **Missing graphics or silent sounds**, mostly in battles, come from assets the bundled free RTP doesn't have yet. Copying the missing files from the original RTP into the game's own folder and rescanning it fills the gap, because a game's files win over the RTP's. diff --git a/docs/troubleshooting/netplay.md b/docs/troubleshooting/netplay.md index 5a6450b2..2094486d 100644 --- a/docs/troubleshooting/netplay.md +++ b/docs/troubleshooting/netplay.md @@ -22,9 +22,20 @@ Full config: [Configuration File → `emulatorjs.netplay`](../reference/configur You created a room as host but other players don't see it. - **They need accounts on your instance.** Netplay doesn't federate, which means a user with no account or on a different instance can't see or join. +- **They can't see the game.** Rooms are only listed to players with `roms.read` and access to the ROM, so a game hidden from them or above their [age limit](../administration/parental-controls.md) shows no rooms. A room password doesn't get around this, because they can't load the game in the player either, so unhide the game for them or raise their age limit. - **WebSocket connection is broken.** Open devtools → Network → WS tab. If socket.io is disconnecting, see [Authentication Troubleshooting → WebSockets](authentication.md#400-bad-request-on-the-websocket-endpoint). - **Other player didn't open the Netplay panel.** They need to click the 🌐 icon on the emulator toolbar to see the room list. +## `Not authorized to open a room for this game` + +The host's account can't see the game, or lacks the `roms.read` scope. Check the user's permission group, [hidden entities](../administration/users-and-roles.md#hidden-entities) and age limit. The same message on join (`Not authorized to join this room`) means the joining player can't see the game and the room has no password. + +## Room never gets created, `Invalid session` in the logs + +Releases before 5.4 opened Netplay's socket with HTTP long polling, which breaks when gunicorn runs several workers (`WEB_SERVER_CONCURRENCY`, 4 by default). On those releases, `WEB_SERVER_CONCURRENCY=1` works around it. + +Netplay connects over WebSockets only, so the WebSocket upgrade must reach RomM. If rooms still fail, check that your proxy forwards it (see [Authentication Troubleshooting → WebSockets](authentication.md#400-bad-request-on-the-websocket-endpoint)). + ## Joined but video never appears - **Host's browser stopped streaming.** Check host's tab is still foregrounded, because browsers throttle background tabs. diff --git a/docs/using/account-and-profile.md b/docs/using/account-and-profile.md index f6b0796e..aba7b0b7 100644 --- a/docs/using/account-and-profile.md +++ b/docs/using/account-and-profile.md @@ -11,6 +11,14 @@ Every user (User or Admin) can manage their own profile, and Admins can also edi If you're an OIDC user and want to show your `preferred_username` from the token instead of your email local-part, the server owner can set `OIDC_USERNAME_ATTRIBUTE=preferred_username` (see [OIDC Setup](../administration/oidc/index.md)). +## Forgotten password + +The sign-in page can send you a password reset link, valid for 10 minutes. If the server has [email](../administration/email.md) set up and your account has an email address, the link is sent to that address. Otherwise it goes to the server log, so ask an admin for it, or to set a new password for you. A new password must be 6 to 255 ASCII characters. + +## Notifications + +Scans, tasks, role changes and other events leave a notification in your inbox. You can forward them to a webhook, an email address or (for admins) chat services like Discord and Telegram (see [Notifications](notifications.md)). + ## Client API tokens These are long-lived API tokens for companion apps, scripts, and integrations. Each token is scoped to a subset of your user's scopes, and optionally expires. diff --git a/docs/using/devices.md b/docs/using/devices.md new file mode 100644 index 00000000..ee3d59ed --- /dev/null +++ b/docs/using/devices.md @@ -0,0 +1,54 @@ +--- +title: Devices +description: The browsers, handhelds and apps your saves sync with, and installing games on them +--- + +# Devices + +A device is anything that syncs saves with RomM on your behalf: a handheld running [Grout](../ecosystem/first-party-apps.md#grout), a phone running [Argosy](../ecosystem/first-party-apps.md#argosy-launcher), [RetroArch](../ecosystem/retroarch-cloud-sync.md), or a web browser you play in. Each device belongs to one user and keeps a record of the last version of each save it synced, which is how an ordinary update is told apart from a conflict. + +Your devices are listed at `/devices`, where you can rename a device, turn sync off for it, or remove it. + +## Browsers are devices too + +All five in-browser players ([EmulatorJS](in-browser-play/emulatorjs.md), [EasyRPG](in-browser-play/easyrpg.md), [`js-dos`](in-browser-play/js-dos.md), [PICO-8](in-browser-play/pico-8.md) and [Ruffle](in-browser-play/ruffle.md)) sync saves through device sync. Each browser profile registers itself as a device the first time you play, named after the browser and OS, such as "Firefox on macOS", so a second browser, or a second profile in the same browser, is a separate device. + +When a game launches, the browser negotiates its saves with the server, the same way a handheld does at the start of a sync (see [Saves & States](saves-and-states.md#syncing-saves-between-browsers-and-devices)). A save made in one browser is waiting in another, and when both changed the same slot, the browser plays the server's copy and archives its own so neither is overwritten. + +A browser keeps its saves in local storage until they reach the server. Because that storage can be cleared or evicted, a save the browser no longer holds is treated as lost rather than deleted, and the server offers it back on the next launch. + +An account without the `devices.write` permission can't register devices, so its browsers keep their saves locally and skip device sync. + +## Managing devices + +- **Renaming** changes only its display name, for example to tell two handhelds of the same model apart. +- **Turning sync off** means that device's sync requests are refused, so nothing moves in either direction until you turn it back on, and a browser with sync off keeps its saves locally. +- **Removing** a device deletes it, its sync history and any installs queued for it, but the saves it uploaded stay on the server. A browser registers itself again the next time you play in it, and an app re-registers on its next sync, so removing is mostly for cleaning up a device you no longer use. + +## Install on device + +A game you send to one of your devices is added to its download queue and fetched the next time the device is online. + +Only devices whose app reports that it accepts installs are offered, which leaves out browsers and RetroArch. Devices also show their online status, meaning the app holds an open connection to the `/devices` socket, as a supporting app does while it runs. + +A request waits in the device's queue until the device takes it and reports back. You get a [notification](notifications.md) when it finishes or fails, and you can cancel a request any time before it finishes. A request nothing picks up expires after `DEVICE_INSTALL_REQUEST_TTL_DAYS` days without a change (2 by default). + +The device receives the game's own files, along with any update and DLC files in its folder, but not manuals or other extras. A game with none of those files on disk can't be installed. + +### Server settings + +| Variable | Default | Description | +| ---------------------------------------- | ------------------------------ | -------------------------------------------------------------------------- | +| `DEVICE_INSTALL_ENABLED` | `true` | Turn installs on or off for everyone | +| `DEVICE_INSTALL_REQUEST_TTL_DAYS` | `2` | Days an unfinished request waits after its last change (`0` waits forever) | +| `DEVICE_INSTALL_EXCLUDED_PLATFORM_SLUGS` | `win,win3x,win9x,windows-apps` | Comma-separated platform slugs that can never be sent to a device | + +Install requests live in Valkey rather than the database, so they're lost if Valkey loses its data, and devices then see an empty queue. + +Apps that accept installs follow the request lifecycle and `/devices` socket events described in the [Device Sync Protocol](../developers/device-sync-protocol.md#installs-on-devices). + +## See also + +- [Saves & States](saves-and-states.md): how they sync +- [RetroArch Cloud Sync](../ecosystem/retroarch-cloud-sync.md): saves and states over WebDAV +- [Client API Tokens](../developers/client-api-tokens.md): how apps pair with RomM diff --git a/docs/using/downloads.md b/docs/using/downloads.md index 187f3b80..a7ce0557 100644 --- a/docs/using/downloads.md +++ b/docs/using/downloads.md @@ -40,6 +40,24 @@ This makes ROM and firmware download URLs work unauthenticated. !!! danger "Only enable this behind upstream auth" This flag makes your library world-downloadable from whatever URL serves it. Only set it when you have authentication at the reverse-proxy layer (Authelia, Cloudflare Access, an IP allowlist, or a VPN). +## Downloading in another format + +When an admin turns on [download conversion](../administration/library-conversion.md#download-conversion), a single-file game can be downloaded as any format rom-converto can produce from it, such as a `.zso` or `.iso` of a PSP game stored as CHD. The web UI lists these formats as "Download as" on the game page, and the same list is in the `download_formats` field of a game's API response. + +API clients ask for one with `?format=` on the download URL, listing every format they can read in order of preference: + +```bash +curl -H "Authorization: Bearer rmm_..." \ + -OJ \ + "https://demo.romm.app/api/roms/123/content/game.chd?format=zso,iso" +``` + +The stored file is served as-is when its format is in the list. Otherwise it's converted, with a `202` and a `Retry-After` header while the conversion runs, and a `406` when none of the listed formats can be produced. Converted copies are cached, so the next download of the same format is immediate. Only signed-in users can start a conversion. + +## Downloads and player loads + +The download endpoint takes a `purpose` query parameter, `download` (the default) or `play`. In-browser players send `purpose=play` when they fetch a game to run it, so the [audit log](../administration/audit-log.md) records a player load rather than a download, though the file served is the same. + ## Nintendo 3DS direct install The 3DS built-in QR scanner can install compatible `.cia` files directly from a URL. RomM produces compatible QR codes, so a 3DS with FBI (or another CIA installer) can install over the air given network access to your instance and either basic-auth on the 3DS side or `DISABLE_DOWNLOAD_ENDPOINT_AUTH=true` behind upstream auth. diff --git a/docs/using/in-browser-play/easyrpg.md b/docs/using/in-browser-play/easyrpg.md new file mode 100644 index 00000000..d355b7e9 --- /dev/null +++ b/docs/using/in-browser-play/easyrpg.md @@ -0,0 +1,52 @@ +--- +title: EasyRPG +description: Play RPG Maker 2000 and 2003 games in the browser +--- + +# EasyRPG + +[EasyRPG Player](https://easyrpg.org/) is an open-source engine for RPG Maker 2000 and 2003 games, and its web build ships with the full container image. It plays games on the `rpg-maker` platform (an `easyrpg` folder maps to it too). + + +!!! important "Games must be extracted folders" + The web player fetches each file of a game as it needs it, so it can't read a `.zip`, `.7z` or other archive. Extract each game into its own folder under `rpg-maker/`, with `RPG_RT.ldb` at the top of that folder. + +```text +roms/rpg-maker/ +├─ Yume Nikki/ +│ ├─ RPG_RT.ldb +│ ├─ RPG_RT.lmt +│ ├─ Map0001.lmu +│ ├─ CharSet/ +│ └─ ... +└─ another game.zip ← not playable in the browser +``` + +A game is playable in EasyRPG when `RPG_RT.ldb` is in the game folder itself, which the API exposes as `is_easyrpg_game` on every ROM. Other games on the platform, archives included, can still be downloaded but don't offer browser play. + +## The RTP + +Many RPG Maker games rely on the RTP (run-time package), a shared set of graphics, sounds and music the original engine installs separately. [EasyRPG RTP](https://github.com/EasyRPG/RTP), a free replacement under CC-BY-4.0, is bundled and merged into each game, so games that need it start without you supplying one. A game's own files always win over the RTP's. + +The free RTP doesn't have every asset of the commercial ones yet, mostly ones used in battles, so a game can show a blank graphic or stay silent where it uses a missing one. Games that bundle their own copy of the assets they use aren't affected. + +## Saves + +The player keeps a game's saves in the browser while it runs, and they sync with the server through [device sync](../saves-and-states.md), the same way as the other in-browser players. Each save slot (`Save01.lsd`, `Save02.lsd` and so on) syncs as its own save, so a save made in one browser is there when you launch the game in another. They're kept per user, so two RomM accounts on the same browser keep separate slots. EasyRPG has no save states. + +## Disabling it + +Set `DISABLE_EASYRPG=true` to turn EasyRPG off for everyone, which keeps the games in your library and downloadable but without browser play. + +The slim image doesn't bundle the player, so these games need the full image (see [Image Variants](../../install/image-variants.md)). + +## API + +The player loads everything through one route, `GET /api/roms/{id}/easyrpg/{path}` (`roms.read`): + +- `index.json` returns a file index the player reads at boot, built from the game's files with the RTP merged in. +- Any other path returns that file of the game, or the RTP file standing in for it. + +It answers `404` when EasyRPG is disabled, the ROM isn't an RPG Maker 2000/2003 game folder, or the caller can't see the ROM. + +For more troubleshooting, see [In-Browser Play Troubleshooting](../../troubleshooting/in-browser-play.md). diff --git a/docs/using/index.md b/docs/using/index.md index a095d7cb..554272f5 100644 --- a/docs/using/index.md +++ b/docs/using/index.md @@ -12,6 +12,7 @@ description: Browse, play, collect, patch, and share from the UI - **[`js-dos`](in-browser-play/js-dos.md)** - **[PICO-8](in-browser-play/pico-8.md)** - **[Ruffle](in-browser-play/ruffle.md)** +- **[EasyRPG](in-browser-play/easyrpg.md)** - **[Emulator Streaming](emulator-streaming.md)** - **[Downloads](downloads.md)** - **[Uploads](uploads.md)** @@ -40,3 +41,18 @@ description: Browse, play, collect, patch, and share from the UI ## Settings - **[UI Languages](languages.md)** + +## Search + +Search matches a game's title, its file name, and every known alternative title, so a search for a Japanese title finds the game listed under its English name and the other way around. Alternative titles come from IGDB (Hasheous matches included), MobyGames and ScreenScraper, localized and regional names among them. A matched game picks up new alternative titles on its next metadata refresh. + +You can also set a game's alternative titles yourself in its manual metadata, for an abbreviation like "ACNH" or a fan translation's name. A hand-set list **replaces** the providers' titles for that game, so include any provider title you still want to match. The API takes it as `alternative_names` in `raw_manual_metadata` on `PUT /api/roms/{id}`, and returns the resolved list as `alternative_names` on the game. + +A few more things a search understands: + +- **Several searches at once**: separate them with `|`, as in `zelda | metroid`, to match either. +- **Hashes**: a CRC32, MD5, SHA-1 or RetroAchievements hash finds the game whose file has it, including a CHD's uncompressed SHA-1 as DAT files list it. + +Until you pick a sort, results are ordered by relevance on every database, with exact title matches (or exact alternative titles) first, then titles that start with your search, then the rest, and multi-word searches favour titles holding the words in order. Once you pick a sort, relevance only breaks ties. In the API, this is an empty `order_by` alongside `search_term` on `GET /api/roms`, and `char_index` comes back empty under relevance order because the results aren't alphabetical. + +Platform pickers and the platform list search more than the display name: a platform's slug, its folder name, its abbreviation (such as `SNES` or `PS2`) and its other known names all match. `PlatformSchema` exposes the last two as `abbreviation` and `alternative_names`. diff --git a/docs/using/jukebox.md b/docs/using/jukebox.md index e3a4d21b..6e1d7f89 100644 --- a/docs/using/jukebox.md +++ b/docs/using/jukebox.md @@ -26,7 +26,11 @@ Scanning reads each file's tags (title, artist, album, genre, year, track number Console sound files aren't read for tags yet. They show their filename and have no duration in the library, which also leaves them out of Free Radio and Decade Mix. Their length appears once they start playing. -Media keys and lock-screen controls don't work while a console sound file plays. +## Playback + +The player hands the playing track to your operating system through the browser's Media Session API, so media keys, the OS media overlay and phone lock screens show the track and its cover and can play, pause, skip and seek, except while a console sound file plays. + +An opt-in "Resume music after reload" user setting saves the queue and position per user and per browser, and picks the last track up where it stopped. If it was playing, it starts again on your first click or key press, since browsers block audio that starts on its own. ## Browsing diff --git a/docs/using/languages.md b/docs/using/languages.md index 1ff30ed7..6d000e81 100644 --- a/docs/using/languages.md +++ b/docs/using/languages.md @@ -5,7 +5,9 @@ description: UI translation # Languages -Pick the language you want for the UI. The choice is stored server-side, so it follows you across devices. If you're not signed in (Kiosk mode), the UI picks based on browser `Accept-Language` and falls back to `en_US`. +By default the UI follows your browser's language. This is the **Auto** setting, which takes the first language in the browser's preferred list with a translation, matching a regional variant to its base language when there's no exact match (so `it-CH` gets Italian), and falls back to `en_US` when none match. Visitors who aren't signed in, such as in Kiosk mode, always get Auto. + +Picking a specific language overrides Auto. The choice is stored server-side with your other UI settings, so it follows you across devices, and switching back to Auto lets each browser follow its own language again. ## Supported locales diff --git a/docs/using/netplay.md b/docs/using/netplay.md index 025d378c..f65ade58 100644 --- a/docs/using/netplay.md +++ b/docs/using/netplay.md @@ -16,6 +16,7 @@ Netplay lets you play [in-browser](in-browser-play/emulatorjs.md) games with oth - EmulatorJS Netplay enabled in `config.yml` (server owner-level) - ICE servers configured (STUN + TURN). Without them, Netplay only works when all players are on the same LAN. - All players need access to your instance, as Netplay doesn't proxy the ROM to people without accounts. +- A reverse proxy that forwards WebSocket upgrades, since Netplay connects over WebSockets only, with no long-polling fallback (see [Reverse Proxy](../install/reverse-proxy.md)). See [Configuration File → `emulatorjs.netplay`](../reference/configuration-file.md#emulatorjsnetplay) for the server owner setup. @@ -23,6 +24,8 @@ See [Configuration File → `emulatorjs.netplay`](../reference/configuration-fil The host creates a room with an optional password from a supported game and becomes Player 1. Up to three more players can join as Players 2-4 (depending on the core, since some only support 2). Other players must be on the same instance, since Netplay doesn't federate across instances. +Opening a room needs the `roms.read` scope and access to the game, so a game that's hidden from you, or above your [age limit](../administration/parental-controls.md), can't be hosted. The same check applies to joining a room without a password, and only players who can see the game find its rooms in the list. A password-protected room admits anyone with the password instead. + ## Controls Every player controls their own gamepad/keyboard locally, and inputs are sent to the host, which runs the game and broadcasts video. Slight lag is expected over the internet (~50-150 ms typical), with higher latency on transatlantic hops or when using TURN servers. @@ -38,7 +41,7 @@ WebRTC, the protocol Netplay uses, needs help to punch through some consumer rou - **Not all cores support Netplay.** SNES9x, Mupen64Plus, Mednafen PSX, Genesis Plus GX are the most battle-tested. - **Frame-perfect fighting isn't realistic.** Netplay is meant for casual co-op. For tournament-level fighting games, use something like [FightCade](https://www.fightcade.com/). -- **All players need an account.** Guests can't join a friend's game without an account, so each one needs at least a User account on your instance. +- **All players need an account.** Without a User account on your instance nobody can reach the player at all, and the account has to see the game unless the room has a password. - **RTC over TURN uses real bandwidth.** Hosting a 4-player N64 session over TURN can saturate a modest uplink. ## Security diff --git a/docs/using/notifications.md b/docs/using/notifications.md new file mode 100644 index 00000000..722620ca --- /dev/null +++ b/docs/using/notifications.md @@ -0,0 +1,152 @@ +--- +title: Notifications +description: Your notification inbox, and forwarding it to webhooks, email and chat services +--- + +# Notifications + +Each user has a notification inbox at `/notifications`. It stores each one and then pushes it to every tab you have open, so one that arrives while you're away is still there the next time you sign in. The inbox keeps your 200 newest notifications, and you can mark them read or dismiss them one at a time or all at once. + +## What sends a notification + +| Event | Level | Who gets it | Topic | +| ----------------------------------------------------------- | --------------- | -------------------------------------------------------------------------------------------------------------------- | ----------- | +| A scan finished | success | Whoever started it. A scheduled or watcher scan notifies every admin, but only when it found new games or platforms. | `scans` | +| A scan failed | error | Whoever started it, or every admin for a scheduled or watcher scan | `scans` | +| A task finished | success | Whoever ran it by hand. Scheduled runs that succeed stay quiet. | `tasks` | +| A task failed | error | Whoever ran it, or every admin for a scheduled run | `tasks` | +| Someone else ended your streaming session | warning | The user whose stream was ended | `streaming` | +| Your role changed | info | The user whose role an admin changed | `account` | +| One of your channels was turned off | warning | The channel's owner, after 10 failed deliveries in a row | `account` | +| A game was installed on one of your devices, or couldn't be | success / error | The user who sent the [install request](devices.md) | `devices` | +| A notification sent through the API | any | The recipients the sender named (see [Sending notifications](#sending-notifications)) | `custom` | + +A notification caused by another user, such as a role change or an ended stream, names that user as its sender. + +## Channels + +A channel forwards your notifications somewhere outside RomM. Each user sets up their own, up to 20, and each channel has two filters: + +- **Minimum level**: `info` forwards everything, `warning` forwards warnings and errors, and `error` forwards errors only, with success notifications counting as `info`. +- **Topics**: any of `scans`, `tasks`, `streaming`, `devices`, `account` and `custom`, or all of them (the default). + +A background worker delivers to channels, so a slow destination doesn't hold up RomM. A failed delivery is retried after 30 seconds, 2 minutes and 10 minutes. After 10 failed deliveries in a row the channel turns itself off and you get a notification saying why, with the last error kept on the channel. Any channel can send a test notification on demand, which reports the destination's error right away and isn't retried. + +Text that leaves RomM is in English, whatever language the UI is set to. When [`ROMM_BASE_URL`](../reference/environment-variables.md) points at a real host (not `localhost`, a loopback address or the default `0.0.0.0`), messages carry an absolute link back to the page the notification is about. + +Channel settings, including webhook URLs, secrets and service tokens, are encrypted with `ROMM_AUTH_SECRET_KEY`. Changing that key leaves existing channels unreadable, so they have to be set up again. + +### Webhook + +A webhook channel `POST`s RomM's own JSON to a URL you give it: + +```json +{ + "event": "notification", + "id": 1234, + "kind": "scan_completed", + "level": "success", + "title": "Scan completed", + "body": "12 new games", + "url": "https://romm.example.com/scan", + "data": { + "scanned_roms": 340, + "new_roms": 12, + "identified_roms": 11, + "new_platforms": 0 + }, + "actor": null, + "created_at": "2026-10-05T09:12:44.512000+00:00" +} +``` + +- `kind` is one of `scan_completed`, `scan_failed`, `task_completed`, `task_failed`, `streaming_session_ended`, `role_changed`, `channel_disabled`, `device_install_completed` and `device_install_failed`, or `custom` (or a client's own kind) for a notification sent through the API. +- `title` and `body` are rendered English text, and `data` holds the raw values they were built from (a scan's `data` carries all of its counts, trimmed in the example above). +- `url` is `null` when `ROMM_BASE_URL` doesn't point anywhere shareable. +- `actor` is `{"id": ..., "username": ...}` when another user caused the notification, otherwise `null`. + +The request has `Content-Type: application/json` and `User-Agent: RomM`, and must answer with a 2xx within 15 seconds. Redirects aren't followed, so a `3xx` counts as a failure. + +Only an admin's webhook may point at a private or local address (`192.168.x.x`, `10.x.x.x`, a container name, and so on), so other users' webhooks are limited to public hosts. + +#### Verifying the signature + +Give the channel a secret, and every request carries an `X-RomM-Signature` header holding `sha256=` followed by the hex HMAC-SHA256 of the raw request body, keyed with that secret. Compute the same digest over the body exactly as received (before parsing the JSON) and compare the two in constant time: + +```python +import hashlib +import hmac + + +def is_from_romm(secret: str, body: bytes, header: str | None) -> bool: + expected = "sha256=" + hmac.new(secret.encode(), body, hashlib.sha256).hexdigest() + # Bytes, because compare_digest raises on a str with non-ASCII characters. + return header is not None and hmac.compare_digest( + expected.encode(), header.encode() + ) +``` + +### Email + +Each notification arrives as a plain-text message whose subject starts with `[RomM]`. This channel is only offered once the server has [email set up](../administration/email.md). + +A new address first receives a 6-digit confirmation code, which you enter to confirm the channel. The code expires after 30 minutes and allows 5 tries, and a new one can be requested after a minute. Changing a channel's address asks for a new confirmation. + +### Apprise + +Admins can also forward notifications to any service [Apprise](https://github.com/caronc/apprise) supports, such as Discord, Telegram, Slack, ntfy, Gotify, Matrix, Pushover or Microsoft Teams. A channel takes the service's fields, or the service's own URL (a Discord webhook URL, for example) or an Apprise URL, which is split into those fields. Each service links to its setup guide on the [Apprise wiki](https://github.com/caronc/apprise/wiki). + +Apprise channels are admin-only because Apprise opens its own connections, which could otherwise be pointed at the local network. Services that act on the machine running RomM (desktop notifications, syslog, D-Bus and the like) aren't offered. + +## Sending notifications + +Anyone can send a notification to themselves, so scripts and companion apps can report back. Admins can also send one to other users, to every admin or to everyone, and the recipients see the admin as its sender. + +```sh +curl -X POST https://romm.example.com/api/notifications \ + -H "Authorization: Bearer $ROMM_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "title": "Nightly backup finished", + "body": "Saves and states were copied to the NAS.", + "level": "success", + "kind": "backup.done", + "link": "/settings", + "icon": "mdi-backup-restore", + "recipients": "admins" + }' +``` + +| Field | Required | Description | +| ------------ | :------: | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| `title` | yes | Up to 255 characters | +| `body` | | Up to 1000 characters | +| `level` | | `info` (default), `success`, `warning` or `error` | +| `kind` | | Your own identifier, such as `argosy.sync_done` (lowercase letters, digits and `_ . : -`). Defaults to `custom`, and RomM's own kinds are reserved. | +| `link` | | A path inside RomM, such as `/rom/12`. External URLs are refused. | +| `icon` | | A [Material Design Icons](https://pictogrammers.com/library/mdi/) name, such as `mdi-sync` | +| `data` | | Any JSON object up to 4096 characters, forwarded as-is in webhook payloads | +| `recipients` | | `null` (yourself, the default), a list of user ids, `"admins"` or `"all"`. Anything but yourself needs an admin with the `users.write` scope. | + +The response is the list of notifications created, one per recipient, each in the `custom` topic that channels filter by. Disabled users are skipped when sending to `admins` or `all`, and naming a missing or disabled user by id returns `404`. + +## API + +| Method | Path | Scope | Description | +| ------ | ----------------------------------------------------- | ---------- | -------------------------------------------------------------- | +| GET | `/api/notifications` | `me.read` | Your notifications, newest first | +| POST | `/api/notifications` | `me.write` | Send a notification (`users.write` too for other recipients) | +| POST | `/api/notifications/read` | `me.write` | Mark the given `ids` read, or all of them when `ids` is `null` | +| DELETE | `/api/notifications/{notification_id}` | `me.write` | Dismiss one notification | +| DELETE | `/api/notifications` | `me.write` | Dismiss all your notifications | +| GET | `/api/notification-channels` | `me.read` | Your channels | +| POST | `/api/notification-channels` | `me.write` | Add a channel (an email address is sent a confirmation code) | +| PATCH | `/api/notification-channels/{channel_id}` | `me.write` | Change a channel | +| DELETE | `/api/notification-channels/{channel_id}` | `me.write` | Delete a channel | +| POST | `/api/notification-channels/{channel_id}/test` | `me.write` | Send a test notification now | +| POST | `/api/notification-channels/{channel_id}/confirm` | `me.write` | Confirm an email channel with its `code` | +| POST | `/api/notification-channels/{channel_id}/resend-code` | `me.write` | Email a new confirmation code | +| GET | `/api/notification-channels/apprise-services` | `me.read` | Every Apprise service and its fields (admins only) | +| POST | `/api/notification-channels/apprise-services/parse` | `me.read` | Read a service URL or Apprise URL into fields (admins only) | + +Open tabs hear about changes over the `notifications:new`, `notifications:read` and `notifications:dismissed` socket events (see [WebSockets](../developers/websockets.md)). diff --git a/docs/using/saves-and-states.md b/docs/using/saves-and-states.md index 9a006f09..88dec426 100644 --- a/docs/using/saves-and-states.md +++ b/docs/using/saves-and-states.md @@ -45,7 +45,7 @@ An upload the server doesn't take (it's down, the connection dropped) is held in ## Save slots -Saves are organized into slots, the same model used by the [sync clients](#device-sync), so the same slots show up whether you played in the browser or on a device. +Saves are organized into slots, the model the [sync clients](#syncing-saves-between-browsers-and-devices) use too, so you see the same slots whether you played in the browser or on a device. - **`autosave`** is where ordinary play goes, and it keeps a capped history, so it prunes itself as you play. - **A named slot**, created on launch, keeps every version. Use one when you want a checkpoint you can always come back to. @@ -54,15 +54,31 @@ Each slot lists its newest version first, tagged **Latest**, with older versions The server prunes a slot to `MAX_SAVES_PER_SLOT` (50 by default, `0` to disable the cap). A client that asks for a tighter limit of its own gets the tighter of the two. Saves uploaded without a slot, such as one you imported by hand, are never pruned. -## Device sync +## Syncing saves between browsers and devices -Saves and states can sync to/from registered devices (Grout on muOS, DeckRommSync on a Deck, etc.). These pages cover it in depth: +Every in-browser player ([EmulatorJS](in-browser-play/emulatorjs.md), [EasyRPG](in-browser-play/easyrpg.md), [`js-dos`](in-browser-play/js-dos.md), [PICO-8](in-browser-play/pico-8.md) and [Ruffle](in-browser-play/ruffle.md)) syncs saves the same way a handheld does. Each browser profile registers as one of your [devices](devices.md), and when a game launches it negotiates its saves with the server: it downloads what's newer on the server, uploads what's newer locally, and drops a save that was deleted on the server. +When both sides changed the same slot, the browser plays the server's copy and uploads its own as a separate archived save, named after the game and the time it was captured, so neither version is lost. A browser only pairs saves written by its own player and leaves a save from another emulator in the same slot alone. + +Saves and states also sync with companion apps on other devices (Grout on muOS, Argosy on Android, DeckRommSync on a Deck, etc.), and with [RetroArch](../ecosystem/retroarch-cloud-sync.md) through its built-in Cloud Sync. These pages cover it in depth: + +- [Devices](devices.md): registering, renaming and removing them +- [RetroArch Cloud Sync](../ecosystem/retroarch-cloud-sync.md): setup and what syncs - [Device Sync Protocol](../developers/device-sync-protocol.md): wire-level reference - [SSH Sync](../developers/ssh-sync.md): server owner config - [Argosy Launcher](../ecosystem/first-party-apps.md#argosy-launcher)/[Grout](../ecosystem/first-party-apps.md#grout): per-app setup -Once a device is paired and sync is running, saves made on the device appear server-side within a couple of sync cycles (default: 15 minutes). Conflicts (same ROM saved on two devices between syncs) surface as two separate save entries, so pick which to keep. +Once a device is paired and sync is running, saves made on it appear server-side on its next sync. When the same slot is saved on two devices between syncs, RomM reports the conflict to the app, which decides what to keep, and most apps keep both as separate save entries. + +## Favorites, labels and names + +You can mark your own saves and states as favorites and tag them with free-text labels, for example to name a run or flag the save before a boss. A save or state takes up to 20 labels of up to 255 characters each, and labels that differ only in case count as one. You can also favorite, label or delete several at once. + +A save or state can be renamed, and its screenshot follows it. The new name has to be free among that game's saves (or states), ignoring case, and keep a name before the extension. A device or RetroArch that syncs by file name sees a renamed file as a different file. + +Saves and states written in the browser are named after the game and the local time they were captured, so the timestamp in the name matches your clock. + +When you share a save or state with other users, they see its author but not its favorite flag or labels, which stay yours. ## Format/core compatibility @@ -86,6 +102,8 @@ If you're moving saves between the bundled EmulatorJS and a stand-alone emulator Save states are always core-specific (a SNES9x state will not load in bsnes), so switching cores leaves your existing states unusable. +States made through [emulator streaming](emulator-streaming.md) record the RetroArch core that wrote them. When you resume from a state another core wrote, such as after a per-platform core override, the container starts the game fresh instead of loading a state its core can't read. States from before cores were recorded count as the platform's default core. + If you use states heavily, stick to one core per platform, or use save files (which are interchangeable) as your primary persistence. ## RetroAchievements and states diff --git a/scripts/gen_scheduled_tasks.py b/scripts/gen_scheduled_tasks.py index beb3efc3..7c1ce0a2 100644 --- a/scripts/gen_scheduled_tasks.py +++ b/scripts/gen_scheduled_tasks.py @@ -10,8 +10,9 @@ Sources, all fetched at the ref pinned in sources.toml: - backend/tasks/registry.py the SCHEDULED_TASKS and MANUAL_TASKS registries - backend/tasks/**.py each task's title, enabled flag and cron default + backend/tasks/registry.py the SCHEDULED_TASKS and MANUAL_TASKS registries, + and the TaskSpec each entry maps to (title, + enabled flag and cron default) env.template resolves env constants to documented defaults Every env var name in the output is resolved through env.template. A task @@ -19,7 +20,7 @@ printing an invented name, which is how this table drifted for several releases (see the `*_INTERVAL_CRON` names that never existed upstream). -Watchers aren't Task subclasses, so they can't be discovered the same way. They +Watchers aren't registered tasks, so they can't be discovered the same way. They stay declared in WATCHERS below, but their env vars go through the same env.template check as everything else. """ @@ -28,7 +29,7 @@ import ast import sys -from typing import Iterable +from collections.abc import Iterable, Iterator from scripts._sources import fetch_text, romm_raw_url, write_snippet from scripts.gen_env_vars import parse as parse_env_template @@ -50,38 +51,24 @@ }, ] -TASK_BASES = {"Task", "PeriodicTask", "RemoteFilePullTask"} ABSENT = object() # Distinguishes "resolved to None" from "keyword not passed". +# Tasks whose `enabled=` is a function call rather than an env constant. The +# call can't be resolved statically, so its meaning is declared here instead. +ENABLED_CALLS = { + "streaming_enabled": "Runs only while emulator streaming is enabled.", +} + class UpstreamDrift(RuntimeError): """Upstream no longer matches what this generator knows how to read.""" -def import_map(tree: ast.Module) -> dict[str, str]: - """Map imported task singletons to the module path they came from. - - `from tasks.scheduled.scan_library import scan_library_task` - -> {"scan_library_task": "backend/tasks/scheduled/scan_library.py"} - """ - out: dict[str, str] = {} - for node in ast.walk(tree): - if not isinstance(node, ast.ImportFrom) or not node.module: - continue - # `tasks.tasks` holds the base classes and TaskType, not task singletons. - if not node.module.startswith("tasks.") or node.module == "tasks.tasks": - continue - path = "backend/" + node.module.replace(".", "/") + ".py" - for alias in node.names: - out[alias.asname or alias.name] = path - return out - - -def assigned_dict(tree: ast.Module, name: str) -> ast.Dict: - """The dict literal assigned to a module-level `name`. +def module_assignments(tree: ast.Module) -> Iterator[tuple[str, ast.expr | None]]: + """(name, value) for every module-level `name = value` or `name: T = value`. - Both registries are module-level, so `tree.body` is enough and nothing is - gained by walking into function bodies. + The registries and their TaskSpecs are module-level, so `tree.body` is + enough and nothing is gained by walking into function bodies. """ for node in tree.body: if isinstance(node, ast.AnnAssign): @@ -90,15 +77,23 @@ def assigned_dict(tree: ast.Module, name: str) -> ast.Dict: targets = list(node.targets) else: continue - if not any(isinstance(t, ast.Name) and t.id == name for t in targets): + for target in targets: + if isinstance(target, ast.Name): + yield target.id, node.value + + +def assigned_dict(tree: ast.Module, name: str) -> ast.Dict: + """The dict literal assigned to a module-level `name`.""" + for target, value in module_assignments(tree): + if target != name: continue - if not isinstance(node.value, ast.Dict): + if not isinstance(value, ast.Dict): raise UpstreamDrift( f"`{name}` in backend/tasks/registry.py is a " - f"{type(node.value).__name__}, not a dict literal, so this " + f"{type(value).__name__}, not a dict literal, so this " "parser can no longer read the registry." ) - return node.value + return value raise UpstreamDrift( f"no `{name}` dict found in backend/tasks/registry.py. The registry moved " @@ -106,66 +101,73 @@ def assigned_dict(tree: ast.Module, name: str) -> ast.Dict: ) -def registry_module_paths( - tree: ast.Module, imports: dict[str, str], name: str -) -> list[str]: - """Module paths for the task singletons a registry dict maps to. +def task_specs(tree: ast.Module) -> dict[str, dict[str, ast.expr]]: + """Keyword args of every module-level `X_SPEC = TaskSpec(...)` assignment. + + `SCAN_LIBRARY_SPEC: Final = TaskSpec(title="Scheduled rescan", ...)` + -> {"SCAN_LIBRARY_SPEC": {"title": , ...}} + """ + specs: dict[str, dict[str, ast.expr]] = {} + for name, value in module_assignments(tree): + if not ( + isinstance(value, ast.Call) + and isinstance(value.func, ast.Name) + and value.func.id == "TaskSpec" + ): + continue + # Positional or `**` arguments would hide a title, flag or cron from + # this parser, which would then print a default instead of failing. + if value.args or any(kw.arg is None for kw in value.keywords): + raise UpstreamDrift( + f"{name} passes positional or ** arguments to TaskSpec, which " + "this parser cannot resolve." + ) + specs[name] = {kw.arg: kw.value for kw in value.keywords if kw.arg} + return specs + - `SCHEDULED_TASKS = {"scan_library": scan_library_task, ...}` - -> ["backend/tasks/scheduled/scan_library.py", ...] +def registry_specs( + tree: ast.Module, specs: dict[str, dict[str, ast.expr]], name: str +) -> list[tuple[str, dict[str, ast.expr]]]: + """The (spec name, kwargs) pairs a registry dict maps to, in source order. - Registry order is the table's order, so the dict is read in source order. + `SCHEDULED_TASKS = {"scan_library": SCAN_LIBRARY_SPEC, ...}` + -> [("SCAN_LIBRARY_SPEC", {...}), ...] """ - paths: list[str] = [] + out: list[tuple[str, dict[str, ast.expr]]] = [] for value in assigned_dict(tree, name).values: if not isinstance(value, ast.Name): raise UpstreamDrift( f"{name} in backend/tasks/registry.py maps a key to a " - f"{type(value).__name__} rather than an imported task " - "singleton, which this parser cannot resolve." + f"{type(value).__name__} rather than a TaskSpec constant, which " + "this parser cannot resolve." ) - path = imports.get(value.id) - if path is None: + if value.id not in specs: raise UpstreamDrift( - f"{name} references {value.id}, which is not imported from a " - "tasks.* module in backend/tasks/registry.py." + f"{name} references {value.id}, which is not a module-level " + "TaskSpec(...) in backend/tasks/registry.py." ) - if path in paths: - # Registry keys are unique, so two of them resolving to one module - # means two task singletons share a file. `task_kwargs` reads only - # the first task class it finds there, so the second would vanish - # from the table without a word. + if any(spec_name == value.id for spec_name, _ in out): raise UpstreamDrift( - f"{name} maps two tasks to {path}. task_kwargs() reads only the " - "first task class in a module, so one would be dropped silently." + f"{name} maps two keys to {value.id}, which would list the same " + "task twice." ) - paths.append(path) - return paths - + out.append((value.id, specs[value.id])) + return out -def task_kwargs(module_src: str, path: str) -> dict[str, ast.expr]: - """Keyword args of the `super().__init__(...)` call in a task class.""" - tree = ast.parse(module_src) - for node in ast.walk(tree): - if not isinstance(node, ast.ClassDef): - continue - base_names = {b.id for b in node.bases if isinstance(b, ast.Name)} - if not base_names & TASK_BASES: - continue - for call in ast.walk(node): - if not isinstance(call, ast.Call): - continue - func = call.func - if not isinstance(func, ast.Attribute) or func.attr != "__init__": - continue - return {kw.arg: kw.value for kw in call.keywords if kw.arg} - - raise UpstreamDrift(f"no task class with a `super().__init__(...)` call in {path}") +def env_name(var: str, env: dict[str, dict], where: str, field: str) -> str: + if var not in env: + raise UpstreamDrift( + f"{where} passes {field}={var}, but env.template does not define " + f"{var}. Either the variable was renamed upstream or it is " + f"undocumented, and printing it here would be a guess." + ) + return var -def resolve(node: ast.expr | None, env: dict[str, dict], path: str, field: str): - """Resolve a constructor argument to (env_var_name, value). +def resolve(node: ast.expr | None, env: dict[str, dict], where: str, field: str): + """Resolve a TaskSpec argument to (env_var_name, value). A literal resolves to itself with no env var. A `Name` is an env constant, so it is looked up in env.template and fails loudly if absent. @@ -177,40 +179,72 @@ def resolve(node: ast.expr | None, env: dict[str, dict], path: str, field: str): return None, node.value if isinstance(node, ast.Name): - var = node.id - if var not in env: - raise UpstreamDrift( - f"{path} passes {field}={var}, but env.template does not define " - f"{var}. Either the variable was renamed upstream or it is " - f"undocumented, and printing it here would be a guess." - ) + var = env_name(node.id, env, where, field) return var, env[var]["default"] - # Anything else (a call, an f-string, a conditional) is beyond what this - # parser claims to understand, so say so rather than print something wrong. + # `f"... {ENV_VAR} ..."`: print the env var's name where its value goes. + if isinstance(node, ast.JoinedStr): + parts: list[str] = [] + for v in node.values: + if isinstance(v, ast.Constant): + parts.append(str(v.value)) + elif isinstance(v, ast.FormattedValue) and isinstance(v.value, ast.Name): + parts.append(f"`{env_name(v.value.id, env, where, field)}`") + else: + break + else: + return None, "".join(parts) + + # `enabled=ENV_VAR > 0`: the env var is still what turns the task on. + if isinstance(node, ast.Compare) and isinstance(node.left, ast.Name): + var = env_name(node.left.id, env, where, field) + return var, env[var]["default"] + + # Anything else (a call, a conditional) is beyond what this parser claims + # to understand, so say so rather than print something wrong. raise UpstreamDrift( - f"{path} passes a {type(node).__name__} for {field}, which this parser " + f"{where} passes a {type(node).__name__} for {field}, which this parser " f"cannot resolve. Extend resolve() to handle it." ) -def build_row(path: str, kind: str, env: dict[str, dict]) -> dict: - kwargs = task_kwargs(fetch_text(romm_raw_url(path)), path) - - _, title = resolve(kwargs.get("title"), env, path, "title") - _, description = resolve(kwargs.get("description"), env, path, "description") - enable_var, _ = resolve(kwargs.get("enabled"), env, path, "enabled") - cron_var, cron = resolve(kwargs.get("cron_string"), env, path, "cron_string") +def build_row( + where: str, kwargs: dict[str, ast.expr], kind: str, env: dict[str, dict] +) -> dict: + _, title = resolve(kwargs.get("title"), env, where, "title") + _, description = resolve(kwargs.get("description"), env, where, "description") + cron_var, cron = resolve(kwargs.get("cron_string"), env, where, "cron_string") + + enabled = kwargs.get("enabled") + note = "" + enabled_value: object = True + if ( + isinstance(enabled, ast.Call) + and isinstance(enabled.func, ast.Name) + and enabled.func.id in ENABLED_CALLS + ): + enable_var, note = None, ENABLED_CALLS[enabled.func.id] + else: + enable_var, enabled_value = resolve(enabled, env, where, "enabled") + if isinstance(enabled, ast.Compare): + # The env var isn't a plain on/off switch, so spell out the test. + note = f"Runs while `{ast.unparse(enabled)}`." if title in (ABSENT, None, ""): - raise UpstreamDrift(f"{path} has no title= in its constructor") + raise UpstreamDrift(f"{where} has no title=") if description in (ABSENT, None, ""): purpose = "-" else: purpose = str(description).rstrip(".") + "." - if kind == "Scheduled" and not enable_var: - purpose += " Always on, not configurable." + if note: + purpose += " " + note + elif kind == "Scheduled" and not enable_var: + # TaskSpec defaults `enabled` to False, so a missing flag means off too. + if enabled_value is True: + purpose += " Always on, not configurable." + else: + purpose += " Off, not configurable." return { "name": str(title), @@ -224,14 +258,17 @@ def build_row(path: str, kind: str, env: dict[str, dict]) -> dict: def collect(env: dict[str, dict]) -> list[dict]: registry = ast.parse(fetch_text(romm_raw_url("backend/tasks/registry.py"))) - imports = import_map(registry) - scheduled = registry_module_paths(registry, imports, "SCHEDULED_TASKS") - manual = registry_module_paths(registry, imports, "MANUAL_TASKS") + specs = task_specs(registry) + scheduled = registry_specs(registry, specs, "SCHEDULED_TASKS") + manual = registry_specs(registry, specs, "MANUAL_TASKS") + scheduled_names = {name for name, _ in scheduled} - rows = [build_row(p, "Scheduled", env) for p in scheduled] + rows = [build_row(n, kw, "Scheduled", env) for n, kw in scheduled] # A task in both registries is scheduled and also runnable by hand, so it is # already listed above. - rows += [build_row(p, "Manual", env) for p in manual if p not in scheduled] + rows += [ + build_row(n, kw, "Manual", env) for n, kw in manual if n not in scheduled_names + ] for w in WATCHERS: for field in ("enable_var", "env_var"): diff --git a/scripts/sources.toml b/scripts/sources.toml index 07104db7..bf6f7764 100644 --- a/scripts/sources.toml +++ b/scripts/sources.toml @@ -12,4 +12,4 @@ repo = "rommapp/romm" # Set to a tag (e.g. "v5.0.0") or a full SHA. Generators fetch raw files # from raw.githubusercontent.com/${repo}/${ref}/... -ref = "5.3.0-beta.1" +ref = "9ae2cd0e12d64e9bddd0519e0bfec984e05d5083"