Backend Architecture¶
Overview¶
The Python backend follows Cosmic Python ("Architecture Patterns with Python") adapted for a single-user application. Code is split into layers with a strictly enforced dependency direction:
services/— orchestration. Business logic and the public use cases.adapters/— I/O. Everything that touches the network, the filesystem, the clock, or Steam.domain/— pure compute. Functions in, values out; no I/O, no state mutation, no service/adapter imports.lib/— cross-cutting utilities independent of every other layer.models/— data shapes (TypedDicts, dataclasses) independent of every other layer.host/— the process itself. Not a layer of the application at all: see below.
Services depend on Protocols (defined in services/protocols/), never on concrete adapter classes. Adapters
implement those Protocols. bootstrap/ is the composition root — the only place where concrete adapters meet services.
bootstrap/ also builds the Application — every wired service, the start-up repairs and the shutdown. main.py owns
the process entry (run() and build_backend()) and the endpoints; it holds no business logic, and it reads no
arguments. The pre-install check is an entry of its own, check.py beside it, which the installer runs on a version it
has unpacked and not yet put in place — a file rather than a flag on main.py, because a version whose main.py
predated the flag would ignore it and start a whole backend. It logs to stderr only, copies the live database and the
settings (bootstrap/check.py, the database through SQLite's backup API) under roots of its own, then imports main.py
— whose start sits behind __main__, so importing it starts nothing — and calls build_application(), and nothing
else: no lock, no port, no backend.log, no start-up repair, no network. Its exit status is its answer: 0 when the
Application was built; 1 when it was not, or main.py did not import, with the traceback on stderr, which is also
what Python itself answers for a module that does not import; and 2 when the check was not tried — arguments it cannot
read, a root it may not build under, or live data it could not copy. It refuses to build unless the code root is its own
tree and every other root, the runtime directory among them, is absent or empty. What the installer does with each
answer: Running an installed one.
class Endpoints:
# Holds: the Application's services and the host's status record, both handed in
# Owns: the endpoints — one public method marked @route each
# Delegates: every use case to a service, every I/O to an adapter behind it
The host (backend/host/)¶
The backend runs in a process of its own — ADR-0036. host/ is everything
that used to be a plugin loader's job: the single-instance lock, the start-up order, a loopback port, the panel bundle,
and one WebSocket carrying calls and events. Standard library only.
It is deliberately not a layer of the application, and the boundary runs in both directions. host/ imports nothing
from services, adapters, bootstrap or domain; and nothing but main.py imports host — both are
.importlinter contracts. The second half is the one that would rot without a check: the first service that wanted to
send an event would reach in here for the sink, and the composition root would stop being the only place that knows a
transport exists.
| Module | Owns |
|---|---|
protocol.py |
The four message kinds and the transport-reason vocabulary |
access.py |
The three admission checks — Host, Origin, Token — in that order |
dispatch.py |
Name resolution onto Endpoints, the exception boundary, the answer cap |
connection.py |
One live WebSocket: frame reading, the frame cap, the heartbeat, calls in flight |
events.py |
Where an event leaves the process, and whether anybody heard it |
server.py |
The bind and its fallback, the file route, the upgrade, newest-connection-wins |
single_instance.py |
The flock beside the database, and the port file beside the session |
logging_setup.py |
The root logger, and the token redaction on its file handler |
runtime.py |
The start-up order and the shutdown |
status.py |
What the panel may ask about the run hosting it |
Two pure halves live in lib/ instead, because they are checkable against tables of bytes rather than against a running
server: lib/websocket_frames.py (the RFC 6455 codec) and lib/http_messages.py (request heads in, response heads
out).
The protocol. JSON text frames, both directions, each message stating its kind as a readable word:
call {type, id, method, args} args are positional
reply {type, id, result}
error {type, id, reason, message, traceback?}
event {type, name, payload}
error.reason is the transport layer — method_unknown, payload_too_large, backend_exception,
malformed_message, connection_lost, stranded_panel; the last two are the panel's own socket's answer, never sent
by a backend. An endpoint's own failure is a successful transport and arrives inside result in the
{success, reason, message} shape main.py builds. scripts/check_failure_shape.py reads host/ too, but the
transport's error frame carries no success key, so the gate cannot tell the two apart; keeping them apart is prose and
review. Reachable methods are exactly the endpoints on Endpoints: the public methods marked @route, def or
async def alike. An endpoint's answer is awaited only when it is awaitable. The set is the one
scripts/check_endpoint_parity.py derives, asserted equal by tests/host/test_dispatch.py.
A refusal can be raised, and Endpoints answers it. Every endpoint on Endpoints is wrapped once, by a class
decorator in main.py, so no endpoint carries a second decorator and a new one is wrapped without anyone remembering
to: a raised Refused (lib/errors.py) or DomainRefused (domain/refusal.py) answers
{success: False, reason, message} with the refusal's details beside them; a returned PartialFailure
(lib/partial_failure.py), the result of work that stopped partway, answers the same shape with what was done; and a
raised RommApiError answers classify_error's reason and message. Nothing else is caught there: any other exception,
a raw ConnectionError or OSError included, is still a backend_exception. A translated RomM error is logged as one
line, its type and message without a stack: at info when classify_error answers it server_unreachable, because an
offline handheld is an expected state, and at warning otherwise. A refusal is a decision and is not logged. A def
endpoint stays a def, and an awaitable it hands back is translated when the dispatcher awaits it. The conflict rules
refuse this way too: hold and hold_start raise the refusal of the first rule that holds, so the use case's block
does not run (GLOSSARY.md → Conflict rules). No other module builds the failure shape from a literal;
§4 of Boundary Enforcement holds that rule and names the two results that compute a
failing success outside main.py.
Two size caps, two purposes. ~12 MiB on one call's encoded answer, refused as an ordinary error for that call alone; 16 MiB on the connection's frames, judged on the announced length before a byte is buffered, whose breach closes the socket. Breaking the second rejects every call in flight, which is why one oversized cover image may not reach it.
No reply store. A call whose answer was in flight when the socket went is not redelivered — its task is cancelled,
and the caller's own pending register answers it connection_lost. A lost answer fails visibly; it never disappears.
A panel another backend process loaded is told so, and admitted nowhere. Its upgrade carries that process's token; the server completes the handshake and closes it at once with a code of its own, and attaches nothing to it — which code, and what the panel does: loading-the-panel.md.
Events leave through a seam build_application is handed, never a module-level call. It is the host sink's emit,
an EventEmitter — one emit(name, payload) that answers whether anybody heard. Nothing is buffered: an event with
no panel attached is dropped with a log line, because every event this backend sends is a statement about now and
"sync finished" delivered three hours later lands in a session that never started one.
The answer is what a prune claim on an event needs. Three events carry one, because their Steam-side work outlives the
backend's, and the service that emits each takes it when it emits, through ConflictRules.emit_under_lease: the library
service for sync_complete and sync_stale, and the shortcut artwork service for each shortcut_artwork batch. What
the frontend does under the two library-sync claims is
Steam Non-Steam Shortcuts — Explicit cleanup of vanished versions.
A service's EventEmitter is the sink's own emit, so it answers whether anybody heard too. A claim handed to a panel
that is not there holds off every removed-game cleanup until it expires — so whichever side took it gives it straight
back the moment the answer says nobody heard. One consequence follows and nothing checks it: an event carrying a claim
is awaited, never scheduled as a task (the answer would arrive after the claim was handed out).
The start-up order is what makes the port file meaningful:
lock (with a short retry window) ← beside the database, which is what it protects
→ schema migration + start-up routines
→ bind the port (27737, then the next free)
→ write the port file
So "the port file is there" means "the backend is ready", and there is no readiness flag for a call to wait on. A second backend never reaches the port fallback — the lock refused it first — so a fallback always means some other program holds the port. The port file is a hint; connecting to it is the proof.
A start-up routine's failure is counted, not fatal (bootstrap/startup.py). Application.run_startup_repairs runs
them all, in one order, and reports each failure to the recorder it is handed. One edge binds two of them:
report_missing_installs runs only after detect_retrodeck_path_change succeeded, because the report reads the
pending homes the detection writes. build_application runs none of them; the entry point runs them after building and
before the port is bound. The shutdown (Application.shutdown) reaches the host only on the BackendBuild the build
answers with, and so does the one step the host runs before it lets go of Steam's debugger (before_detach, the focus
watcher's release of its pin,
Controller focus in Desktop Mode); a build that failed
answers none, and the host shuts nothing down.
The shutdown stops what runs in a fixed order (bootstrap/application.py, pinned by
tests/test_bootstrap_application.py): first the tasks the start-up repairs started, the focus watcher's among them,
then the shortcut artwork job, the update install, the library sync, the removed-game cleanup, the downloads, the
migration, the session lifecycle and playtime. The host stops the server and releases the single-instance lock only
after all of it. The library sync sets its cancel flag, then cancels its run and waits until it has ended. There is no
grace wait between the two: a run waiting on a RomM request sees the flag only once the request ends, which can outlast
the 90 s systemd gives a stopping unit; each unit of work commits or rolls back in its own transaction, so the
cancellation leaves nothing half written; and the start-up repair (reconcile_orphaned_sync_runs) records the run it
finds still running as interrupted by a restart, which is what happened. The resume offer the next start computes
reads only committed data, so a run cancelled in the middle of an apply chunk is offered for resumption like any other
interrupted run. Cancelling the run does not stop work it already handed to a worker thread: a unit's commit running
there finishes in its own transaction after the run's task has ended, possibly after the lock is released. Whether a
second backend can start against the database in that window is not established.
The services with no place in that order hold only short tasks — the removal's and the adoption's progress frames, the
save-status check a refresh starts, the wait on a cancelled removal's thread, the retry frame, and the prune conflicts'
releases. Nothing shuts those down: asyncio.run cancels them once the host has stopped.
Dependency Diagram¶
host/ (the process: lock, port, protocol, lifetime — imports none of the below)
↑ dispatches onto
main.py (run(), build_backend() — the process; Endpoints — the endpoints; the only importer of host/)
↓ calls
bootstrap/ (composition root: build_application() composes adapters.bootstrap() and services.wire_services() into an Application)
↓ creates
┌─────────────────────────────────────────────────────────┐
│ Adapters (own all I/O — implement Protocols) │
│ RommHttpAdapter / RommApiAdapter — RomM REST │
│ SteamConfigAdapter — Steam VDF, grid dir, Steam Input │
│ SteamGridDbAdapter / SgdbArtworkCacheAdapter — SGDB │
│ PersistenceAdapter (+ persister adapters) — JSON I/O │
│ SqliteUnitOfWork (+ repository adapters) — SQLite I/O │
│ CoverArtFileStore / DownloadFile │
│ FirmwareFile / MigrationFile / RomFile / SaveFile │
│ RetroDeckFoldersAdapter (RetroDECK's folders) │
│ EmulatorSourcesAdapter (emulator sources) │
│ AtlasCatalogueAdapter (ES-DE catalogue, vendored) │
│ PlatformCoreReaderAdapter (settings platform_cores) │
│ SystemClock / SystemUuidGen / AsyncioSleeper │
│ HostnameAdapter / PathProbe / DebugLogger │
└────────────────────────┬────────────────────────────────┘
│ injected via *ServiceConfig
┌────────────────────────▼────────────────────────────────┐
│ Services (depend on Protocols, not concrete adapters) │
│ LibraryService SaveService │
│ DownloadService PlaytimeService │
│ FirmwareService SteamGridService │
│ MetadataService AchievementsService │
│ MigrationService GameDetailService │
│ ArtworkService RomRemovalService │
│ ShortcutRemovalService SettingsService │
│ CoreService ConnectionService │
│ EmulatorSourcesService │
│ StartupHealingService │
│ SessionLifecycleService │
│ RomAdoptionService RomInstallRecorder │
└────────────────────────┬────────────────────────────────┘
│ depend on
┌────────────────────────▼────────────────────────────────┐
│ Protocols (services/protocols/) — grouped topically; │
│ see Protocol Interfaces below │
└─────────────────────────────────────────────────────────┘
Domain (domain/) — pure compute, imported by services and adapters; imports nothing above it.
Arrow direction: depends-on (A -> B means A uses B).
The XxxServiceConfig constructor pattern¶
Every service takes a single config keyword argument — a frozen dataclass named <ServiceName>Config. All
dependencies live in the config: Protocol-typed adapters, infrastructure seams (event loop, logger, Clock, UuidGen,
Sleeper), persistence callbacks, and settings-derived values. There are no bare-param or mixed constructors.
shortcut_artwork_service = ShortcutArtworkService(
config=ShortcutArtworkServiceConfig(
steam_config=..., # Protocol-typed adapter
source=..., # cross-service Protocol-typed peer
uow_factory=..., # UnitOfWorkFactory Protocol (roms / artwork_slot_outcomes)
emit=..., # EventEmitter Protocol
sleeper=..., # Sleeper Protocol
clock=..., # Clock Protocol
# ...
),
)
Outer services keep the Service token in both names (SteamGridService + SteamGridServiceConfig). Sub-services may
use role-based names without the token when it reads more naturally (SyncEngine + SyncEngineConfig,
SyncOrchestrator + SyncOrchestratorConfig).
Module Responsibilities¶
Services (backend/services/)¶
Six services are packages of sub-services — library/, saves/, firmware/, prune/, rom_adoption/ and
migration/; protocols/ is a package too, but it holds the Protocol interfaces rather than a service. The rest are
single modules. A service over ~1000 LOC is the decomposition signal.
| Module | Domain |
|---|---|
library/ |
LibraryService façade — fetch ROMs, preview/apply sync, per-unit shortcut delivery, roms/SyncRun writes + queries (decomposed; see below) |
saves/ |
SaveService aggregate — .srm upload/download, conflict detection, slots, versions, and the local-save check (check_local_drift, The Local-Save Check), held by the status sub-service (decomposed; its sub-services are described on Save File Sync Architecture) |
downloads/ |
DownloadService façade (service.py) over one state box (_state.py), the queue (queue.py) and the transfer (transfer.py) — progress, bounded-concurrency download queue (Semaphore(2) + reserved-bytes pre-flight); cancel/cleanup never deletes a live install (decomposed; see "DownloadService notes"). The occupancy pre-flight sits beside the disk-space one and refuses rather than writing over content Tender did not place (DownloadTargetGateFn → RomAdoptionService). DownloadUnpacker (unpack.py) — what a finished transfer becomes: a file kept as it is, or a folder unpacked from a zip with its M3U, launch file and ES-DE name, and the partial files a failed download leaves |
rom_adoption/ |
RomAdoptionService (service.py) + AdoptionRenamer (renamer.py) over the shared Target (_target.py). The service owns the download-target gate (describe what occupies the path, refuse with the comparison, or clear what an explicit replace covers — replace_covers, domain/served_shape.py), the candidate search behind it, adoption of on-disk content as an install, and the user-triggered content check against RomM's per-file checksums. The same filter answers the game-detail read through the AdoptionCandidateProbeFn seam — a boolean over the leaner list_top_level_names listing (no size-or-mtime stat per entry), stopping at the name match so the archive reads that rank candidates are skipped, and swallowing every failure into "no candidate" so an unreadable folder never makes a game look uninstallable. The two answer from different knowledge on purpose — a roms row against the server payload — and are not held to agreeing: they have diverged on the served shape, the platform folder, the matched name and the directory listing itself. So the click-time search re-runs and is the authority, and the button's promise rests on the last answer in its chain rather than on an invariant over two searches. Every entry is judged by what it is, without following it: a file, a directory or a symlink, and anything else — FIFO, socket, device node — has no kind. One function (_kind_of) decides that and every read of the filesystem asks it, listings and single-path description alike; re-deriving the rule per door is what left a named pipe adoptable at the target path after the listings already excluded it. The two reads differ only in what they do with a kindless entry — a listing drops it, describe_path reports it with kind: None, because something that is there must not come back as nothing for the next write to act on. A symlink is its own kind rather than whatever it resolves to, because claim_source refuses a symlink on every uninstall, so an install row pointing at one could never be removed. That chain, most specific first: adoption_candidates (served shape, adoptable kind), unusable_namesake (the other shape, or a link — one refusal, because the user's choice is the same), and candidate_vanished (the backstop: the page reported a copy, neither of the above applies, so the download stops rather than starting silently — which also covers the file deleted between page open and press). The same rule reaches the occupied-target dialog through describe_path, which answers existence with lstat: a link or a kindless entry at the ROM's own path is reported as occupying it and marked adoptable: False, which also stops the finalize os.replace from overwriting a dangling link in silence. adoptable is one predicate (adoptable_content) rather than a flag each site computes — the gate, the validation immediately before the move and the last check after it all ask it, so a candidate that turns into a link between the dialog opening and the user confirming is refused by the acting site and not only by a disabled button. Both the validation before the move and the last check after the sibling supersede are one function (_refuse_unadoptable), so the pair of answers cannot drift apart: content that vanished is nothing_to_adopt, content still sitting there that no install row may point at — the wrong shape, a link, or no kind at all — is unexpected_content_kind, because "the files are no longer there" said of a file that merely became a link sends the user looking for something that did not happen. Before any of it, the directory must be an ES-DE system (SystemKnownFn over the same es_systems.xml the accept-list comes from), because an empty accept-list for a directory that is no system would remove the extension filter as well. When the target path is free the same gate lists the platform directory's top level (never descending — a single multi-file install can hold tens of thousands of files), keeps what ES-DE's live per-system accept-list admits, subtracts every path a rom_installs row accounts for, and matches on a tag-stripped normalized name (domain/rom_candidates.py); a match refuses the download with a short list ranked by free evidence — a single-member archive's central-directory CRC32 against RomM's file-level digest, an exact fs_size_bytes match, or the name alone — and states when the list was capped. Both exits of that dialog then go through AdoptionRenamer, which is why they cannot drift: Use These Files renames the ROM to the canonical name together with every save and savestate named after it, Download Instead removes the ROM (bounded by the system's own ROM folder, as the target-path replace is) and carries only the saves — same plan (domain/adoption_rename.py), same collision question asked before the first file moves, same answer applied to the same whole set. A discarded multi-file candidate carries nothing, deliberately: its saves are named after a launch file that is still inside an unfetched archive. The move itself is AdoptionMoveStore's link-then-unlink with a rename-with-rollback fallback for a directory or a cross-filesystem set; an Overwrite's replaced targets — always saves or savestates, never the ROM — go through SaveBackup.back_up_save_file rather than any delete of the adoption's own, so the .romm-backup discipline stays single-sourced (#965). A removal that fails after the carry names the saves that already moved, because that abort is not clean. The two companion directories are the resolver's answers — the save directory and the savestate directory, asked of the emulator the ROM launches with, for the launch file under its old name and under its new one (the new one need not exist; the resolver places a game by the path's own coordinates). A directory nothing could establish, and an emulator that keeps no savestates, carry nothing. Order is validate → carry → supersede → record. A directory's files are held to the place RomM names: RomFile.is_top_level compares rom.full_path against file_path, so the two are one coordinate system and the ROM-relative path is a subtraction — an entry the payload does not locate falls back to a by-name search, and one whose relative path escapes the ROM directory is refused rather than looked up. Anything whose name is one RomM reads as an archive is held to its contents instead, because RomM's digest for such a file describes what is inside (its current scanner accumulates over every member, the pre-4.9.0 one took the largest) while file_size_bytes stays the container's: neither the container's digest nor its size is compared, and the local side comes from the ZIP central directory (name, uncompressed size, CRC32, all free). One member inside — the same bytes under every rule RomM has used — is compared against the file-level digest; several members are compared one by one when the payload carries archive_members, and are unverifiable when it does not, since the single number is then either a composite or the largest member with nothing saying which. A size or CRC32 disagreement is reported without decompressing; the MD5 is what earns a match. A container this adapter cannot open leaves the entry unverifiable rather than accused — except a single-member archive whose member size the loose file on disk matches exactly, which is the unpacked case. Each difference is one {name, detail} line, and unverifiable carries a message per reason (no checksums published / could not be read / one checksum for the whole archive). Never deletes on its own initiative; both replace legs share one containment-guarded removal, and only the target-path leg keeps the carve-out that leaves a single file for os.replace to swap atomically. The candidate search is skipped on a resume — that decision was taken when the download was admitted, and the file the user declined must not refuse the transfer they started. Adoption runs the sibling supersede through SiblingSupersedeFn (rule: domain/sibling_supersede.py) and passes its target as adopted_path, so a version recorded at that same path hands its install record over and its files stay as they are. See ADR-0028 |
rom_install_recorder.py |
RomInstallRecorder — the one writer of a rom_installs row (the upsert). Both a completed download and an adoption reach an install through here, so an adopted row cannot drift from a downloaded one |
firmware/ |
FirmwareService façade — joins the RomM firmware listing with what the installed emulators want (read live per platform through FirmwarePlatformResolver), downloads, per-core filtering; get_platform_firmware_status ships one platform's bios_level (the resolver's verdict, unknown/ok/partial/missing via domain.bios_status.compute_bios_level) + required_count/required_downloaded/server_count/local_count/known_count/unknown_count/deletable_count/one_of_groups, one entry per emulator the resolver answered for (emulator_tabs) and the resolver's words about the launching emulator so the platform detail reads the decision off the payload (#461), while get_firmware_status beside it answers only whether the RomM listing could be read and pays no reading at all (decomposed; see below) |
session_lifecycle.py |
SessionLifecycleService — post-exit orchestration (playtime + post-exit save sync + achievement sync + migration refresh) |
migration/ |
MigrationService (service.py) — RetroDECK path-change detection + file migration — over FileMover (_moves.py), which moves one item to a path something may already occupy. A save directory that moves inside one home is not this package's: each game follows its own save directory the next time Tender touches that game's saves (services/saves/save_directory.py). A home changed again before migrating accumulates a pending-home set (_previous + a _hops array) instead of overwriting the marker, and collection matches each tracked record by longest-prefix against every pending home, so files under an intermediate home are never stranded (#1042; pure transition/remap kernel in domain/migration_paths.py) |
shortcut_artwork.py |
ShortcutArtworkService — the background job that gives every bound shortcut its SteamGridDB icon, or Tender's logo where SteamGridDB has none, and its SteamGridDB wide grid, logo and cover, each where the slot is not filled and its recorded outcome does not rule it out (Shortcut artwork) |
steamgrid/ |
SteamGridService façade (service.py) — the API key, the SteamGridDB game a ROM's artwork comes from, icons — over the artwork cache (_files.py, SgdbArtworkFiles, the one holder of its index), the ARTWORK tab's reads and writes (_slots.py, ArtworkSlots) and its endpoints (_tab.py, ArtworkTabUseCases), keeping automatic artwork current (_current.py, KeepingCurrent), the images handed to Steam and Steam's answer for them (_handover.py, HandedOverImages), and the refusals SteamGridDB's errors answer with (_refusals.py) |
artwork.py |
ArtworkService — cover art download into the per-ROM cover cache ({runtime}/covers/{rom_id}.png, the source of RomM's cover for every reader but Steam, which shows a cover chosen from SteamGridDB in its place; the game page's GAME INFO tab shows what Steam shows), cover-cache invalidation against the persisted cover_source fingerprint (refresh_changed_covers, the per-unit pass that re-downloads changed covers and feeds the frontend's tile re-apply, #1386; a ts-only fingerprint change revalidates the cached bytes with a conditional request via the {rom_id}.cover-meta.json validator sidecar instead of re-downloading, #1454), publish of the active version's cache cover onto the Steam grid as {app_id}p.png (a copy so every sibling keeps its own file, ADR-0021 / #1346), the read-only get_artwork_base64 query, the cache-first fetch_cover_base64 used by the version picker (a cache miss downloads the ROM's cover from RomM — works for a server-only version with no local row), cache/staging cleanup + startup orphan pruning, the full grid-image sweep on shortcut removal (remove_artwork_files deletes every portrait/wide/hero/logo/icon × png/jpg/jpeg form for the removed appId, not just {app_id}p.png), and the user-triggered orphaned grid-image cleanup (cleanup_orphaned_grid_images, Data Management): deletes grid files whose parsed appId is in the non-Steam-shortcut range (domain/artwork_paths.is_shortcut_app_id, [0x80000000, 0xFFFFFFFF] — store-game custom art is never a candidate) and absent from the frontend's full live-shortcut scan (RomM-owned AND foreign), refusing outright (incomplete_scan) when any bound roms.shortcut_app_id is missing from the submitted live set — a provably partial scan deletes nothing; dry-run mode feeds the QAM's two-tap confirm count |
game_detail.py |
GameDetailService — game detail page data aggregation. Network-free and bounded: for an uninstalled ROM it runs one stat on the computed target path and, only when that is free, one bare readdir through AdoptionCandidateProbeFn for a candidate elsewhere in the folder (#260) — no size-or-mtime stat per entry, since a name match reads neither. An install row answers both questions, so neither runs for an installed ROM. The whole read is offloaded to an executor by the service's own use case: it opens a UoW and touches storage that may be asleep, and every game page opens it |
game_process.py |
GameProcessService — the Stop Game ladder: find the processes Steam started for the ROM's shortcut, ask each to exit once, wait out a bounded grace window, force the survivors. Steam's overlay does not open over a Flatpak game in Desktop Mode, and only the game the ROM's shortcut started may be ended, so the kill is backend-side; the never-re-request rule is a save-safety invariant (see State-aware Resume button) |
controller_focus.py |
ControllerFocusService — the focus watcher: while Steam's 64-bit client runs outside gamescope, pins a focused X window Steam cannot attribute itself (one under a Steam launch without STEAM_GAME) to that launch's app id, hands the choice back to Steam with Steam's own id and 0 once focus leaves it, clears a pin an earlier backend may have left with the same pair at its first answer in a Steam run, sends nothing while no pin stands otherwise, and releases a pin still standing at stop, before the host detaches from Steam's debugger (Controller focus in Desktop Mode) |
playtime.py |
PlaytimeService — session recording into rom_playtime + the rom_playtime_sessions outbox, native play-session ingest (session-end fold+enqueue+flush to /api/play-sessions grouped per stored device_id with a bounded-retry quarantine + pull-only reconcile-on-view that restores total_seconds, session_count, and last_played via monotonic clamps, ADR-0018 / #903); owns the durable kv_config["playtime_scope_notice"] re-sign-in flag (a reconcile 403 sets it, a later 200 or a fresh sign-in clears it) surfaced via get_playtime_scope_notice |
achievements.py |
AchievementsService — progress, caching, RA username |
settings.py |
SettingsService — settings reads/writes, Steam Input config |
data_inventory.py |
DataInventoryService — the Data Management page's own figures: the installs this device holds and the size RomM reported for them, read from the database, and the sealed recovery bundles and their size measured on disk |
rom_removal.py |
RomRemovalService — ROM file deletion + rom_installs cleanup via the UoW; keeps the roms row, playtime, and saves per ADR-0007. Deletion runs under a descriptor-relative no-follow claim whose discipline follows the recovery bundle, not the caller: content-bound whenever a bundle exists to bind the hashes to, identity-only when none does — a user-initiated uninstall, or a cleanup run with recovery off. Only the identity-only form may adopt interrupted staging, and only it leases per unlink rather than over the whole tree — see Removed-game cleanup. Forget this download (forget_download) is the uninstall without the deletion: the same claim and record drop, no file touched, refused with file_present (FilePresent) while the recorded file or folder exists. One removal per rom_id at a time, across all three entry points (uninstall, forget, bulk uninstall): a bulk uninstall reads its install list and claims every rom_id on the loop thread before dispatching its worker, so the single-ROM guard is what refuses a press during a bulk run, and an already-claimed ROM is what refuses the bulk run itself. Both refusals are in_progress; the bulk one carries no removal payload, which is the frontend's existing refusal discriminant. A removal — an uninstall, a forget or a bulk uninstall — whose call is cancelled keeps its claims, and an operation on the prune conflicts, until its worker thread ends, so neither a second removal nor a removed-game cleanup starts against files still being deleted or a record still being dropped. Every mutation of the claim set runs on the loop thread. Start/finish logged with duration; per-file uninstall_progress frames for multi-file ROMs. A single removal refuses not_installed (NotInstalled) and, for a removal that failed the ways a removal can (an OSError, a refused path, a removal the store reported as failed), uninstall_failed; a bulk uninstall that removed only part answers uninstall_incomplete (UninstallIncomplete) with its counts, the bound app ids and the lease. A forget refuses not_installed and in_progress the same way and file_present with the path it found among the refusal's details; it removes no file, so a failed record drop is not a refusal but propagates, like an uninstall's database error. The removed-game cleanup's delete_rom_files never refuses: it answers an InstalledContentRemoval |
prune/ |
PruneService façade — the explicit cleanup of local ROM entries RomM no longer has (decomposed; see Explicit removed-game cleanup) |
prune_leases.py |
PruneLeaseService — the frontend's hold on its leases: renews and releases one by its token, and disowns every lease a gone frontend context left behind. Checks no conflict rule |
running_games.py |
RunningGames — which ROMs' games Steam lists as running: one reading of the host's SteamInterfaceReader.running_apps per call, matched by roms.shortcut_app_id; the save services and the removed-game cleanup ask it (A game that is running) |
cores.py |
CoreService — per-platform emulator-info lookup (get_platform_core_info — the classified emulators list + emulator_data_available, and for an installed game whether its file starts with its chosen entry; get_emulator_launches — the same for each entry, for the emulator menu), per-game core pin/clear (roms.emulator_override), per-platform core write (settings.json platform_cores); neither write touches a shortcut; a pin's LABEL is validated before any write against the options the resolver does not state not installed, and may name a standalone emulator, a libretro core or an entry the resolver gives no complete start command for; see Core and Emulator Selection |
emulator_sources.py |
EmulatorSourcesService — Settings › Emulator sources: lists every detected source (root, health findings by code and data, switch, whether Tender starts games through it, catalogue state, and which source answers) and stores the switch (emulator_sources_off) and the order (emulator_source_order) through the settings' one writer; refuses by raising; see Core and Emulator Selection |
tender_folder.py |
TenderFolderService — Settings › Tender folder and Main's notices about it: the folder's place and state, the drives a place may lie on, a typed place checked and asked about, a choice (which switches the setting and nothing else, and removes an old folder holding only its id file), going back to and forgetting a folder something is recorded under; creates the folder at start, makes it ready for a write (do_prepare_for_write) and answers the folder rule. What is recorded under a folder is each store's own answer (TenderFolderRecordReader); no store answers it yet. The place lives in settings.json (tender_folder), the folder's id in kv_config |
platform_picks.py |
PlatformPicks — the one writer of the per-platform emulator picks in settings.json, used by CoreService and the refiling: replaces the map whole under one lock, saves it, and puts the old one back where the save fails; see Core and Emulator Selection |
platform_refiling.py |
PlatformRefiling — handed each read of RomM's platform listing that keeps or uses it (the Platforms tab's, a sync's, the firmware listing's and the platform system's own, each through the ListingRefiler protocol; the connection test's probe discards the listing and is not one); first has the platform system keep the listed platforms' ids (the ListingKeeper protocol), then moves a download record or a per-platform emulator pick kept under a key that is no listed fs_slug to the platform the listing places it with, where it places it at all (domain/firmware_platforms.py), so an entry already under a listed fs_slug is never moved and a later read moves nothing. Never raises: the keeping, the records and the picks run independently, one refile at a time, the picks written through PlatformPicks, and a failed step is logged here — the traceback of the same failure once per process — and a failed move given back; see Core and Emulator Selection |
platform_systems.py |
PlatformSystemService — the system a RomM platform is in an emulator source, asked with the ids kept in kv_config (platform_ids, read once from RomM where none are kept), which it owns — replaced by each sync's listing (replace_listing, the ListingReplacer protocol) and kept from each listing the refiling is handed (keep_listing) — and the system an installed game keeps from its install record; every reader of a platform's system asks it; see Core and Emulator Selection |
disc.py |
DiscService — the disc picker's two use cases: get_disc_selection (whether a ROM is multi-disc, its launchable discs, the current and default target) and select_disc (pin a disc on the Rom aggregate or clear the pin, which the next start asks with); see Multi-disc selection |
disc_launch_resolver.py |
DiscLaunchResolver — the per-ROM read seam that folds the persisted disc pick over the live enumeration of the install's disc images and answers the path every start is asked with; it never rewrites the install's file_path (see The read seam: DiscLaunchResolver) |
version_switch.py |
VersionSwitchService — the game-detail version picker (ADR-0021). get_version_list(app_id) resolves the bound ROM's sibling group by roms.sibling_group_key (migration-010 index) and merges the local rows with RomM's live get_rom_simple(bound).sibling_roms view — versions not yet synced are marked synced: false; returns each version's markers (active = the bound row, is_default = RomM's default-version flag and nothing else, each version's is_main_sibling from the bound read — the bound version's rom_user, each direct sibling's stub — and the stored flag (a server-only one's fetched detail) for a version that read states nothing about or when it failed, never written to the row; every version whose flag is set carries it, gone, without a file, on another shortcut or in a conflicting match alike, and no version does where RomM holds no default — Tender's resolution chain never stands in; a member outside the bound ROM's direct siblings is one the read states nothing about, so a default version picked in RomM since the last sync can leave the badge on its stored pick there too, installed, switchable, no_file = RomM holds no file for it, the stored column for a local row and the fetched detail for a server-only one, and bound_elsewhere = another shortcut of the game is on it, by domain.sibling_group.target_bound_elsewhere, the predicate the switch's bound_elsewhere guard decides by, set on a server-only version only under a bound row with no group key, read off the version's own local row, which also marks that version synced whether or not another shortcut is on it (#2502 D6) — and the picker disables such a row; and hidden = the user hid it in RomM, from one request per list (RommApiAdapter.read_hidden_rom_ids; none when the server view cannot be fetched), false everywhere when that answer cannot be read, refusing nothing) plus an additive server_query_failed flag when the server view can't be fetched. switchable is the SAME membership authority switch_version decides by (domain.sibling_group.target_in_sibling_group): a local target is a sibling_group_key match (the component keys encode membership, ADR-0022); a server-only target is judged by canonical compatibility — parse the bound key to {source}:{value} and require the target's id at that source to be absent-or-equal (the persisted key doubles as the group's canonical summary). A RomM sibling in a conflicting metadata match — a locally-synced ROM under a different sibling_group_key (#1359) or a never-synced ROM carrying a different id at the bound canonical source (#1360) — is listed but switchable: false, so the picker disables the row instead of offering a switch the backend would reject with not_in_group. A romm: bound key (no metadata source) admits no server-only target; a target that merely lacks the canonical id is in-group and adopts the bound group's key on switch (#1368 / ADR-0022). switch_version(app_id, target_rom_id, allow_stranded) checks the update, migration and prune rules at its entry and moves the group's binding to the target — a pure roms write (the repository's collision-unbind clears the old representative); no Steam call, no name change, no save migration (ADR-0021 §2/§4). Applies to downloaded games too (#1298): on success it returns target_installed; the sticky shortcut's launch options stay its appId, and the next start asks for the target's command. Switching away from a downloaded version whose local saves the local-save check's stranding verdict holds (The Local-Save Check) is a soft block (unsynced_saves, carrying stranding / server_reachable / unsynced_rom_id / unsynced_version_name) so the picker can offer sync-first / switch-anyway / cancel; allow_stranded=true is the "Switch anyway" override, honoured even offline. Guards, each a raised refusal: not_found / download_in_progress (any group member has an active download — cancel first) / not_in_group / bound_elsewhere / unsynced_saves (soft, UnsyncedSaves) / version_vanished (VersionVanished, a proven 404 for the target) / no_file_on_romm (NoFileOnRomm, a target RomM holds no file for that is not installed here) / invalid_target (a server-only target that is a sibling but whose RomM detail the Rom aggregate rejects); what any other RomM error from fetching a server-only target becomes is under Version-switch target liveness. The drift and reachability probes run before any write UoW opens; the write UoW re-checks membership + bound-elsewhere (TOCTOU; ADR-0006). A server-only target is persisted from its RomM detail first via domain.shortcut_data.extract_version_metadata, adopting the bound group's sibling_group_key (not its own coalesce-first key) so the next sync re-canonicalizes the whole component together (ADR-0022) |
active_core_resolver.py |
ActiveCoreResolver — the single per-ROM read seam: resolve_rom(rom_id) reads the per-game DB pick and the per-platform settings.json pick and hands them with the ROM's emulator options to choose_emulator (per-game pick → per-platform pick → the shipped default → no entry, and the start is refused (#2195 D43); no bundled snapshot). A stored label stands while option_for_label finds its entry, whatever that entry's state; active_emulator_for_rom and active_core_for_rom are its projections. Every per-game core read and every start draws from it |
game_start.py |
GameStartService — one start of a game (prepare_game_start): the entry in effect and the game's own path (domain/rom_files.py::asked_path), the resolver's command for them (StartCommandReader), and that command handed to the shortcut launcher for that start only (StartHandover); answers the launch options the shortcut is set to, its appId, or refuses — the game's start beside cannot_start where it has no complete command; see Core and Emulator Selection |
shortcut_removal.py |
ShortcutRemovalService — shortcut removal; unbinds the ROM in roms (keeps the row per ADR-0007) |
shortcut_file.py |
ShortcutFileService — the launch options Steam's shortcuts.vdf holds for a set of shortcuts, read afresh on every call and never written; the fallback of a launch-options confirm Steam's own reports did not settle (see setLaunchOptionsConfirmed in Steam shortcuts) |
shortcut_relocation.py |
ShortcutRelocationService — which of our shortcuts name a launcher outside the launcher's home, handed to the frontend to repoint; a one-time transition with a recorded completion (see Repointing the shortcuts that already exist) |
metadata.py |
MetadataService — ROM metadata reads from rom_metadata (7-day TTL): the per-ROM get_rom_metadata lookup, the paged get_metadata_cache_page(offset, limit) the frontend reads at each panel load (returns {items: {str(rom_id): entry}, total}, page and total read under one short read UoW), and the app_id → rom_id map the launcher resolves session ROMs through. Paging replaced a single whole-cache dump so a large library (5–10k ROMs) never pushes a multi-MB response as a single answer, which the host caps (host/dispatch.py) (#1025); the frontend pages at 500/page until total is reached |
startup_healing.py |
StartupHealingService — logs the rom_installs rows whose file and folder are missing on load, deleting nothing + reconciles orphaned running SyncRuns (a hard crash leaves a running row → marked interrupted) + get_bound_launch_options() lists the bound shortcuts whose launch options in shortcuts.vdf are not their appId, for the reconcile at panel load and after a sync (see StartupHealingService notes) |
leftover_tmp_cleanup.py |
LeftoverTmpCleanupService — removes the partial .tmp / .zip.tmp transfer files a stopped backend left under RetroDECK's ROM folder and in Tender's firmware store, once at start-up, and none in a folder the resolver only assumed |
ps3_folder_naming.py |
Ps3FolderNamingService — the one-time start-up step that renames a PS3 game installed as a folder before downloads named it <game>.ps3 to that name, then lifts a game folder lying one level inside it into its place, each install record following, each of the two steps stamped done in kv_config once nothing it could still move is left (see Launch-target validation) |
connection.py |
ConnectionService — connection test + RomM minimum-version gate + Client API Token lifecycle (mint/establish via credentials, validate/store a user-pasted token, or exchange a short-lived pairing code for a token — the latter two for OIDC accounts; host-bound to the minting origin; see ConnectionService notes) |
update_check.py |
UpdateCheckService — whether a newer Tender release is out: the daily release check, run at panel load and by itself while the backend runs, the per-version dismissal and the daily-check switch, Check now, and the release the last check stored (see UpdateCheckService notes) |
update_install.py |
UpdateInstallService — installing that stored release from the panel: what a press waits for, the download and its digest check, the installer started as a unit of its own and watched, and the update rule every conflicting use case asks (see UpdateInstallService notes) |
update_outcome.py |
UpdateOutcomeService — what the last update did: the version each start records and compares, the one announcement an update that went through is owed, and the installer's record of an update that did not go through — rolled back, or refused by the pre-install check — with its per-record dismissal (see UpdateOutcomeService notes) |
update_output.py |
UpdateOutputService — what the installer printed for one failed update, read back from the journal: which run belongs to the failure the panel shows and, after a rollback, the failed version's own lines (see UpdateOutputService notes) |
protocols/ |
Protocol interfaces grouped by concern (see Protocol Interfaces) |
Version-list liveness¶
VersionSwitchService.get_version_list separates retained local membership from current RomM availability. On each lazy
Game Page load, the bound read (RommRomReader.get_rom_simple, GET /api/roms/{id}/simple) proves the bound id live
and supplies the direct sibling ids; only non-bound local members absent from that direct sibling view are exact-id
probed through RommRomReader.get_rom_once, the adapter's single-attempt short-timeout request_once path on the same
route, concurrently on the existing executor fan-out. Only RommNotFoundError produces vanished: true. All other
probe failures and malformed/falsy responses fail open. No result is persisted.
The response carries vanished on every version and bound_vanished even when multi_version is false. A singleton
also carries its bound_version, which lets a definitively vanished synced binding expose scoped cleanup without
inventing a multi-version picker. Retained vanished rows keep the independently-computed switchable membership verdict
and RomM's default flag. A bound-id 404 is a successful entity verdict (bound_vanished: true,
server_query_failed: false), not a global reachability signal. Artwork and save reads do not contribute availability
evidence; fetch_cover_base64 remains a nullable data endpoint.
Version-switch target liveness¶
VersionSwitchService.switch_version obtains a fresh exact-id verdict immediately before every actual binding move onto
an already-local target. The save-stranding guard runs first, so its initial soft block performs no target request; both
follow-up choices (Sync now retry and Switch anyway) re-enter the service and receive the same liveness guard.
allow_stranded bypasses only save stranding. The active-target no-op, invalid local context, active download,
non-member target, target bound to another shortcut, and a target without a file that is not installed
(no_file_on_romm) all stop before the request because none can enter the binding write.
The local-target request uses RommRomReader.get_rom_once on the worker executor: one attempt, three-second timeout, no
retry-progress event, and no open UoW. Only RommNotFoundError refuses, with VersionVanished (version_vanished).
Timeout, transport/DNS/SSL, authentication, server errors, and malformed or empty successful payloads are logged and
fail open, preserving the fast offline switch path. The existing short write UoW still rechecks membership and
bound-elsewhere after the request; the request narrows the race but cannot form a transaction across RomM and SQLite.
A server-only target still needs its full detail before it can be validated and persisted. That mandatory fetch keeps
the normal retry policy and is not preceded by a redundant exact-id probe; only its typed 404 is peeled into
version_vanished, and any other RomM error from fetching a server-only target passes on unchanged; Endpoints answers
it with classify_error's reason. A refusal performs no binding, row/install/applied-launch-options write,
launch-command resolution, event, cache invalidation, sync, or completion-stamp update.
Explicit removed-game cleanup (services/prune/)¶
| Module | Role |
|---|---|
service.py |
Use-case facade, atomic preview/run admission, and claimed frontend action leases |
executor.py |
Serial per-group liveness, recovery, Steam-action, reconciliation, and finalization state machine |
preview.py |
Local candidate snapshot from generation-gated stamps and the gone-from-RomM marks (the sync's and the cleanup's), complete affected-group disclosure, sizing, and paging |
recovery.py |
Lossless aggregate snapshots, recovery artifact assembly, and sealed-state comparison |
registry.py |
Short SQLite reads, action/final race validation, reconciliation, and final cascade delete |
requests.py |
Preview/option decoding, bounded selections, and lossless bounded Steam snapshot validation |
PruneService is the only path that deliberately deletes retained roms aggregate roots. A bulk preview asks RomM one
thing, the ROMs the user hid, and never offers one of them (#2502 D5); the rest is local: for each platform it requires
a non-empty completed fetch generation and selects rows whose last_fetch_id differs, including NULL row generations.
An inline preview offers the ROM it names without generation evidence, read as gone because the version picker offers it
only after RomM answered it with a 404, or every version of that ROM's game where the bulk rule finds all the others
gone too. Both forms return pages of at most the requested row limit under an ephemeral preview id, whose row
fingerprint the backend keeps and compares at start; the panel fetches them one after another until it holds every row.
Pages include every member of an affected sibling group, with the offered rows marked separately. A run removes only
offered rows: it selects those both offered and proven gone (domain/prune.py::selected_prune_ids), and a game found
gone whole that holds a row the preview did not offer is skipped. A game the preview offered whole that has a version
back on RomM is skipped too, so it never loses part of itself
(Removed-game cleanup). start_prune consumes a finalized preview-bound
installed-content selection. The frontend stages that selection a page at a time, so the per-call page limit does not
cap the total selected set. Start atomically refuses any registered conflicting use case and reserves the run before
rebuilding the preview; concurrent starts cannot consume one token twice, and shutdown owns/cancels an admitted refresh
before it can spawn a run. Each sync, download, migration, version-switch, save-write, session, uninstall,
connection-identity change (including a successful connection test), or cache-mutation use case registers for its full
lifetime before its first await. Detached status, download, and playtime tasks transfer the claim to their task
lifetime. Core/disc writes, launch evaluation, and Steam Input application are included because they mutate recovered
state. Frontend-owned shortcut removal, version switches, SGDB/icon application, startup healing, the start check, and
every post-sync Steam branch (launch options, collections, playtime, and overview metadata) hold tokenized conflict
leases through their final Steam write and bounded release. Active continuations heartbeat those leases once per minute.
A global frontend registry signals cooperative cancellation by owner. A continuation with an owner captures that owner's
mount generation once, when it starts, and carries it through every backend wait; the owner's teardown tombstones its
generation synchronously, so a lease-bearing response that arrives afterward is released without running its
continuation. Only a genuine remount opens a new owner generation. Cancellation stops every not-yet-started Steam
mutation and lease renewal, but explicit backend release waits for any already-started non-cancellable Steam promise to
settle. An unresolved operation stops renewing after a bounded five minutes; the backend's five-minute no-heartbeat
expiry is the abandonment backstop if it never settles. The Play button carries the admission captured at its press
through every gate, modal, and launch-options confirmation wait. The version picker likewise rechecks its captured owner
admission after save-sync and modal waits before any successor switch_version mutation, so an unmounted chain cannot
resume under a new picker. Each non-empty sync_stale event carries its own lease through the paced removal tail; a
later sync_complete lease overlaps and joins that same promise, so success composes both leases while a post-stale
backend failure still leaves the tail covered. Event delivery failure releases a token that never reached the frontend.
This closes the reciprocal start/refusal race, and each path refuses while a prune claim is active. The update,
migration and sync rules additionally guard preview and start, checked in the prune service; the start asks them only
once its reservation is held.
The executor processes sibling groups serially and catches ordinary exceptions per group. It rejects multiple shortcut
bindings, active downloads and a group whose game Steam lists as running
(A game that is running), pins the preview's canonical RomM
origin/token-origin/user namespace, probes every local group member with the single-attempt three-second get_rom_once,
and treats only a typed RommNotFoundError from that same namespace as destructive authority. A connection/server/user
change before, during, or after an exact-ID request is uncertain. Live, malformed, wrong-id, transport, authentication,
timeout, server, and unknown outcomes retain data. Long recovery and frontend round trips are followed by another
exact-ID proof before local source removal. The cleanup never moves a shortcut onto another version: a bound vanished
version whose game still has a live version is kept with its shortcut and row.
Recovery is coordinated through ordered ROM save locks, acquired with no UoW open. The lock set is retried until shared save ownership is stable. Short read UoWs close before filesystem work, and no RomM request, event emission, or frontend wait runs under those locks. The bundle is sealed before mutation. Aggregate, save-inventory, bundle/source, controller, and fresh frontend Steam state are validated before the irreversible Steam action. After that action the service reacquires the same recovery-set locks and repeats the guards immediately before local finalization; backup ownership is separately projected only for rows being deleted. The locks remain held through the save backup, filesystem-only installed-content removal, and the final parent cascade. Source sets record expected presence or absence, the root's sealed no-follow identity and regular-file hash, and every descendant identity, mount ID, and regular-file hash. Every resolvable exclusive save path is represented even when it was absent. Traversal refuses nested mount transitions. After the anchored root rename, deletion obtains kernel read leases for every regular root/descendant before removing any entry and holds them through the last descriptor/hash check and unlink. An existing writer, an unsupported lease, or any inability to establish exclusion restores/retains the source rather than deleting bytes not in recovery. The controller VDF's held claimed inode uses the same writer exclusion from its final identity/hash validation through claim unlink on both normal publication and collision/rollback discard; setup failure preserves the claim and teardown failure after unlink is an ambiguous mutation. Content excluded from the bundle and recovery-off saves still receive a final complete no-follow claim; their remove/backup uses anchored parents and atomic no-replace rename rather than a raw path fallback, so a concurrently created backup is never overwritten. Writer-exclusion teardown faults after mutation are reported as ambiguous, not exact success. Controller rewrites revalidate the held claimed inode before every restore/discard branch and retain a newer claimed inode at a surfaced path when a concurrent Steam file wins publication. Every exclusive current-save path is expected absent after the backup, regardless of whether it existed during inventory, and the whole set is collectively rechecked after all filesystem work and immediately before the aggregate cascade. Validation and claim decoding share one held bundle descriptor and return a digest that every later guard compares to the cached claims. Recovery root, staging, bundles, destination files, and the sealed bundle remain descriptor-anchored through copy, metadata application, hashing, cross-directory rename, validation, and both-parent fsync. Failure cleanup uses the same mount-aware descriptor-relative remover; if cleanup cannot prove a safe tree, it leaves and reports the full anchored staging path instead of recursively entering uncertain data. Adapter outcomes carry actual and durability-ambiguous changes into the mutation ledger even when a later item or parent fsync fails. The final SQLite delete is one short UoW that revalidates complete row/binding state and invalidates intersecting collection stamps; platform stamps are deliberately preserved (removed-game-cleanup.md).
Frontend Steam events use a claim/complete protocol. A claim checks the run, token, discriminant, appId, exact
single-binding group, and current binding before the frontend rechecks the live tender-rom-launcher executable and
mutates Steam. The lease is monotonic-clock bounded and rechecked after asynchronous validation; an identical repeat
claim is idempotent, while mismatched or expired claims cannot authorize a mutation. Shortcut removal is immediately
reconciled to an unbound local row after Steam confirms absence. A claimed action whose completion is lost, or a
RemoveShortcut that was attempted but could not be confirmed, is an explicit ambiguous partial. A pre-mutation refusal
remains an ordinary failure. A later run can confirm a shortcut is already absent and reconcile without repeating Steam
removal. A later guard failure is likewise an explicit partial result with the committed action and actual mutation
categories recorded, not a claim that the group was unchanged. Cancellation can stop final guards and every later group;
shielding starts only with the first irreversible local mutation and awaits that phase to a known result. Cancellation
remains authoritative if that shielded child faults: the current group's truthful fault/ledger result is recorded and no
later group starts. A terminal result's ROM-id and warning lists are capped per group beside their full counts, its text
is sent whole, and chunks are built to a serialized byte budget. Every action/progress/completion frame carries the
originating preview ID, allowing the frontend to adopt a matching run even if the successful start response is delayed
or lost while still rejecting foreign frames; completion finalizes only a contiguous chunk sequence. An accepted
contiguous terminal sequence seals that run against every later action, progress, or completion frame, and the modal
exposes its terminal controls immediately even if the start response is still pending. Mixed runs continue unrelated
groups.
LibraryService decomposition (services/library/)¶
The library sync subsystem is a façade over the sub-services below, which coordinate through a shared
LibrarySyncStateBox. bootstrap/library.py builds the box and every sub-service and hands the façade the ones it
calls (wire_library, which answers the service and its parts as LibraryParts), so the façade builds none of them:
| Module | Role |
|---|---|
service.py |
LibraryService façade — public use cases; checks their conflict rules and delegates to the parts it is handed |
fetcher.py |
LibraryFetcher — read-only RomM roundtrips: list platforms/collections, per-unit work-queue construction (a collection unit carries its listed rom_ids), a platform unit's incremental/full pagination loop. The outward half of the read pair below |
collection_fetcher.py |
CollectionFetcher — a collection unit's ROMs, for all three kinds: the collection incremental skip against its CollectionSyncState stamp, the registry rebuild of a skipped collection's members, and otherwise the walk narrowed to the platforms the sync leaves off (a foreign smart collection walked whole); and, for an apply, every collection unit of the run fetched in queue order before the first applies, each with the clock read before its own skip check; see "Incremental skip" below |
local_library_reader.py |
LocalLibraryReader — the inward pair of fetcher.py: what THIS DEVICE recorded about the library, read back out of SQLite (the Rom rows and their bindings, the completion stamps, the persisted sibling keys, the last finished run), shaped into the projections a run decides against and the reachable set the collections listing counts against. Declared read-only (scripts/check_read_only_module.py) |
sync_orchestrator.py |
SyncOrchestrator — the run's lifecycle and control (start, apply a staged preview, cancel, the heartbeat route, shutdown), the run loop that hands each unit to the unit applier (and, at the first collection unit, has it fetch every collection unit of the run), the finalize (the stale removal and the Steam collections), and which terminal status a stopped run earns. The one part of the sync that starts a task |
unit_apply.py |
UnitApplier — one unit's apply: a platform unit's fetch and component keys, or a collection unit's delivery, the shortcut build, the sibling-group collapse and the delta, the covers through the cover preparer, the platform stamp's DELETE at apply start, and the hand-off to the dispatcher; and, before the first collection unit applies, the component keys over every collection unit's delivery and the gathering of each group no binding holds (see "Group-aware sync" below). It reads the clock for a platform unit's fetch_started_at |
sync_preview.py |
SyncPreview — the read-only preview (every unit fetched into one union, collapsed and classified as the apply's units would), the snapshot it stages for the apply, the read that hands it back to a remounted panel, and its cancel. It claims and releases its own run slot through the box's verbs |
sync_progress.py |
SyncProgress — the sync_progress frames: emit_progress (every part that narrates a run is handed SyncProgress or its emit_progress), the CANCELLED and ERROR terminals, and the get_sync_status read a remounted panel re-seeds from |
chunk_dispatcher.py |
ChunkDispatcher — one unit's apply as durable chunk round-trips: emit sync_apply_unit, wait out the heartbeat clock for the ack, commit the chunk through the reporter, and stash or discard a chunk whose ack never came. It opens no transaction, and the run's try_begin_run / finish_run stay with the orchestrator and, for a preview, with the preview |
cover_preparer.py |
CoverPreparer — one unit's covers, readied before its shortcuts are emitted: the refresh_changed_covers invalidation pass, the download for the ROMs getting a shortcut, and each emitted entry's cover_path. Delegated to ArtworkService through the ArtworkManager seam, with the run's progress and cancel signals bound in |
sync_run_recorder.py |
SyncRunRecorder — the run's own SyncRun row: the running row opened with the planned counts, and the single terminal transition (completed / cancelled / interrupted / paused / errored) that closes it, each in its own short write UoW. It writes the outcome; which terminal a stopped run earns stays the orchestrator's, and admitting or releasing the in-flight run stays _state.py's |
reporter.py |
SyncReporter — the apply's ack and each chunk's commit (artwork filenames, the per-unit roms upsert with its cached metadata, the completion stamps and the applied shortcuts), and Force Full Sync's reset of those stamps and commands. It emits nothing |
run_finalizer.py |
RunFinalizer — the end of a run: the closing registry write (the stale removal's skip revocations, the collection-stamp deletions, the gone-from-RomM marks, the platform-names cache) with the Steam-collection maps built in the same write UoW, and the terminal signals (sync_complete and the run's last sync_progress frame) |
views.py |
LibraryViews — the façade's reads: the roms-derived platform counts, and the sync_runs history and the completion stamps behind the last-sync line, the resume offer and the run list. Declared read-only (scripts/check_read_only_module.py) |
session_budget.py |
SessionBudgetMonitor — Steam's per-session renderer-heap budget: the GC-settled RSS measurement, the chunk-boundary pause verdict, the headroom clip that trims a chunk's additive cover work to what that verdict left, the post-preview prognosis, the run-start baseline, the get_session_budget_status payload, and whether a reading stands at or above the ceiling, which the shortcut artwork job asks through LibraryService.renderer_above_ceiling before each batch |
_state.py |
LibrarySyncStateBox — shared mutable in-flight sync state; single source of truth threaded through every sub-service, and the sole owner of the run-lifecycle pair (sync_state / current_sync_id) and of the staged preview snapshot (pending_delta, TTL and the run-in-flight withholding included) via its verb methods |
Two modules hold the artwork seam, and only two. cover_preparer.py holds it for the apply path's covers (the
invalidation pass, the download, each emitted entry's cover_path); reporter.py holds it for commit-time cover-path
finalisation (finalize_cover_path). Those are different questions asked at different points of a unit, which is why
the confinement names a pair rather than one owner — scripts/check_seam_owner.py enforces the pair, so a third holder
fails CI instead of quietly reopening the spread the cut closed.
The pipeline is split fetch (read-only) / apply (owns persistence): neither fetcher ever mutates the roms registry
or rom_metadata, and the reporter's per-unit commit upserts each acked ROM's roms row and stamps its cached
rom_metadata in the same write Unit of Work (Rom row first, then metadata — FK-safe). So a preview never mutates
state, and an interrupted apply leaves only the units it already committed — incremental, per-unit delivery.
Where the synced-ROM state lives. The registry of synced ROMs, the last-sync timestamp, and the sync stats are
SQLite, not JSON. The reporter upserts each acked ROM into the roms table via Rom.synced(...) / update_cover_path
/ assign_sgdb_id (artwork and steamgrid patch cover_path / sgdb_id on the same aggregate during the per-unit
commit) and, in the same write UoW, stamps the ROM's cached rom_metadata (build_rom_metadata maps the live RomM
metadatum — Rom row saved first so the rom_id FK holds); SyncRunRecorder writes the SyncRun row (start at
apply-dispatch, complete / mark_cancelled / mark_interrupted / mark_paused / mark_errored at finalize), with
the orchestrator deciding which of those terminals a stopped run earns. The terminal sync_complete event (and its
progress frame) is emitted LAST — RunFinalizer.emit_sync_complete, called by the orchestrator only AFTER that terminal
SyncRun status is persisted — so a frontend stats refetch triggered by the event reads the fresh run status instead of
racing the write (the #39 lag); RunFinalizer.finalize_per_unit_run keeps only the stale removal's stamps +
sync_collections emit and returns the app-id maps. The rows a run removes keep their binding until the panel reports
their shortcuts gone (report_sync_removals, #2653; Steam Non-Steam Shortcuts, "A
removal the panel did not finish"). sync_stats.roms is a registry-derived bound-shortcut count computed at read time
(the ROMs still bound to a shortcut in roms, i.e. shortcut_app_id not NULL), not a stored scalar. The old JSON
shortcut_registry / last_sync / sync_stats are gone from this path; all writes go through the roms / sync_runs
Repository Protocols behind a narrow Unit of Work (per
ADR-0006 the UoW
spans only the DB write, never the up-to-60s frontend ack). The platform fs_slug → name map resolves live from RomM
each sync and is cached in a kv_config row for offline reads. Removing a shortcut unbinds the ROM
(Rom.unbind_shortcut() NULLs shortcut_app_id, keeping the row and its per-ROM children) rather than deleting it, per
ADR-0007. The
full schema and aggregate model are in Database Design.
Delta-restricted apply: emit only new + changed
(ADR-0025).
Between the sibling-group collapse and chunking, UnitApplier.apply_unit runs domain/sync_diff.py::classify_roms over
the unit's collapsed entries against the bound-shortcut registry and emits only the delta — new + changed, a rebind
entry counting as changed because it moves its binding (outside a unit the run skipped, whose apply moves none).
"Changed" is content, not identity-only: on top of the identity triple (name / fs_name / platform_slug) the
classifier reads whether the item's roms row records its shortcut applied (shortcut_applied). An applied row whose
identity matches is content-unchanged and skipped — it never reaches the frontend, so no Set* walk and no ~2 s
AppDetails confirm poll. Every bound shortcut carries the same launch options, its appId
(#2221 D7, D10), so an install, an uninstall or a core or
disc pin changes nothing the apply writes. A row not recorded as applied (a freshly created row, or one Force Full Sync
cleared) is never skipped, so it is re-applied once, recorded, and only later syncs skip it. Skipped rows are still
committed (chunking routes their groups to chunk 0's leftover), so no DB work is dropped and the per-platform
completion stamp still rides the final chunk only — a platform is stamped exactly when its whole delta is durable, and
an empty-delta platform emits one empty chunk that commits every row and writes the stamp. The record is set by the
sync's commit alone, for each binding target the frontend acknowledged (D17); a row it misses only ever causes a
harmless spurious re-touch, never a wrong skip (the benign-failure asymmetry). The per-unit sync_apply_unit's
unit_total is therefore the delta size, so the progress counter shows net progress and a resume converges quickly.
The preview does not count a skipped unit's launch commands. sync_preview classifies its whole union with the same
classify_roms, and passes as skipped_rom_ids the ROMs it took from a unit the fetcher reported as skipped — platform
and collection units alike. Apply returns before it classifies such a unit, so it never rewrites those shortcuts and
never records their launch commands. A built launch command there that differs from the recorded one, a NULL recorded
value included, is therefore counted unchanged: counted as changed, it would show the same updated games on every
preview until a change on the server or a Force Full Sync ended the skip. An identity difference on such an entry still
counts as changed. Steam's copy of an installed game's command is kept current by the launch-options reconcile instead
(StartupHealingService notes). The summary's changed_count, unchanged_count,
changed_names and platform_breakdown, and the pause_likely prognosis, all come from that one classification.
Cover-cache invalidation: the cover_source fingerprint (#1386). The per-ROM cover cache is only valid while the
server's cover is unchanged, and the delta apply never re-downloads a skipped ROM's cover — so each roms row records a
cover_source fingerprint: the full RomM cover source string (path_cover_large else path_cover_small, the embedded
?ts=… cache-buster included) whose bytes the cache file holds, compared as an opaque string (catches both a ts bump
and a path change). Two compare layers run per unit, both in ArtworkService:
- Apply-path covers (
_resolve_cached_cover): a stored fingerprint that differs from the fresh one blocks both the cache-hit reuse and the grid→cache seed, forcing a fresh download. A bound ROM whose cover is set in Steam (the rule is "Set in Steam" under Steam Non-Steam Shortcuts) is left out of the apply's cover download altogether: nothing is downloaded for it, the commit copies nothing over its{app_id}p.png, and its fingerprint is not advanced (#2381 D4, D9), and so is a cover the user chose from SteamGridDB in the ARTWORK tab (D1). The rule reads the ROM already bound to the shortcut, so a sync that moves the binding to another version of the game writes that version's cover over a hand-set one, as a version switch does (D8). An automatic cover that is Tender's SteamGridDB pick, given while RomM had none, is left out of the download as well: only the invalidation pass replaces it, so it is never seeded from the grid file (D24, D27). - The invalidation pass (
refresh_changed_covers, reached throughCoverPreparerfromUnitApplier.apply_unitbefore the cover download): for every BOUND fetched ROM whose stored fingerprint differs — delta-skipped ROMs included — it re-downloads the cache file (atomic tmp+rename), republishes the grid{app_id}p.pngcopy, persists the new fingerprint in a small write UoW (an observed server fact, deliberately NOT ack-coupled likeshortcut_applied), and collects{rom_id, app_id}. An automatic cover that is Tender's SteamGridDB pick counts as changed whenever the fetch carries a RomM cover, whatever its fingerprint; it is downloaded whole, and its record and hand-over go once RomM's cover is published (services/artwork/_steamgriddb_cover.py, #2381 D24). A changed cover that is set in Steam, or that the user chose, is skipped with its cache and its fingerprint as they are (_covers_left_alone_io), so RomM's change stays pending (#2381 D9). A later sync does not reliably fetch it: once the platform's apply completes, the platform skip rebuilds its rows without their covers until something else on it changes. Setting the cover back to automatic in the ARTWORK tab reloads RomM's cover at once (refresh_cover_unchecked, underset_artwork_slot_automatic's rules), and Refresh Artwork reloads it while the cover is automatic (refresh_cover, which itself does not ask; the frontend asks the tab's slots first). The pass scans against the unit's bound-row registry projection (LocalLibraryReader.do_read_apply_registry, which carriescover_source) — the same read the group collapse diffs against, so it opens no per-ROM DB lookups. That list rides the unit's firstsync_apply_unitchunk ascover_refreshes, clipped to the session-budget headroom left after the chunk's own projected cost (each refresh ≈ one transient cover,COVER_TRANSIENT_KB) — clipped, never pausing the run: the grid files are already updated, a Steam restart shows the rest. The frontend re-applies each entry viaSetCustomArtworkForAppat the 50 ms cadence before the chunk ack — without that push the tile stays stale in-session, sincert_custom_image_mtimeonly bumps throughSetCustomArtworkForAppor a client restart. - The preview-side count (
sync_preview→CoverPreparer.count_cover_refreshes→ArtworkService.count_cover_refreshes):classify_romsis deliberately cover-blind (ADR-0025), so a cover-only server change yields an empty shortcut delta — and the QAM's preview flow used to short-circuit on "no changes", meaning the apply (and with it the invalidation pass) never ran and the fingerprint stayed stale forever. The preview therefore counts the fingerprint mismatches itself, with the SAME pure kernel the apply pass scans by (scan_cover_refresh_candidates— shared viadomain/cover_refresh.py), as an in-memory compare of the fetched union (platform and collection units alike) against the registry projection the classify already read, and leaves out the covers set in Steam or chosen by the user through the same reads the pass makes (services/artwork/_cover_set_in_steam.py: one read UoW over the changed covers' recorded outcomes and hand-overs, the grid file and the cover cache of each; and a second short read UoW over theirartwork_slot_choicesrows). Preview count and apply set agree while Steam's grid directory does not change between the two. No downloads, no writes. The result rides the summary as the additivecover_refresh_count(absent/0 tolerated by old consumers); the frontend treats a cover-only preview (all diffs zero, count > 0) as apply-able, showing "No shortcut changes — N cover updates" with the normal Apply/Cancel confirm. NULL-fingerprint rows are not counted: their adopt-vs-download fork needs a cache-file check the preview must not perform, and the adopt is invisible to the user — except a cover that is Tender's SteamGridDB pick, which counts whatever its fingerprint (D24). The wholesale platform-skip gate needs no cover awareness to fetch a cover change — the change bumps the ROM'supdated_at, which already forces the fetch; a skipped platform's registry-reconstructed thin ROMs carry no cover fields and count 0. So a change the pass skipped because the cover was set in Steam is not fetched again by a later skipped sync (above). - The empty-unit chunk vehicle: a cover-only unit has an empty apply delta, but
build_unit_chunksyields exactly one empty chunk for it (ADR-0023's empty-unit round-trip), so thecover_refresheslist still rides async_apply_unitframe withshortcuts: []; the frontend processes the refreshes and acks the empty chunk, and the commit advances the unit normally. No special-casing exists on either side.
A NULL fingerprint (every pre-migration-016 row) with an existing cache file is adopted: the fresh fingerprint is
persisted without a download, so the upgrade never mass re-downloads a library; a NULL with no cache file downloads on
the apply path as before. The fingerprint is only ever advanced when the cache is actually confirmed against the server
— a fresh download, a cache reuse/seed (the reporter's commit merges the confirmed value from the unit's
pending_cover_sources staging, else preserves the existing one), or the NULL-adopt — so a failed download keeps the
old fingerprint and the change is retried next sync. A cover-only change never re-applies the shortcut itself (that
apply path writes launch options under the ADR-0025 invariant — pure churn for a cover).
url_cover fallback on a 404 RomM asset (#1450). When the RomM-local cover asset returns an HTTP 404 (the server's
own cover resources are missing while its web UI still renders from url_cover), the cover download retries once
against the ROM's url_cover — an external metadata-provider CDN (SteamGridDB / IGDB / …) — before giving up. The retry
lives in ArtworkService._download_cover_atomic: only a RommNotFoundError (404) with a non-empty url_cover triggers
it, so a transient transport error keeps today's retry-ladder behaviour and never falls back, and an empty/absent
url_cover is exactly today's failure (warning, gray tile). The cover routes are exactly why the entity-verdict rule in
what makes a 404 an entity verdict carves them out: a
static-mount miss answers with the generic route-404 body, so demanding an entity answer there would disarm this
fallback. The fallback fetch goes through a separate, bearer-free adapter seam
(RommRomReader.download_cover_from_url → RommHttpAdapter.download_external): the host-bound RomM bearer must never
reach a third-party origin, so only Tender's User-Agent is attached (the CDN behind Cloudflare Bot Fight Mode also
403s the default Python-urllib UA). Because url_cover is untrusted server-supplied input, download_external
scheme-allowlists it to http/https before any fetch — a file:/data:/ftp:/scheme-relative URL is rejected with
a RommApiError so a file:///etc/passwd never reaches urlopen (nothing revalidates the origin on a redirect hop,
here or on the RomM download paths — #1889 — and nothing blocks a private or link-local address after DNS). The
fingerprint records the source actually applied — url_cover on a fallback, threaded back through the
download_artwork applied_sources accumulator (sync path) or the direct persist (refresh_changed_covers /
refresh_cover) — so the compare stays truthful: because the fresh RomM path_cover string never equals the stored
url_cover, a later fixed RomM asset (or a changed url_cover) is always re-checked, at the cost of re-downloading the
fallback ROM's cover each sync until the RomM asset is repaired.
Conditional-request revalidation on a ts-only change (#1454). The ?ts= in a RomM cover path is the ROM row's
updated_at (path_cover_small / path_cover_large in RomM's backend/models/rom.py), which every update of the row
moves (onupdate, backend/models/base.py, and update_rom always writes). Which scans move it without touching the
cover file is RomM's, read from its source at 5.3.0 and unchanged on its master at ca2a36ca5:
- A quick scan — RomM's default, and what its scheduled scan and its file watcher's per-platform rescan run — sends
every existing ROM through
refresh_rom_files(backend/handler/rom_files.py), which callsupdate_romonly when what it reads from disk differs from the stored row, so an unchanged ROM's?ts=stays. The exception: every ROM of a platform whose folder is missing is re-stamped on each scan (mark_missing_platforms). - An Update or Unmatched scan, a metadata refresh of chosen games, and the watcher's library-wide update
re-stamp every ROM they cover (each goes through
download_rom_resources→update_rom) and keep its cover file unless the cover's source URL changed (get_coverwrites over the file only then). This is the case the revalidation exists for: across everything such a scan covers, thecover_sourcefingerprint changes while the bytes do not, and the #1386 compare alone would download each of those covers again. - A Complete scan deletes every cover and downloads it again, so the file changes too.
So when a fingerprint change is ts-only (the paths are equal once the ts query param is stripped —
domain/cover_refresh.cover_ts_only_change) and a stored validator exists, the cached bytes are revalidated with a
conditional request rather than re-downloaded: every regular download records the response's HTTP validator (ETag /
Last-Modified) in a {rom_id}.cover-meta.json sidecar beside the cache file, and a ts-only change re-requests with
If-None-Match (else If-Modified-Since) through the normal authenticated RomM adapter path
(RommRomReader.download_cover → RommHttpAdapter.download_conditional, bearer + UA as today). A 304 Not Modified
keeps the cached bytes, adopts the fresh fingerprint (so the next sync is clean — this is the whole point), and
refreshes the validator if the 304 carried one, with no grid republish and no in-session tile re-apply (the tile
is already current); a 200 replaces the bytes, validator, and fingerprint. Validators are an optional capability
— a ROM with no stored validator (first sync, or a server/proxy that strips them) plain-downloads and seeds one for next
time, and a conditional-request transport error is exactly today's failed-download path (retry ladder, old bytes + old
fingerprint kept). The sidecar lives beside the cache file (pruned with it on shortcut removal and by the orphan sweep);
the #1450 url_cover/CDN fallback path is out of scope for revalidation and keeps its once-per-sync re-fetch. No RomM
minimum-version change — the floor is untouched.
Unstamped-platform re-run: the restamp_platform_count signal (#1416). A heartbeat-timeout run's late-ack recovery
(ADR-0023) leaves a platform complete but unstamped: the timed-out apply cleared the stamp at its start and its late
ack re-binds the chunk without re-writing it (the late-ack path never passes a platform_stamp). The wholesale-skip
gate then full-fetches that platform on every future sync (no stamp = no skip authority), and — because its shortcut
delta is otherwise empty — the QAM's preview would short-circuit on "no changes", so the re-walk that would re-stamp it
never runs and the run's interrupted status lingers indefinitely (only an apply run records a fresh SyncRun).
sync_preview therefore counts the enabled platforms holding no PlatformSyncState stamp that may skip — no stamp at
all, or one whose skip was revoked ("Incremental skip" below) — and the platforms it fetched in full whose stamp does
not match that fetch (no checksum or fetch start, other ids, or a ROM updated after the stamp's reference;
domain/platform_skip.py::restamp_due, #743), through
LocalLibraryReader.do_count_platforms_to_restamp, a side-effect-free read, and rides it as the additive summary field
restamp_platform_count (absent/0 tolerated by old consumers). The frontend treats a restamp-only preview (all diffs
zero, count > 0) as apply-able — "No shortcut changes — recording what RomM has now." with the normal Apply/Cancel
confirm — mirroring the cover-only flow. One sentence serves every cause and both counts, because the apply does the
same for each: its skip gate refuses the platform as the preview's did, so the apply fetches it in full, and the 0-delta
empty final chunk writes a stamp of what that fetch returned and records a fresh completed SyncRun — so the platform
skips again, unless a ROM's updated_at is still inside the margin at that fetch's start ("Incremental skip" below),
and a run status left interrupted by a late-ack recovery heals. The stamp write stays pipeline-owned: the preview only
counts, it never stamps (check_sync_lifecycle_owner / the final-chunk-only rule). Collections owed a fresh stamp ride
a count of their own, restamp_collection_count, which the frontend reads the same way; what it counts is under
"Incremental skip" below.
Per-unit apply is chunked, durable per chunk
(ADR-0023). The delta
emit list is split into fixed-size chunks (_APPLY_CHUNK_SIZE, 200) by the pure
domain/sync_chunking.py::build_unit_chunks, and ChunkDispatcher (services/library/chunk_dispatcher.py) processes
them one at a time: emit sync_apply_unit → wait for the ack → commit that chunk's roms rows durably → next chunk.
Each sync_apply_unit carries chunk_index / chunk_count / chunk_offset / unit_total, and shortcuts is the
chunk slice; the reporter's per-chunk commit is the same group-aware two-pass write (every fetched sibling upserted,
only representatives bound; Rom row then rom_metadata, FK-safe) over that chunk's row subset, so a committed chunk is
crash-safe on its own and a mid-unit failure forfeits only the in-flight chunk. Chunks cut only at sibling-group
boundaries (overflowing 200 to keep a game's dumps whole so they never straddle two commits); no-emit groups and
unmatched leftovers ride chunk 0, and an empty unit is one empty chunk (the empty round-trip still commits its unbound
rows). The whole-unit staging (pending_sync / pending_all_roms / pending_cover_sources / pending_followed_keys)
is set once by the dispatcher, before its first chunk goes out; only the per-chunk coordination is re-armed each chunk.
Every chunk's commit writes the followed keys (below, "Group-aware sync"), so the first commit that writes a member of a
game writes the key of the row its listing left out as well. The motivating field crash is
#797: a 3084-shortcut unit emitted in one frame lost ~24
minutes of work when steamwebhelper OOM-crashed before any ack — a 200-chunk caps both the frame payload (~200 KB vs
~3 MB) and the crash blast radius (~2 min vs 24+).
Per-chunk wait: timeout vs. cancel. The dispatcher emits each chunk's sync_apply_unit, then waits on
unit_complete_event (heartbeat-clocked) for the frontend's report_unit_results ack. When the wait returns None the
teardown branches on the cause (#1052) — and either way, every chunk committed before this one stays committed:
- User cancel (
is_cancelling()already True at the return) — the in-flight chunk is intentionally discarded: clear the staging, nullunit_complete_event, and clearactive_unit_id+active_chunk_index. A stray late ack then no-ops. - Heartbeat timeout (still RUNNING) — the frontend has already created this chunk's Steam shortcuts and will fire a
late
report_unit_results, but in production after the run has already wound down. The dispatcher moves the abandoned chunk into anabandoned_chunkstash on the box (stash_abandoned_chunk): its run/unit/chunk identity plus this chunk's ROMs (only the abandoned chunk, never the whole unit, themetadatumsource), while it keeps the whole-unit staging (pending_sync/pending_all_roms/pending_cover_sources/pending_followed_keys) live for the recovery commit to read and clears the dispatch identity (unit_complete_event+active_unit_id+active_chunk_index). It then marks the runinterruptedand flips CANCELLING so the loop stops. The stash lives outside the run-lifecycle state and deliberately survivesfinish_run(which nullscurrent_sync_id), so a late ack arriving after teardown can still recover it (#1367 — an earlier design keptactive_unit_idlive and lost the ack once the run wound down).SyncReporter.report_unit_resultsthen matches the stash by identity (take_abandoned_chunk) and drivescommit_unit_resultsitself over the stashed rows — every fetched sibling upserted (identity + metadata), only the acked representatives bound, and never aplatform_stamp(a timed-out platform is incomplete) — persisting the delivered bindings instead of leaving orphan shortcuts that the next sync re-creates as duplicates. The stash has a bounded lifetime: the next run'stry_begin_runclears it, so a frontend that crashes and never acks just leaves inert data. The committed binding is mapped by the next sync's existing-shortcut scan, so no active orphan deletion is needed (a Steam shortcut is the sole record of its tile).
Session-budget gate: GC-before-measure at each chunk boundary
(ADR-0024). Steam's
SharedJSContext renderer OOM-crashes at ~2.45–2.53 GB RSS and never self-recovers within a session; each created
shortcut costs 0.7–1.5 MB permanently (measured on-device). Chunking (ADR-0023) makes hitting that cliff a cheap resume;
this gate stops the run before it, at a chunk boundary, as a controlled pause. Every renderer reading and verdict
below belongs to SessionBudgetMonitor (services/library/session_budget.py), which the chunk dispatcher holds
directly — the fakeable seams are the two renderer adapters below, not the monitor. At each chunk boundary the monitor
reads the renderer's RSS (RendererRssFn, adapters/renderer_rss.py — the max VmRSS across steamwebhelper
processes from /proc) and, when that raw reading is at/above GC_SKIP_BELOW_KB (1.5 GB), forces a renderer GC
(RendererGcFn, adapters/renderer_gc.py — an HeapProfiler.collectGarbage over the CEF debugger on localhost:8080,
so the reading reflects settled heap not transient garbage that Steam's measured-unreliable natural GC hasn't reclaimed)
and re-reads; below the floor the raw reading (which can only over-estimate) already clears every threshold, so the ~5 s
GC is skipped and a small sync pays zero GC cost. It then runs the pure domain/session_budget.py::gate_decision over
the chunk's composition-priced cost (chunk_worst_cost_kb): now that the emitted chunk is new + changed only (the
delta-restricted apply above), a create is priced at the worst-case create rate plus its transient cover term while a
changed/rebind item is priced at the lighter Set*-walk rate — the gate no longer prices every item as a cover-applying
create. Pause iff rss + chunk_cost ≥ limit; the pause log line names the composition ("… + N creates + M updates
projects …"). The frontend decides create-vs-update itself via its existing-shortcut scan, so a small backend/frontend
mismatch only ever overprices (worst-case safe). Both modes are predictive and differ only in the limit line: every
later chunk projects against cliff − margin (≈2.2 GB, keeping the anti-thrash safety margin), while the run's
very first chunk projects against CLIFF_KB (≈2.45 GB) instead. Forward progress must be guaranteed (the run has to
apply at least one chunk or it loops forever on a no-progress pause), so the first chunk is allowed to spend into the
safety margin — but the predictive projection still stops it before the crash line. A resume's first chunk therefore
proceeds only when its worst-case peak stays below the cliff (≈1.95 GB for a full 200-item chunk of cover-applying
creates, each priced create + cover) and can never be projected past it; a resume attempted without a Steam restart
re-pauses cleanly rather than driving a chunk into the cliff. On a pause it sets run_paused + a distinct
interrupt_reason and requests cancel — the loop returns cleanly with prior chunks committed and the terminal write
records the new terminal status paused (migration 014, its own status distinct from a crash's interrupted; both
resumable, but the split lets the UI say "(paused)"). That reason is the one backend sentence the panel shows the reader
verbatim (as the pause toast), so it names an action and never a control: nothing on this side can know what the
Sync page's start button currently says, since that turns on the frontend's Skip-preview setting and on a resume
question read from the stats. Completed platforms keep their PlatformSyncState stamps, so a resume redoes only the
remainder. Every step is fail-open: an unavailable RSS reading (no steamwebhelper, unreadable /proc) skips the
gate — and short-circuits further GC attempts — for the rest of the run (logged once), a seam error is caught locally,
and a failed GC only makes the reading less precise; measurement never blocks a sync. The same seams feed the UI
surfaces: sync_preview returns pause_likely (a predict_run_crosses prognosis pricing only new creates + changed
updates, never fully-unchanged items, so an unchanged re-sync never warns), a clean run's sync_complete carries
restart_recommended (post_run_advisory, RSS > ~1.8 GB, read GC-first), and the get_session_budget_status endpoint
returns a live RSS reading (no GC) plus the three fixed threshold lines (warn_kb ≈1.8 GB, ceiling_kb ≈2.2 GB,
cliff_kb ≈2.45 GB) for the persistent QAM banners (a blue "paused" banner, a yellow high-heap banner) and the
always-on "Steam memory" status row. The row's value text is traffic-light coloured against those three thresholds —
green / yellow (warn_kb) / red (ceiling_kb) — so the frontend holds no threshold magic numbers, and while a sync
runs (or a paused banner is showing) the row polls the endpoint (~5 s during a sync, ~10 s while paused) so the number
tracks the climbing RSS and the blue paused banner notices once a Steam restart frees memory. That notice is driven by
resume_ready on the endpoint (domain.session_budget.resume_would_proceed:
rss + RESUME_HEADROOM_CHUNKS × FULL_CHUNK_WORST_KB < ceiling — room for TWO worst-case chunks, ≈1.2 GB bar, because a
one-chunk bar sits exactly on the pause point where Steam's own small frees flicker the verdict; None when RSS is
unreadable) — when it flips true the blue banner announces memory is free and hides the restart button. The endpoint
also carries the paused run's progress (run_done_items / run_total_items), so that banner can read "1200 of 2001
games done": the counters are run-scoped fields on LibrarySyncStateBox, stamped with the plan's ROM total and grown by
the delta-restricted apply's SKIPPED entries (already correct on their shortcut), each wholesale-skipped unit's ROMs,
and every COMMITTED chunk's acked items — an emitted-but-uncommitted chunk (cancel / heartbeat timeout / the pause
itself) never counts, so the number can't over-report. They live in the backend deliberately: the backend process
survives the Steam restart the banner asks for, while the frontend reloads. In-memory only — a backend restart wipes
them and both come back None, which the banner renders by dropping the sentence rather than showing a zero. The banner
names no button of its own (#1789): the panel hands it the sync button as one value — the label and whether pressing
it resumes — so it quotes whatever the panel actually rendered ("Check for changes", "Resume Sync", "Sync Library",
"Apply Sync"), rather than deciding from the paused status, which survives a Force Full Sync that the resume does not.
When nothing can be resumed it also drops the progress sentence, because "1200 of 2001 games done" promises a head start
the clear has just discarded. That row also shows the last run's signed RSS growth, appended inline after the value
("X.X GB · last run ±Y"), measured at EVERY terminal (completed / paused / cancelled / interrupted) so a paused run
reads as its own consumption-so-far rather than a prior clean run's: a RAW read taken unconditionally at run start is
the baseline (run_start_rss_kb — captured before any chunk, so even a fully-incremental-skip run still records one and
reports ≈ +0.0 GB), the terminal RSS read is the end, and session_memory_delta differences them (an approximation for
information only, which a raw start baseline is fine for). The value is retained in last_run_delta_kb so
get_session_budget_status surfaces it on a QAM remount (in-memory only, lost on reload, no migration; None when
either reading was unmeasurable, so a stale delta is never shown); the UI reads it from that endpoint, so it is
deliberately NOT put on the sync_complete wire. Both banners also offer a Restart Steam now button that calls
SteamClient.User.StartRestart directly from the frontend — a deterministic full client restart that resets the
renderer's per-session budget to the ~430 MB baseline. The button is disabled while a game Steam lists is starting,
running or exiting, or reads a status that cannot be read, and checked again on click (isAnyAppHolding), so a restart
does not close a game Steam reports (Is the game running). The RSS
reader and GC trigger are wired through SessionBudgetMonitorConfig; the gate's per-item cost is a parameter, and
because the apply now pushes each created shortcut's cover through Steam's artwork API (SetCustomArtworkForApp,
transiently resident but GC-reclaimable — hence the GC-before-measure), the monitor prices each create at the worst-case
create rate plus the transient cover term (COVER_TRANSIENT_KB) at both the chunk gate and the preview prognosis,
while a changed item stays at the lighter update rate.
Run/unit/chunk identity on the ack (#1041). Every sync_apply_unit event carries the run_id (the run's
current_sync_id UUID), the unit_id (the WorkUnit.id), and the chunk_index; the frontend echoes all three back on
the report_unit_results ack. The chunk dispatcher stamps the dispatched unit's id into active_unit_id and the
current chunk into active_chunk_index just before it emits, and report_unit_results validates the ack against them:
the run_id must match current_sync_id, the unit_id must match active_unit_id (both compared by string value,
since a platform's id is numeric and a collection's is a string), and the chunk_index must match
active_chunk_index (as an int). An ack that fails the active-unit check falls through to the abandoned-chunk stash
(take_abandoned_chunk): a heartbeat-timed-out chunk's late ack — which in production arrives after the run wound down,
so active_unit_id / active_chunk_index are already cleared and current_sync_id is null — matches the stash by
identity and drives the recovery commit (the per-chunk timeout branch above, #1367). Only an ack matching neither
the active chunk nor a stash — a late ack from a cancelled run arriving while a fresh run is in flight, a stray ack
for a different unit, or a superseded chunk — is ignored (logged at debug, returns
{success: True, count: 0, ignored: True}): it is neither recorded, signalled, nor committed, so it can never be
credited to the wrong run/unit/chunk. active_unit_id + active_chunk_index are cleared once the chunk commits, is
cancelled, or times out (its identity moves into the stash). On the frontend side, the unit handler does not send the
ack at all once cancel has been requested — the first line of defence against a cancelled run's bindings landing in
whatever run started next.
Run identity on the cancel (#1198). cancel_sync(run_id) is run-scoped, mirroring the ack-path
_ack_matches_active_unit check above. A Cancel click meant for run N can land after run N finalized to IDLE
(current_sync_id nulled) and run N+1 started fresh — an unscoped cancel would flip run N+1 to CANCELLING (a sync the
user never cancelled), which would abort it and report cancelled. So when the requested run_id is truthy and does
not match current_sync_id, the cancel is ignored (logged at INFO, returns
{success: True, message: "Cancel
ignored (stale run)"}) — sync_state is left RUNNING. A matching id (or no active
run) flips RUNNING → CANCELLING as before. A falsy/None run_id cancels unconditionally — legacy callers and
the "no active run id captured yet" safety case — so cancel is never made less reliable. Every cancel logs one INFO line
recording the requested run_id, the active current_sync_id, and the sync_state at call time. The frontend sources
the run id from the backend-fed sync_progress store — its additive runId field (see below) — and passes it on the
Cancel click; in the pre-run "Fetching library…" window the store's runId is still "", which maps to the
unconditional cancel path rather than a stale prior-run id the backend would reject as a cross-run mismatch.
sync_cancel_preview is not run-scoped: it only discards the pending preview delta (pending_delta), never touches
sync_state, and sync_apply_delta independently validates the preview_id, so a stale preview-cancel cannot abort a
fresh sync.
Group-aware sync — one Steam shortcut per sibling group
(ADR-0021).
A game frequently exists in a RomM library as several dumps of the same title (region / language / revision variants).
These share a sibling group (same roms.sibling_group_key, derived client-side by domain/sibling_group.py: a
connected component over RomM's sibling_roms edges, keyed by the highest-priority metadata source the component agrees
on — compute_group_keys_with_unfetched, stamped onto the raw dicts before the shortcut build: per platform unit, once
over every collection unit's delivery, and over the preview's union in the same order, the ROMs the platforms it fetched
in full returned first and every other ROM after them against the keys those units commit, ADR-0022). Each stamp reads
the key every stored row holds, unbound rows' included (LocalLibraryReader.do_read_resident_group_keys): RomM lists a
version the user hid among its siblings' sibling_roms but leaves it out of the platform listing, so a new version
whose only sibling is hidden joins its game only through that row's key (#2617). The sync pipeline treats a group as one
game = at most one new Steam shortcut; the sibling row holding shortcut_app_id is the group's active version.
- Persist all siblings, emit one shortcut per group. The reporter's per-unit commit upserts a
romsrow (identity + version metadata, via the sync UPSERT) for every fetched ROM of the unit, but only the group's representative carries a Steam-shortcut binding — a non-representative sibling is a tracked, unbound row. Persisting the whole group is what the version picker (#1297) and the incremental-skip gate need; binding only the representative is what stops the same-named-collision and duplicate-shortcut bugs the ADR describes. - A row its listing leaves out follows its game (#2646). A platform unit's listing, and the preview's for a platform
it fetched in full, is a complete view: RomM leaves a version deleted there, or hidden by either kind of hide, out of
it, and the listing alone does not tell them apart. The stamp then also answers the new key of each stored row the
listing left out that follows its group — a row an edge of the group reaches, and a bound row whose stored key a
member of the group held at the last sync — and, where every member holding a stored key held the same one, does not
count that key as such a row's summary when it alone would split the group (the rules: ADR-0022's amendment). The
apply and the preview lay those keys over the bound-row registry before the convergence and the collapse
(
services/library/_followed_keys.py), and the commit writes them (Rom.follow_group; which commit, above under the chunked apply), so a game whose version carrying its canonical id is deleted keeps its one shortcut on that version, and a hidden version takes the key its game moves to. The preview reads its registry before it stamps, since the bound rows a listing left out are read off it. A collection unit's keying is unchanged. - The collapse happens at the same
build_shortcuts_databoundary in both paths (domain/sync_diff.py::collapse_sibling_groups). Preview and apply build the full per-ROM shortcut list, then collapse it to one entry per group against the bound-row registry (preview readsLocalLibraryReader.do_read_preview_baseline, apply readsLocalLibraryReader.do_read_apply_registryper unit). The collapse takes an explicitcomplete_group_view: a sibling group is per-platform, so each platform apply unit sees every member of a group its platform listing returns (complete_group_view=True); a version hidden in RomM is not among them, so a bound one is collapsed as one gone from RomM. A collection apply unit spans platforms and may fetch only one unbound sibling of a group whose bound member it never fetched, so it is a partial view (complete_group_view=False): a group that already holds a binding anywhere is grandfathered untouched — never rebound onto the partial member, never given a second shortcut. The whole-library preview passes the set of group keys it holds whole, so each group gets the view the apply's units have: every group a platform unit of the run delivered a member of is complete, whether or not that unit is skipped as unchanged, and a group only collections reach is partial (#2536). The rebinds the view decides are therefore counted as the apply makes them (the #1292 counts-vs-reality bug class). Collapsed whole, a game only collections reach would count a rebind of a bound member gone from RomM, or without a file, that its collection unit never makes. Where the group's platform is synced, its bound member rides that platform unit in the same run; where only collections reach the group, the stale removal keeps the binding (below). (Inferring "bound sibling absent ⇒ vanished" from a partial view would rebind a live installed game onto an uninstalled sibling — #1296.) A collection skipped as unchanged hands the preview the bound members it lists, rebuilt from their stored rows, among them one its platform's fetch no longer returns: a user's own hide in RomM drops the member from/api/romsbut not from a standard or virtual collection's listing. The apply's platform unit commits before that collection unit runs and collapses the group without the member, so it rebinds the shortcut where a same-stem sibling is fetched. The preview collapses such a group a second time without the rebuilt rows (services/library/_preview_union.py::collapse_counting_platform_rebinds); where that pass rebinds a row, its answer is taken and the row is no longer classified as skipped, and where it does not, the row keeps its place and stays skipped, so the platform-collection diff files it as the apply does (#2544 D1). The run counts the row as synced either way, which the stale removal reads.classify_romsruns over the collapsed set, so its new / changed / unchanged / stale buckets count games, not dumps, and an unbound sibling stops reading as a perpetual "new". - A game only collections deliver is minted over every version they deliver (#2543). Each collection unit collapses
what it delivered, and a later unit sees the binding an earlier one committed, so a game whose versions sit in
different collections would be minted by the first of them over its own versions alone, and the preview, collapsing
the union, would name another. When the run reaches its first collection unit it therefore fetches every collection
unit of the run (
CollectionFetcher.fetch_collection_units, in queue order, each against the run'ssynced_rom_idsas a unit-by-unit fetch is, so a ROM two collections list is delivered by the first), stamps the component keys once over all of their deliveries with the resident keys the preview reads, and moves every member of a group no bound row holds into the first delivery holding one (domain/sync_diff.py::gather_unbound_groups, which groups bound rows as the collapse does). That unit mints the group over every version the run delivers, the installed one and RomM's default included, as the preview's union does; the preview needs no change. A group that already holds a binding is untouched, so a shortcut an earlier sync minted, in collection order or before its other versions were delivered, keeps its version. A skipped collection delivers bound rows only and takes part in neither step. The fetch adds no request — the same skip probes and walks, made earlier — and holds the deliveries until each unit applies, never more than the preview holds at once. A cancel or a RomM failure during it applies no collection unit (the platform units before it stay applied), and the first collection's shortcuts appear only once every collection is fetched. Its frames carry the first collection unit's step, with the bar counting the collections fetched, so no later row reads done before it applies and the bar never steps back. - Resolution chain (
domain/sibling_resolution.py::resolve_group_representative, total + shuffle-stable): an installed sibling wins; else an existing binding; else RomM's per-user default (is_main_sibling); else the 1G1R ranking — prerelease demotion > region priority > revision (newest) > alphabeticalfs_name_no_ext>rom_id. The first three legs are membership filters; the rest are the total order applied inside the surviving leg. Prerelease demotion ranks first: a member whose structuredtagsname a draft build (Alpha / Beta / Proto / Sample / Demo, case-insensitive, tolerant of a trailing number;Unl/Aftermarket/ unknown tags are neutral) is demoted below every retail sibling across regions, so a finished(Japan)release beats a(USA) (Beta). Revision ranks just after region: within one region the newestrevisionwins (natural compare; base dump = lowest), but never lifts a lower-ranked region (a(USA)base still beats a(Europe) (Rev 9)). The alphabetical leg also keeps a base dump ahead of a filename-only re-dump ((Virtual Console),(Extended Edition)) RomM does not parse into a tag. Region priority ranks a version by its bestregionsentry against a fixed build-time order —World > USA > Europe > Japan, every other named region after these (alphabetically), a no-region version last (a fixed constant, not language/system detection) — which the user may re-head with a singlepreferred_regionsetting ("auto"= the fixed order; any other value lifts that region to the top). The leg is evaluated at resolution time, so the setting takes effect on the next sync and an existing binding shields its group (the installed/binding legs win before region priority is consulted). Used wherever one version must be chosen (a new group's shortcut, the pick among several same-dump siblings a rebind could move onto). The QAM dropdown that setspreferred_regionis populated from fixed anchors (Default, World, USA, Europe, Japan) plus the distinct regions read from the local library (get_known_regionsoverroms.regions— no server call), and a confirmation modal states the apply-at-next-sync / no-rename semantics before persisting. A member RomM holds no file for (has_file_on_diskfalse: its file left RomM's disk, or it is a physical game) stays in the installed and binding legs, while the default and ranking legs leave it out as long as a member with a file remains — so a new group mints its shortcut on a dump with a file whenever one exists and no member is installed, and with no file anywhere the chain runs over every member as before. The collapse moves a binding off such a member only onto the same dump in a new shape (Steam Non-Steam Shortcuts — a bound ROM RomM holds no file for). - Canonical name at mint (
domain/sibling_resolution.py::canonical_group_name): a NEW group's shortcut is named after the member ranked first by the pure order (prerelease demotion > region priority > revision > alphabetical >rom_id), ignoring the installed/binding/default filters — so a Japanese default still binds Japan but the shortcut carries the USA name (never majority voting: two Japan dumps + one USA yields the USA name). This is mint-time only: the name is sticky forever after (ADR-0021 §2), so a bound or rebind entry carries the persisted bound name verbatim and a live shortcut is never renamed. The appId is minted from the canonical name by the frontendAddShortcut; the DB binding lands on the representativerom_id; the reporter is agnostic to the name↔rom mismatch (it binds onrom_id+ the acked appId, and persists each sibling's own RomM name frompending_all_roms).preferred_regionis read from settings bysync_previewandunit_applyand threaded into every collapse call site (preview + apply) as the same value within a run. - A version gone from RomM stays bound; only the same dump takes its binding
(ADR-0021 §2's amendment). A bound sibling that
vanished from the server emits nothing and keeps its binding, its shortcut, name and launch command, installed or not;
the game page says it is gone, and the version picker switches it — unless its game holds another bound shortcut,
which converges (below, "Convergence"). It stays in its game's group where it carried the group's canonical id alone,
since it follows the key the remaining versions take (above), so no second shortcut is minted beside it. The one move
left is onto the same dump under a new id — in a view that sees the group's whole membership (a platform unit, or
the preview for a group a platform unit delivers; a partial collection view never moves a binding), onto a fetched
sibling with a file on RomM, no shortcut of its own and the
fs_name_no_extthe vanished row's storedfs_namehas without its extension. The collapse then emits a rebind entry keyed to the vanished sibling'srom_id(so the frontend reuses its existing shortcut through therom_id → appIdmap — no delete + recreate churn), carrying the target's shortcut data and abind_rom_idmarker. The commit translates the ack throughbind_rom_id, so the DB binding moves onto the target; the collision-safeRomsave then unbinds the vanished row. The shortcut's appId, artwork, collections and playtime all survive. The #1036 committed-appId guard is unchanged (the reused appId is committed this run, so it is never emitted for removal). - The stale removal keeps a game gone from RomM. A bound row no unit of the run returned is sorted by
domain/sync_diff.py::sort_stale_rows. A row the user hid in RomM whose game the run did not sync is removed first, on any route and unasked (the next bullet). Otherwise, on a platform the run read whole it is gone from RomM and stays bound — and stays in its platform's Steam collection. So does a bound row a collection's skip rebuilt from its stored row (above) on a platform the run read whole without returning it: the run counts it as synced, so the sort never sees it, but where that collection is fetched instead the row is stale and kept.domain/sync_diff.py::rebuilt_rows_filed_as_keptpicks such rows, in the apply's finalize from the rows its skipped units rebuilt (LibrarySyncStateBox.run_skip_rebuilt_rows) and in the preview from the rows it classifies as skipped, and both file them as kept rows, so the game's platform collection does not depend on whether the collection was skipped (#2549 D1). A stopped run, which keeps no stale row, files none of them either. On a platform the run did not read, a row whose game the run synced stays bound: when another member of its sibling group is among the ROMs a unit returned, a skipped unit's stamp members included (#2533 D1), the row is never removed. A row's group is read off its stored key: a bound version whose stored key no longer matches its game's, after RomM matched the game anew, is judged by that stored key, as before this rule (#2533 D10). It is still asked about wherever any other row would be, below, and RomM's answer sets or clears its mark as for any asked row (D6). A version switch binds the shortcut to a member no unit delivers while a collection still delivers the game through another version, and collecting any version puts the game's one shortcut in the collection (the group fallback below). A row without a group key is judged alone. Any other row was reached either through a collection or through its platform. The row itself says which:roms.delivered_throughrecords the kind of unit whose commit last upserted it (Rom.record_delivery, written bySyncReporter.commit_unit_resultsfor the unit the chunk dispatcher names; a commit that names none, such as a unit acknowledged after its run ended, keeps the row's value). A row last delivered through a collection is asked for once by its exact id (LibraryFetcher.ask_romm_for_roms, single attempt), and removed only when RomM returns the very ROM, since it left the collection — a 404 and an answer that establishes nothing both keep the shortcut, so the row is asked again on every later run until it is found or unbound, and a stopped run, which sorts nothing, leaves it as it was. A row last delivered through its platform is removed unasked, its platform turned off — unless RomM no longer lists that platform, below. A row no commit has recorded a route for since migration029falls back to the collections' completion stamps, read once before the run's first unit rewrites any of them (do_read_collection_listed_rom_ids): one a stamp lists is asked, any other removed. A platform RomM no longer lists is gone from RomM, not turned off: the fetcher records thefs_slugof every platform RomM's listing names on the run state (LibrarySyncStateBox.run_listed_platform_folders) when it builds the work queue, preview included, and a row whose platform is not among them keeps its shortcut, with no request, unless it was last delivered through a collection. A platform is known here by itsfs_slugthroughout — the platforms the run read, the ones it listed and the ones it processed — because two platforms RomM files under one slug are two platforms (#2452 D1). A row with nofs_slugrecorded — a row no sync has written since migration043, such as one on a platform switched off or no longer listed — counts as on a platform the run did not read (D5). It is asked for by its exact id; one a collection last delivered is asked and settled as a collection's row. Any other one RomM returns with itsfs_slugis sorted as a row on that platform — removed on a platform RomM lists and the run did not read, kept otherwise — and the apply records thatfs_slugon the row (Rom.record_platform_folder, in the finalize's write UoW). Any other answer keeps its shortcut and marks it as RomM's answer says; a row whose 404 keeps it stays without anfs_slugand is asked again on every later run. A collection holding a member with nofs_slugrecorded is not skipped (domain/collection_skip.py::holds_a_game_without_its_platform_folder), so the fetch brings the member itsfs_slug. The kept ids underkv_config'splatform_idsare not the answer, because a game page that asks for an unknown platform's system during the run writes it back there (PlatformSystemService). A listing that fails stops the run, so a run that reaches the stale removal always has its own. Which platforms are on in Tender is keyed by RomM's platform id, which an unlisted platform no longer has, so the route is what says the platform was on: a row last delivered through its platform had it on. A kept row stays in its platform's Steam collection; for a platform RomM no longer lists, that collection keeps the name theplatform_namescache holds; which name names every other platform is under Collection management. Every finalize, a stopped run's included, rewrites the cache with the run's listed names and carries forward the name of each platform outside them a bound row is still on (RunFinalizer._names_with_bound_platforms_kept), so a cancelled or paused run in between does not rename the collection to thefs_slug. The preview reads the same name (do_read_kept_platform_names, throughservices/library/_stale_rows.py::with_kept_platform_names) and counts the collection as kept. A row RomM was asked about and did not return keeps its shortcut without being one of the kept rowssettle_stale_rowsanswers, and so does one whose game the run synced on a platform the run did not read: each stays bound, so it is in its platform's Steam collection exactly where platform groups are on — except a row with nofs_slugrecorded, which is in no platform's collection (#2452 D5) — since the apply's end-of-run scan files every bound row throughdomain/sync_diff.py::platform_collection_name, and the preview files each stale row it neither removes nor keeps through the same function (#2490 D3). The preview's platform-collection diff names every platform it counts as the apply does, by the listed name, or, for a platform the listing does not name that a kept row, a row left bound or an emitted entry is on, the name theplatform_namescache holds, else thefs_slug— an emitted entry and a row left bound throughplatform_collection_name, a kept row by the same lookup — never by theplatform_namean entry carries, which a collection's skip fills with the slug and a collection's fetch with RomM's display name (#2558 D1). The preview's per-platform table is named by the same names (QAM panel — Sync). Force Full Sync leaves the routes in place. The preview asks the same question for the same rows, through the sameservices/library/_stale_rows.py::settle_stale_rows, so the removals it counts are the ones the apply makes. Both judge a row's game by the group keys of the ROMs the run counts as synced (domain/sync_diff.py::group_keys_of): the apply reads them off the rows its commits upserted (do_read_resident_group_keys), and the preview, which writes nothing, lays the keys its fetch stamped over the same read. Only the apply records what it established (StaleSort.gone_from_romm_marks, written in the finalize's write UoW byRunFinalizer.finalize_per_unit_run): an asked row RomM answered with a proven 404 is markedroms.gone_from_romm = 'not_found', one RomM returned has its mark cleared, and an answer that establishes nothing writes nothing; a row kept on a platform RomM no longer lists is markeddelisted. A row kept on a platform the run read whole is not marked, since that platform's stamp already names it. Any commit of a row clears its mark, because the column rides the sync UPSERT with no merge. The mark is what lets the removed-game cleanup list such a row (Removed-game cleanup); Force Full Sync leaves it in place. -
A game the user hid in RomM leaves Steam (#2502). RomM has two kinds of hidden. A ROM an admin hid from a user or group (
hidden_entities) is answered 404 by its exact id and listed nowhere, so nothing tells it from a deleted one and Tender treats it as gone from RomM (D3). A ROM the user hid (rom_user.hidden) is left out of every/api/romslist, a collection walk included, while its exact-id reads answer it withrom_user.hiddenset and a standard or virtual collection's listing and its siblings'sibling_romsstill name it. Tender reads that kind as the user asking for the game to leave Steam (D1). Each run's work queue, preview and apply alike, asks RomM once for the ids of every such ROM (RommApiAdapter.read_hidden_rom_idsoverlist_hidden_roms:GET /api/roms?statuses=hidden&limit=1, itsrom_id_index, read bydomain/hidden_roms.py::hidden_rom_ids) and holds them onLibrarySyncStateBox.run_hidden_rom_ids, the run's only source of what is hidden (D2). An answer that raises or cannot be read — one whose first item is not marked hidden included, as from a RomM that ignoredstatuses— is logged by the adapter and held asNone, and nothing is taken for hidden. The set reaches four rules:- The stale sort removes a stale row in the set whose game the run did not sync, unasked, on any route, and clears
its
gone_from_rommmark, since RomM still serves it (StaleSort.hidden). A hidden row whose game the run synced is sorted as any other row, so a game with a visible version keeps its one shortcut. A row without a group key — one left NULL from before migration011— is judged alone, as #2533 D6 judges every such row: hidden by the user, it goes the stale way even where a visible version of its game is synced, and that version gets a shortcut of its own, under a new appId (D9). No current sync writes a NULL key. - The platform skip leaves out of its rebuild a bound row in the set whose game no version of the stamp's fetch
holds (
domain/fetch_generation.py::hidden_without_a_listed_version), so the row is stale and goes as after a full fetch — also on the run after one stopped between the fetch that left it out and the stale removal. - A collection unit's members are its listing's ids less the set (D4, "Incremental skip" below).
- The convergence leaves out a hidden bound row of a group no listed version holds, so it neither keeps the
group's shortcut nor takes another bound row's away, and the group's other bound rows converge without it. In a
group a listed version holds, a hidden version ranks as an unlisted one on both routes — on a platform, and in a
collection delivery's convergence, where any other bound row a delivered member lists in
sibling_romsranks as listed, by its stored version fields (D8).
The removal is the stale removal's own: the panel reports it; files, saves and the install record stay; and no running-game guard holds it (D7). Unhiding brings the game back as a new shortcut on the next run that delivers it. The version picker and the removed-game cleanup ask the same request, once per open and once per scan (D5). - Convergence. A group that carries several bound shortcuts (a library synced before this model, or a group RomM joined after each part got its shortcut) keeps one (#2645, ADR-0021's amendment). A platform unit asks
domain/sync_diff.py::converge_bound_groupsover its listing and its platform's bound rows, the bound rows it does not list included: of a groupgroups_to_convergenames, the shortcut stays on an installed listed version (the chain ranking several), else an installed unlisted one, else the chain's pick among the listed ones (one RomM holds a file for before one without), else the unlisted one with the lowestrom_id. Installs of the unlisted rows come fromLocalLibraryReader.do_read_installed_rom_ids. A group one of whose bound shortcuts Steam lists as running is left out (#2645 D5):services/library/_convergence.py::converge_unless_runningreadsRunningGames.running_rom_ids— the reading the removed-game cleanup leaves a running game out by — only when a group would converge, in the apply's platform unit, once the apply has fetched its collection units, and in the preview, each at its own time. The finalize of a run that converged something reads it once more before it removes a shortcut, since the unit's reading may be minutes old by then, anddomain/sync_diff.py::without_running_groupsleaves out every group one of whose shortcuts runs by then, the kept one's included (#2645 D6). The preview therefore counts the removals the apply makes only while no game starts or quits between the two: a game started after the preview keeps the shortcuts the preview counted as removed, and one that quit after it loses the ones the preview left alone. A platform whose rows hold such a group is never skipped (domain/fetch_generation.py::convergence_pending), so it is fetched in full on every sync until the game has converged, a game left out while it runs included. The collapse takes the losers (converged=), emits nothing for them and still counts their shortcuts as held, so no shortcut moves onto one in the same run. The run keeps them onLibrarySyncStateBox.run_converged; the finalize of a run that was not stopped takes them out of the stale scan, passes them throughselect_stale_removals(#1036) and sends them insync_stalewithkept_rom_id/kept_app_id, the row its game keeps once the run commits (with_kept_rows_rebound); each keeps its binding, as a stale row does, until the panel reports its shortcut gone, and until then the group still counts as holding two shortcuts, so a removal the panel did not finish converges again on the next sync. A stopped run removes nothing. The preview runs the same rule over the listings of the platform units it fetched in full and counts the losers as removals. A game a collection brings in, its platform turned off, converges too (#2661): a collection holding any version brings in the whole game, as RomM's own game page shows every version whichever one a collection holds. OnceUnitApplier.fetch_collection_unitshas fetched every collection unit, one decision for the run goes over the ROMs the fetched units delivered:domain/sync_diff.py::brought_in_by_deliveryanswers, in each group where it is more than one, every bound row a delivery holds and every bound row whose stored key is the group's and that a delivered member of the group lists insibling_roms;services/library/_convergence.py::converge_collection_deliverieshands those to the sameconverge_bound_groups, scoped to the platforms the sync leaves off (WorkUnit.unwalked_platform_folders). A brought-in row no collection delivered ranks as listed, by the chain over what its last fetch stored (domain/sync_diff.py::stored_version: regions, revision, tags, whether RomM holds a file, thefs_namestem) and RomM's default as the sibling list states it (#2661 D4), unless the user hid it ("A game the user hid in RomM" above); its install, like every brought-in row's, comes from its install record. So which version a collection holds decides neither whether the game converges nor which shortcut stays. The losers joinrun_converged, which every collection unit's collapse leaves out, so a game split over two collections is decided once. A skipped collection's rebuilt rows deliver nothing, and the preview runs the same helper over the ROMs its collection units fetched, never over those rows. A bound row of the group that no delivered member lists — deleted, hidden from the user by an admin, or regrouped in RomM — is not converged, and keeps its shortcut while a ROM the run synced still carries its stored key (#2661 D1, D3). A bound row RomM joined to the game after Tender last read it is not brought in either: its stored key is still its old game's, no collection delivers it, and Tender cannot key it without reading it. On a platform the sync leaves off it goes the stale way, as it would without the join, so the game ends with one shortcut, the delivered version's, without the keeper order and without carry-over; its install record stays, and the version picker, which reads the siblings from RomM, still offers it (#2661 D5). A game on a platform RomM no longer lists keeps its shortcuts for good: no platform unit reads that platform and no collection walk asks for it. The panel carries each loser's Steam state over before the removal (frontend/src/utils/shortcutCarryOver.ts), then reports the removals Steam no longer lists throughreport_sync_removals, which unbinds each row, resets the Steam Input setting of the removed appId and deletes its grid images, and leaves the per-ROM cover cache. - Downstream group semantics. The incremental-skip gate compares RomM's whole visible id set — every sibling, bound or unbound — with the checksum the completion stamp recorded (#743), so skip parity on platforms with sibling groups holds. The backfill gate reads the stamp's fetch generation: a NULLsibling_group_keyforces a full fetch only on a row the last complete fetch returned, because a row for arom_idthe server has since dropped (retained per ADR-0007 — nothing is deleted) is never returned again and no fetch could ever fill its key in — letting it demand one would wedge the platform into a full fetch on every sync, forever (#1504). A stamp with no generation predates migration031too, so it carries no checksum and its platform is fetched in full before the backfill gate is asked. The convergence gate is not generation-gated: a platform any of whose bound rows share a group key with another bound row (domain/fetch_generation.py::convergence_pending, overgroups_to_converge) is fetched in full, because a bound version RomM no longer lists converges too; the full fetch converges it, so the skip returns on the run after. A collection holding a member whose stored key is such a group's on a platform the sync leaves off is not skipped either (domain/collection_skip.py::holds_a_game_to_converge, asked after the listing verdict and before the probe, and logged): the fetch hands the convergence RomM's current sibling list. The removal drops the stamp of a collection that lists the loser, so such a collection is fetched once more before it skips again. Where the version no collection delivers is the one RomM no longer lists as a sibling — deleted, hidden by an admin or regrouped — no unit returns its row, so its stored key is never rewritten (the stale removal's question by its id, asked where its route calls for one, records onlyroms.gone_from_romm), the game keeps both shortcuts and the collection is fetched on every sync (#2661 D3); where the delivered version is the one regrouped, its commit writes its fresh key (Rom.syncedinSyncReporter._persist_synced_rom), the stored keys part, the row left behind goes the stale way in that run as a game no collection holds, without carry-over, and the collection skips again after one fetch (D5). Artwork downloads only for the emitted representatives (+ the bound siblings a collection unit keeps), never eager sibling covers. Steam-collection membership resolves each RomM collectionrom_idwith a group fallback — an unbound sibling maps to its group's bound sibling's appId — so collecting or favouriting any version puts the game's single shortcut in the collection. - The stale sort removes a stale row in the set whose game the run did not sync, unasked, on any route, and clears
its
Same-named collections union into one Steam collection (#1503). RomM enforces name uniqueness only per-table and
per-(name, user_id), so two enabled collections can share a display name — a standard collection and a smart/virtual
collection, or (multi-user) another account's public collection the list endpoints return. Steam's collection namespace
is by-name (RomM: [<name>] (host)), so both must resolve to the one Steam collection. The finalize accumulator
(_state.py pending_collection_memberships) is therefore keyed by a collision-free (collection_kind, collection_id)
identity — the same identity the #742 completion stamp uses — with the display name carried in the
CollectionMembership value; the real-sync and preview write paths share one key builder so they cannot drift. The run
finalizer (_resolve_collection_memberships) then groups the accumulator's entries by name and unions their
resolved appId sets (order-preserving, de-duplicated across collections; each collection's own resolution already dedups
within), emitting the unchanged by-name romm_collection_app_ids: {name → [appId]} contract — a single-collection name
unions a set of one and is byte-for-byte the pre-#1503 output. The union runs after the owner scope (below): once a
foreign collection is dropped from the work queue it never reaches the accumulator, so the own scope narrows what is
unioned rather than changing how the union works.
Collection kinds — one virtual kind covers RomM's browsable virtual types (#1538). get_collections and
build_work_queue group collections into three internal kinds: standard, smart, and virtual. RomM's own UI calls
the ownership-carrying first kind Standard (Tender's earlier name for it was user, a misnomer renamed internally
by #1539 — display "My" → "Standard"; see the enabled-bucket migration note below). The virtual kind is RomM's
ownerless VirtualCollection (base64 id, no user_id, no updated_at worth comparing — stamped by its member set,
under "Incremental skip" below), and it carries a virtual_type sub-field on each collection dict/setting so the
Collections tab can list it under Franchises or IGDB collections. RomM's VirtualCollection has five type values, but
Tender syncs only the two RomM itself surfaces as browsable collections: IGDB franchise and the default IGDB
collection (series). genre, company, and mode are intentionally excluded — RomM treats genre/company as
ROM filter facets (not collections) and mode as neither, so they never appear in RomM's Collections view. The
supported set is a single constant (services/library/fetcher._SUPPORTED_VIRTUAL_TYPES); get_collections fetches each
supported type and merges them under the one virtual bucket. Which types a sync fetches, and what a failed one does to
the sync and to get_collections, is under "A listing that fails stops the run" below. Because the type is baked into
the base64 id, ids are globally unique across types, so one enabled-bucket keyed by id cannot collide. The owner scope,
the skip gate's membership check, and per-unit ROM-fetch dispatch all stay a single kind == "virtual" branch, not
fanned out per type. On disk enabled_collections holds the three buckets standard, smart and virtual, the shape
every settings.json from version 13 on has. The ownership-carrying bucket was once called user: old
collection_sync_state completion stamps keyed collection_kind = 'user' are rewritten to 'standard' by SQLite
migration 022 so an unchanged standard collection still takes the incremental skip across the upgrade (#1539).
Batch collection enable — save_collections_sync (#1539). A single settings write that stamps every id it is given
into one kind bucket and touches nothing else in it; the Collections tab's Enable all / Disable all send it the
ids their table lists, so what is written is exactly what the page shows. An unknown kind refuses with invalid_kind
and a non-list id argument with invalid_ids, before any write; an empty id list is a success no-op.
Collection owner scope — own is a sync scope, not a display filter (#1532). RomM's collection list endpoints
return the signed-in user's own collections plus every other user's public collection. The QAM's Other users'
collections toggle writes collection_owner_scope ("own" / "all", default "all"; how the QAM presents it is
docs/architecture/qam-panel.md § Library). Ownership is a pure predicate
(domain/collection_owner.is_own_collection): a collection is own when it is a virtual collection (RomM's
VirtualCollection model carries no user_id column — these are global/derived and belong to no one, so they always
survive), when Tender's own identity is unknown (the unknown-identity fallback, which drops nothing), or when the
collection's user_id equals the stored romm_user_id; only standard and smart collections carry a user_id to
compare. get_collections tags each row with is_own from the same rule with one difference
(domain/collection_owner.listing_is_own): while the identity is unknown a standard or smart row carries is_own: null
rather than true, because nothing established that it is the user's; how the page reads it is
docs/architecture/qam-panel.md § Library. Virtual rows are always true. build_work_queue applies the same
predicate: under "own" with a known identity it drops foreign standard/smart units from the queue — so a foreign
collection enabled earlier is never synced — while virtual units and every unit under "all" pass through unchanged.
The scope applies over the per-kind enable state without mutating it, so switching back to "all" restores the
prior enables. Because an unknown identity drops nothing, the feature is non-breaking: it silently no-ops until
romm_user_id is stamped (see the ConnectionService lazy-identity note), then takes effect — no re-login required.
Each collection row states how many of its members are already in Steam, and who owns it (#1833). get_collections
adds two fields to its rows. in_steam_count, on all three kinds, counts the collection's member ROM ids (RomM's
listings carry rom_ids on each) that the sync's collection filing resolves to a shortcut (GLOSSARY.md → Reachable,
which says how it differs from the platform count). It counts members, not shortcuts: two versions of one game both
count although they share one. It is absent, never 0, when it is unknown — the local read failed, which does not fail
the listing and is logged as a warning. The fetcher asks its inward pair
(LocalLibraryReader.do_read_reachable_rom_ids) in one short read UoW once all the listing requests have returned,
never across one; the per-row count is domain/collection_listing.py's. owner_username is RomM's own field on the
standard and smart listings (CollectionSchema / SmartCollectionSchema in every RomM version Tender accepts),
forwarded as null where a listing lacks it; virtual rows carry no such key, since they have no owner.
Incremental skip — the per-platform completion stamp is the sole authority. A platform unit skips only when its
PlatformSyncState stamp exists and its skip has not been revoked
(ADR-0023). The stamp
is keyed by the platform's fs_slug, so two platforms RomM files under one slug each keep a stamp of their own and skip
on their own (#2452 D1, D4). A completed SyncRun's
last_sync is deliberately not a fallback: a run-scoped timestamp cannot see a platform whose shortcuts were
locally removed and only partially re-applied afterwards, so trusting it can silently skip a platform with missing
shortcuts. No stamp means a full fetch. The reporter writes the stamp when a platform work unit's last apply chunk
commits — atomically in the same write UoW as the chunk's roms upserts — so a platform that fully synced inside a run
the user later cancelled/crashed still skips on the next run instead of re-walking every already-applied game through
CEF.
The stamp records its fetch, and the skip asks RomM once whether that fetch still stands
(#743). Besides completed_at, the stamp records two things
about the fetch that wrote it (migration 031): fetch_started_at, Tender's clock when the apply began fetching the
platform, and rom_id_checksum, a checksum of the sorted ids that fetch returned
(domain/platform_skip.py::rom_id_checksum). The skip's reference is fetch_started_at less a fixed ten-minute margin
(SKIP_REFERENCE_MARGIN), not completed_at: an edit RomM records after the fetch began but before the final chunk
committed is newer than the reference, so the next sync sees it. The margin covers a device clock ahead of the server's
and RomM's whole-second timestamps; its price is one more full fetch, and one more re-stamp Apply offered, for a
platform edited within ten minutes before the fetch that stamped it, and for a collection whose member was, since the
collection skip takes the same margin. It covers no lead of the server's clock over Tender's: a ROM RomM updates stays
newer than each new stamp's reference until Tender's clock has passed its updated_at by the margin, so its platform is
fetched in full, and the preview offers the re-stamp Apply, on every sync in that window, and a lead lengthens the
window by its own length. The gate then sends one request per platform, GET /api/roms for the platform ordered by
updated_at descending with limit=1 (RommApiAdapter.list_roms_newest_first), with no updated_after and RomM's id
index left on. Its first item's updated_at says whether any ROM changed since the reference, and its rom_id_index,
which RomM builds from the whole filtered result, is the platform's visible id set. The platform skips only when no ROM
is newer than the reference and the id set's checksum equals the stamp's (domain/platform_skip.py::probe_verdict).
An answer the gate cannot read — no id index, an id that is not an integer, a first item without a readable updated_at
— establishes nothing and is a full fetch, and so is a stamp without a checksum or a fetch start, which is every stamp
written before migration 031. The verdict carries its reason into the sync log: the newest updated_at against the
reference and the number of visible ids with whether their checksum matched, or that the answer could not be read.
RomM's platform rom_count is no longer compared with anything: it counts hidden ROMs, so after a hide it never again
matched what a fetch returned, and the platform full-fetched on every sync. The stamp still records it, for removed-game
discovery. The local guards stand before the request: rows carrying the stamp's generation, at least one bound row, and
no sibling_group_key backfill pending for a row of that generation.
Which RomM changes move updated_at, and which only the id set shows (read at RomM 5.3.0). RomM stamps
roms.updated_at on the application side — default and onupdate of utc_now on its base model
(backend/models/base.py), with no database trigger or ON UPDATE clause behind it — so a change moves it exactly when
it goes through an ORM write of the ROM row. That covers every field Tender stores but one, since all the others are
columns of the ROM row; the exception is the default-version flag, which RomM keeps per user on rom_user:
| RomM change | Moves roms.updated_at |
What the skip sees |
|---|---|---|
| An edit through the API (name, file, metadata match, cover upload) | yes | a newer updated_at |
| A scan, quick or complete, that writes the ROM (metadata, cover, files) | yes | a newer updated_at |
| A ROM added or deleted | added: yes; deleted: — | another id set |
A user hiding a ROM (rom_user.hidden) |
no | another id set |
| An admin or group hide, or an age limit (RomM 5.4) | no | another id set |
A default-version pick (rom_user.is_main_sibling, PUT /api/roms/{id}/props) |
no | nothing |
A raw SQL UPDATE of roms |
no | nothing (none found in RomM) |
A hide writes only the per-user rom_user row or the visibility rules, and the list filters the ROM out of both the
items and the id index (backend/handler/database/roms_handler.py), so the checksum catches it. An add and a delete of
equal count change the id set too. A default-version pick reaches the version picker through the bound ROM's own read
when the game page loads the picker, not through the skip. order_by=updated_at resolves to Rom.updated_at
(roms_handler.py), and the id index is built by default (with_rom_id_index, backend/endpoints/roms/__init__.py).
Confirmed against a live server on 2026-10-19: descending order puts the newest first, the index at limit=1 equals the
id set of the full paged walk, total equals its length, and timestamps are whole seconds with +00:00.
A full fetch whose stamp no longer matches is re-stamped through an Apply. The stamp is written only by an apply,
and many of the changes that make the skip refuse a stamp change no shortcut — an edit that moves updated_at but no
field a shortcut carries (a metadata match, a rescan that rewrites the row), a version the user hid beside a visible
one, a ROM an admin hid, an updated_at inside the margin — so a preview with nothing else to do would short-circuit on
"no changes" and the platform would be fetched in full on every sync. The preview therefore counts such a platform in
restamp_platform_count (below, "Unstamped-platform re-run"), and the gated apply's 0-delta final chunk writes the
fresh stamp, which an updated_at still inside the margin outlives (the margin paragraph above).
A bound row the stamp's fetch did not return no longer holds the skip
(#743 D10) — unless its group holds another bound shortcut:
then the platform is fetched in full until the group has converged (Sibling groups, "Convergence"). Such a row is a
version deleted in RomM, hidden by an admin, or hidden by the user beside a version RomM still lists, and it stays
bound: the stale removal keeps a bound row missing from a platform the run read whole (ADR-0021 §2's amendment), and a
run stopped before its end scans for none at all. The skip rebuilds its unit from every bound row, so a rebuilt unit
carries such a row as if RomM had returned it, and the run's stale-removal scan never names it — which the keep rule
would not act on anyway. The one row the rebuild leaves out is a bound row the user hid whose game no version of the
stamp's fetch holds (domain/fetch_generation.py::hidden_without_a_listed_version): the stale removal takes its
shortcut ("A game the user hid in RomM" under Sibling groups). Before the checksum the gate full-fetched while any bound
row lacked the stamp's generation, because its count guards could miss such a row and a full fetch was the only way to
see an id a re-import reissued. The checksum answers both: it records that the stamp's fetch did not return the row, and
any change to the visible id set since — a reissued id included — forces the full fetch the same-dump move needs. With
an equal checksum and nothing newer there is nothing for a full fetch to learn, so the platform skips. The row stays
bound and, unless the user hid it, still shows in the removed-game cleanup's list
(Removed-game cleanup), which reads the generations, not the skip.
The stamp's contract is a stamp may authorise a skip ⟺ the platform's most recent apply attempt ran to completion, and
neither a local removal nor a stale removal on a run that did not process the platform has unbound one of its rows or
removed its shortcut since, so a stale stamp can never skip a half-mirrored platform. Because unbinding keeps the
roms row (ADR-0007), a platform's persisted rows survive a partial re-apply or a local removal while RomM's id set
stays what the stamp recorded, so a stamp still able to skip would otherwise let the skip drop the un-recreated games
(the #1025 silent gap). Two rules keep the contract true. The unit applier deletes the stamp at a platform unit's
apply start (once the fetch succeeded and the apply is about to emit its first chunk) and only the final chunk
re-writes it, so an apply interrupted by a crash / cancel / heartbeat-timeout before the final chunk leaves none. The
local removals and the stale removal on an unprocessed platform revoke the stamp's skip
(PlatformSyncStateRepository.revoke_skip, the skip_revoked column of migration 025): the local destructive
flows — "Remove all shortcuts" and per-platform removals (via report_removal_results) plus the Steam-UI-deletion
reconcile (reconcile_live_shortcuts) — for every platform they touch, in the same write UoW as the unbind, and the run
finalizer's server-side stale removal for every platform the run did not process that it removes a row's shortcut
on, in the finalize's write UoW, whatever the row's binding by then; the panel's report unbinds that row and revokes
nothing (Steam non-Steam shortcuts). A platform the
run did not process is one that was not among its platform units: its sync is turned off, or RomM no longer lists it.
Where an enabled collection keeps one of that platform's games bound, a stamp still able to skip would let the platform
skip once it is processed again: the bound rows keep the zero-bound guard quiet, and RomM's id set still matches the
stamp's checksum, so the skip rebuilds the unit from the bound rows alone and never re-creates the unbound games'
shortcuts. On a platform the run processed, skipped or fetched alike, the stale removal leaves the skip, because nothing
the skip reads changes: a skipped platform's rebuilt rows are all among the rows the run returned, so none of them is
stale, and a fetched platform was stamped by that run with a generation no stale row carries. Removed-game cleanup's own
unbinds leave the skip too; why is under Removed-game cleanup.
A revoked stamp is kept rather than deleted, because the skip is not its only reader. Removed-game discovery
(Removed-game cleanup), its canary ids (PruneRegistry.canary_rom_ids) and the
Library page's reachable_count read the stamp's fetch generation to tell the rows RomM's last complete fetch returned
from the ones it did not. Such an unbind changes neither side of that: the rows keep their generation, and the
generation still records what RomM served. Deleting the stamp would blind discovery on the platform until its next
completed apply, which never comes while its sync stays off. The skip gate, the plan estimate (predicted_skip and the
stamp-gated collapsed_count) and the preview's restamp_platform_count read the stamp through
domain/platform_sync_state.py::stamp_for_skip, which answers a revoked stamp with none; what each of them breaks by
reading it raw is in the invariant register. The resume offer asks
PlatformSyncStateRepository.has_any, whose query applies the same rule, so that is the one other place it lives. The
final chunk's fresh stamp replaces the row, which clears the flag. The apply start deletes rather than revokes, because
its chunks re-mark rows with the new run's generation and a kept stamp naming the old one would make discovery read
every re-marked row as gone from RomM.
A collection unit fetches only the members no platform unit delivered (#2356). Its members come from the listing
build_work_queue already read: RomM's collection listings carry each collection's rom_ids, and the work unit keeps
them (WorkUnit.member_rom_ids). A member already in the run's synced_rom_ids — a platform unit delivered it, a
skipped platform's rebuilt rows included — is covered without a request. The rest are fetched by one walk of the
collection narrowed to the platforms the sync leaves off: the listed platforms with no platform unit in this run, a
skipped platform having one. The walk sends the collection's own filter plus platform_ids for each of them
(WorkUnit.unwalked_platform_ids), and RomM applies the two filters together. A run with every platform enabled
therefore fetches no page for any collection but a foreign smart one. The collection's membership map is the listing's
ids plus any the walk answered beyond them. A listed member neither covered nor answered is logged with the unit's name
and the count and left out of the run's new ROMs. Three cases are known: a scan racing the run, a ROM the walk does not
show, and an unbound sibling on a skipped platform. The last is left out of the platform's rebuilt rows, which hold only
bound rows, and the walk does not ask for its platform, which has a unit. The collection skip leaves unbound members out
the same way. An apply fetches every collection unit this way when it reaches the first of them, before any applies
("Group-aware sync" above); each unit leaves out what an earlier one delivered, so a member two collections list is
delivered by the one queued first.
A smart collection's listing is not its live membership. Its listed rom_ids are a stored column RomM computes for
the collection's owner and refreshes after the writes that can move it (in RomM 5.3.0 a scan, a ROM edit or deletion, a
save or state upload or deletion, and a membership change), while GET /api/roms?smart_collection_id= evaluates the
criteria live for the caller. A foreign smart collection is walked whole, unnarrowed: with per-user criteria
(favorite, saves, states, last played, statuses) its owner's set and the caller's differ, and the sync follows the
caller's view. Only the caller's own smart collections (domain/collection_owner.is_own_collection, which also reads an
unknown identity as own) take the listing's ids, and an own smart collection walks the platforms the sync leaves off
whenever there are any, even when its listing names no leftover. A refresh that fails is logged by RomM, not raised,
so the stored set can lag the live criteria, and a live member on such a platform that the listing misses would
otherwise go undelivered. Standard and virtual collections list their members live and walk only for leftovers. Under
the own owner scope no foreign collection is queued.
A collection skips by the platform skip's rules (#2477).
The collection sibling of the platform stamp, CollectionSyncState, is written when any collection unit's last apply
chunk commits, and the skip replays its member_rom_ids without a request beyond one limit=1 probe. Like a platform
stamp it records the fetch that wrote it (migration 032): fetch_started_at, Tender's clock read before the unit's
skip check — in an apply, while every collection unit is fetched, so before the earlier collection units apply — and
listing_rom_id_checksum, the platform skip's checksum over the rom_ids the collection listing carried. The checksum
covers the listing's ids rather than member_rom_ids, which adds the ids a walk answered beyond them; an own smart
collection whose walk answers more than its listing would otherwise never match its stamp. First the listing must match
the stamp (domain/collection_skip.py::membership_verdict): every kind compares the checksum of its listed rom_ids,
and a standard or smart collection also its updated_at, which RomM bumps on a membership change, a smart-criteria edit
and a rename. A virtual collection's updated_at says nothing — virtual_collections is a view — but its listed
rom_ids are live since RomM 5.3.0's migration 0101, and its stamp stores an empty updated_at. No rom_count is
compared. Then, for every kind, the scoped updated_after probe keyed off fetch_started_at less the platform skip's
ten-minute margin (domain/platform_skip.py::skip_reference) must answer a total of zero
(domain/collection_skip.py::probe_verdict); it filters with the kind's own collection parameter (collection_id,
smart_collection_id or virtual_collection_id). Between the two, a collection holding a member whose game still holds
more than one bound shortcut on a platform the sync leaves off is fetched without the probe
(domain/collection_skip.py::holds_a_game_to_converge; Sibling groups, "Convergence"). The reference is the fetch start
for the platform's reason: an edit RomM records to a member after the fetch began but before the final chunk committed
is newer than it. A stamp without both values, which is every stamp written before migration 032, never skips. A probe
that raises, or answers with anything but a JSON object whose total is an integer, falls open to a fetch. The sync log
carries each refusal's reason.
A hidden member changes the listing's ids, not its updated_at (read at RomM 5.3.1 and 5.4.0). An admin, group or
platform hide, and in 5.4 the age limit, removes the member from the collection listing's rom_ids and rom_count
without moving the collection's updated_at, so the checksum catches it, and an equal-count swap of members with it. A
user's own hide (rom_user.hidden) changes neither in a standard or virtual collection: it drops the ROM only from
/api/roms. Tender counts such a ROM as no member of the collection
(#2502 D4): LibraryFetcher.build_work_queue takes the run's
hidden set off each collection unit's member_rom_ids, which the checksum covers, so a hide or an unhide fetches the
collection once and a skip never replays a hidden member. In the owner's own smart collection it changes the stored
rom_ids and updated_at too, since RomM computes those for the owner. The collection does not compare the id set
/api/roms returns for it, as the platform does: that costs a request per collection fetch.
A collection fetched in full is re-stamped through an Apply, as a platform is. The preview counts each collection
whose skip it refused for a reason a fresh stamp settles, and whose apply will stamp it, in restamp_collection_count,
a summary field beside restamp_platform_count ("Unstamped-platform re-run" above). Two fields rather than one, so
neither names what it does not hold; the Sync page reads them alike. A fresh stamp settles a missing stamp, one without
a fetch start or checksum, a listing whose ids or updated_at differ, and a member updated after the reference, as for
a platform: the apply fetches the collection, and its final chunk records what RomM serves, built from the same listing
the next gate compares. It settles nothing where the probe raised or answered with something the gate could not read, or
where the collection holds a game still to converge, so such a collection is fetched but not counted, and a probe that
keeps failing does not offer the Apply on every sync (#2477
D3). The gate states which it was (domain/collection_skip.py::CollectionSkip), and the preview counts from that answer
rather than judging the stamp again, so the two cannot disagree. The apply stamps a collection with members and, for a
standard or smart collection, an updated_at its listing carried (domain/collection_skip.py::apply_stamps, which the
final chunk's stamp and the count both decide by); an empty collection and a standard or smart one whose listing carried
no updated_at are never stamped, so they are not counted.
A collection stamp is deleted, not revoked, when a local removal, the stale removal or removed-game cleanup's row
delete reaches one of its members. A skipped collection replays its CollectionSyncState's member_rom_ids and
passes over a member that is no longer bound, so a stamp standing over an unbound member lets the collection skip and
never re-creates that member's shortcut. The local destructive flows and removed-game cleanup's row delete therefore
delete every collection stamp whose member set holds one of the rows they unbind or delete, in the same write UoW as
that write, and the run finalizer deletes every one holding a row whose shortcut the run removes, in the finalize's
write UoW, whatever the row's binding by then (CollectionSyncStateRepository.delete_intersecting). Removed-game
discovery, its canary ids and the Library page's reachable_count — the readers a platform stamp is kept for — read no
collection stamp, so there is nothing to keep one for. The stale removal needs no set of the collections the run
processed, unlike its platform rule: a processed collection adds every member to the rows the run returned — every
stamped member on a skip, every member a platform unit delivered or its walk fetched otherwise — so none of its current
bound members is stale. For an own smart collection that holds because it walks the platforms the sync leaves off
whatever its listing says: a bound member its lagging listing misses would otherwise go unreturned. A collection member
whose shortcut the stale removal removes therefore belongs to a collection the run built no unit for, and that
collection is fetched rather than skipped once a run builds one for it again.
A listing that fails stops the run. The platform listing and the standard, smart and virtual collection listings are
read while the work queue is built, before any unit is fetched, and a failure of any of them raises to the same handler
on each path. A failure is any error the transport raises — a transient one only once its retry ladder gives up — and
also an answer that is not a list: the adapter's platform listing and its three collection listings
(adapters/romm/romm_api.py) raise that as a plain RommApiError, never an empty list and never RommNotFoundError,
which downstream reads as proof that one entity is gone, and a listing is about no single entity. Apply emits a
sync_progress frame at stage error carrying the classify_error message and returns before it opens a SyncRun, so
no stale removal runs and neither sync_collections nor sync_complete is emitted. The preview drops any staged
preview, emits the same error frame and raises the error on: a RomM error is answered with classify_error's reason
and message, and anything else reaches the panel as a transport error, for which the Sync page shows its own general
line. Read as an empty listing instead, the run would build no unit for an enabled platform or collection, and once any
other unit keeps the run from ending as "Nothing to sync", the stale removal would remove the shortcuts of the games
only it brings in; a collection would also lose its stamp (every kind has one), and the frontend's stale-collection
cleanup would delete its Steam collection. A collection absent from a list RomM did answer with is a real removal and
still goes that way. Only a kind with an enabled id is listed, and of the virtual kind only a type an enabled id encodes
(domain/virtual_collection_id.py reads it out of RomM's id; an id that does not decode lists every supported type), so
an endpoint that keeps failing blocks the sync only for a user who has switched on a collection it serves — switched on,
not synced: a foreign collection the owner scope drops from the queue still needs its kind listed. The platform listing
is always read. The Library page's get_collections reads the same collection listings only to display them: a failed
standard listing fails it, while a failed smart listing or virtual type is left out and the rest is listed. The platform
listing's other readers — the Library page's get_platforms and set_all_platforms_sync, and the connection test —
answer a RomM error from it, a non-list answer included, with classify_error's failure shape.
"Force Full Sync" (clear_sync_cache) clears every stamp (and every row's shortcut_applied), which is the entire
full-re-fetch + full-re-apply arm — the stamps are the fetcher's sole skip authority. The sync_runs history is
deliberately preserved (#1318): it feeds no skip gate and is the source of the "Last sync" display, so deleting it
forced nothing and only blanked the panel to "Never" right after a reset.
That preservation is also why the panel's resume offer is derived from the surviving skip authority, not from the run history (#1789). The history says only that a run ended without completing, and after a Force Full Sync it says that while everything it implied is gone — so a history-derived offer promised to continue progress that had just been discarded. The condition it replaced measured the wrong thing in the same way: it paired the incomplete attempt with "bound shortcuts exist", but Force Full Sync does not delete shortcuts, it deletes the stamps and clears the applied records.
A resume rests on skip authority, and Tender keeps two kinds, cleared together by that one reset: a completion
stamp (whole platform or collection skipped at fetch time, ADR-0023) or an applied shortcut (shortcut_applied,
one game skipped at apply time, ADR-0025). Neither subsumes the other, so the offer reads both. A run cancelled inside
its first platform unit reached no final chunk and holds no stamp, but its committed chunks wrote shortcuts and recorded
them applied — the next run genuinely does less work, so that is a resume. A row the sync has not applied carries no
record while its platform's stamp may survive, so an install can hold stamps and zero applied games and still skip those
platforms wholesale.
get_sync_stats therefore carries two additive fields alongside the display ones: resumable_games (bound rows whose
shortcut is recorded as applied) and has_completion_stamp (PlatformSyncStateRepository.has_any() or
CollectionSyncStateRepository.has_any()). Both ride in the read UoW that already scans roms — the game count is one
more condition inside that loop, since iter_all already selects the column, and each stamp probe is a
SELECT 1 … LIMIT 1 over a leaf table — so the panel mount pays nothing measurable.
The game count is bound-AND-applied, never applied alone. classify_roms (domain/sync_diff.py) sends an unbound
row down the NEW branch — if not reg or not reg.get("app_id") — before it reads the record, because the next run has
to mint the shortcut regardless. Requiring the binding is what makes the count fall to zero after "Remove all
shortcuts", where unbinding deliberately keeps the row and its record (ADR-0007) — a count over every row would keep
offering to resume shortcuts that no longer exist. The panel keeps roms > 0 as a conjunct on top, and it is
load-bearing rather than a restatement: has_completion_stamp asks whether any stamp that may still skip survives
anywhere, while the removal path is surgical — it revokes the skip of only the platforms' fs_slugs its removed rows
name, and deletes only the collection stamps whose member set intersects those rows. A stamp naming nothing the roms
table still holds outlives a remove-all. Prune makes that reachable: it deletes roms rows and never touches
platform_sync_state (PruneRegistry.delete_rows invalidates intersecting collection stamps only), so a platform whose
games RomM dropped keeps its stamp with no rows left to name it, and the next remove-all cannot see that fs_slug to
revoke its skip. Verified end to end: with such a stamp standing, a remove-all leaves roms == 0 and
has_completion_stamp == true, so without the conjunct the panel would offer a resume over zero shortcuts. It carries
the boundary rule too — a run stopped before a single shortcut was written begins from scratch. resumable_games is a
count of what is done, never of what is left — naming the remainder needs the server's library — so the QAM line
under the button states what a resume would skip, and is omitted when the count is zero.
Single-owner run lifecycle (#1202). The run-lifecycle pair — sync_state (idle/running/cancelling) and
current_sync_id — is mutated only through four verb methods on LibrarySyncStateBox, never by direct field
assignment from a sub-service. Confining those two writes to the box makes run admission, cancellation, and termination
a single compare-and-swap on the backend's one event loop, so a rapid Sync/Cancel can't interleave a stale terminal with
a fresh run's start and leave a half-reset id:
try_begin_run(run_id)— the admission guard. Compare-and-swap: returnsFalse(no state change) if a run is already in flight, else flips IDLE → RUNNING and stampscurrent_sync_id. Every entry point (start_sync,sync_preview,sync_apply_delta) goes through it, so an overlapping second Sync/Apply is rejected with{success: False, reason: "sync_in_progress"}instead of beginning a concurrent run that would double the terminal events. A rejected apply does not consumepending_delta, so the still-valid preview survives for the legitimate apply.request_cancel(run_id)— the run-scoped cancel routing (the #1198 / #1200 logic, centralized):"no_sync"when idle,"stale"when a truthyrun_iddoesn't match the active run (cancel ignored, run left RUNNING), else flips to CANCELLING. A falsyrun_idcancels unconditionally.cancel_syncandshutdowndelegate here.finish_run(run_id)— the terminal. Compare-and-reset: resets to IDLE + nullscurrent_sync_idonly ifrun_idstill owns the slot; a late, foreign, or doubled terminal is a no-op and can never null a fresher run. Every exit path (success, cancel, error, zero-unit) of the apply pipeline and the preview funnels into onefinally: box.finish_run(run_id)per run — the scattered per-unit / reporter resets are gone.is_in_flight()/is_cancelling()— read-only state probes for the per-unit loop and checkpoints.
sync_preview adds a final cancel checkpoint after the unit loop, immediately before it stages pending_delta, so
a cancel landing in that last window routes into the cancelled branch ({success: False, reason: "cancelled"},
pending_delta left None) rather than staging a delta the user already cancelled. Nothing awaits between that
checkpoint and the staging — the session-budget prognosis is taken before it, and the done frame emitted after it — so
the window the checkpoint closes cannot reopen. The confinement is enforced by scripts/check_sync_lifecycle_owner.py
(an AST gate that fails CI if sync_state / current_sync_id is assigned anywhere outside _state.py).
The preview outlives the panel that asked for it. PreviewDelta carries the exact answer dict sync_preview
returned, and get_pending_preview hands it back verbatim — the QAM card dies with its render, while the backend holds
the snapshot for its full 30-minute TTL, so a user who steps into a submenu used to lose an Apply that was still
perfectly valid. Storing the answer rather than re-assembling one is what guarantees the restored card is what the user
was shown. The answer also carries expires_at — an absolute epoch — which the card counts down against; absolute
rather than a remaining-seconds count because backend and panel share a clock and a deadline survives a suspend.
get_pending_preview answers {"success": True, "preview": None} for all three of nothing staged, a snapshot past its
TTL (dropped on the spot), and a run in flight — "no preview" is a normal answer, not a failure.
That third case is a backend guarantee rather than a frontend courtesy, because the panel cannot get it right on its
own. It derives "a run is live" from the sync_progress store, and abortOptimisticSync retracts the optimistic
running: true on every failed apply — including the sync_in_progress rejection, which is exactly the branch
where the backend deliberately keeps the staged delta (#1202). The store therefore reads idle during a live run, and
a panel that let the restored card decide the body would render it over the run's own progress rows, with the next Apply
retracting the live run's UI a second time. A JS-context rebuild mid-run reaches the same state through an empty store
racing the get_sync_status seed. So the run-in-flight case is refused where the truth is:
LibrarySyncStateBox.read_restorable_preview. The panel's own defence is a rendering rule rather than a second check on
the read: a run in flight owns the body, so a preview held while one is going is kept and not shown, and its card
appears the moment the run ends.
Frontend side: the pending preview is a module store, not panel state. frontend/src/utils/pendingPreviewStore.ts
owns it. The reason is the same lifetime mismatch from the other end — sync_preview is an awaited endpoint call, so
its answer is delivered to the closure of whichever MainPage pressed Sync, and leaving the main page unmounts that
instance while the run carries on. The card then never appeared for the instance actually on screen: the panel showed
the idle Sync buttons over a preview the backend was holding, and only the next mount's get_pending_preview
recovered it. The store is not a second source of truth — it holds the backend's answer verbatim, is only ever filled
from sync_preview / get_pending_preview, and never derives or repairs one. Its ordering rule (a write takes a ticket
when its information was issued; an answer applies only if no later-issued write already has; a read only ever fills,
because preview: None conflates three different situations) is stated in full in that module's docstring. Two triggers
fill it — the panel's mount, and a preview run reaching its terminal stage with the store still empty — and, because
both write to the same destination, the race between them is "who writes first", not "which copy wins".
Withheld, not discarded. A run in flight suppresses the snapshot for the duration and nothing else — the same
payload is handed back on the next mount after the run ends, as long as it is still inside its TTL. And the withholding
belongs to that reader alone: sync_apply_delta's age check goes through the run-blind read_fresh_preview, so an
overlapping apply is still refused by the run-slot claim with sync_in_progress instead of being rewritten into a
staleness verdict.
pending_delta is box-owned, like the run-lifecycle pair. The staged snapshot's whole lifetime runs through the
LibrarySyncStateBox verbs — stage_preview / read_fresh_preview / read_restorable_preview / matches_preview /
discard_preview — and no module outside _state.py assigns the field. The TTL lives with them:
PREVIEW_MAX_AGE_SECONDS and the preview_deadline the answer's expires_at comes from are the box's, so the number
the card counts down to and the verdict read_fresh_preview reaches cannot drift apart. "Too old" has exactly one
expression, PreviewDelta.is_expired, reached only from read_fresh_preview — which both the pending read and
sync_apply_delta's age refusal go through, so the read can never offer an Apply the apply would reject. Unlike the
run-lifecycle pair this has no CI gate; it is prose plus the box's own tests.
In sync_apply_delta the ordering is load-bearing and deliberately left as three visible steps — identity check,
run-slot claim, then discard — rather than one atomic "take" verb: a rapid second apply, or one landing while a run is
in flight, is refused by try_begin_run before discard_preview runs, so the still-valid preview survives for the
legitimate apply (#1202).
sync_progress carries runId. Every sync_progress event (and the persisted get_sync_status snapshot) now
includes an additive runId: str field — str(current_sync_id or "") — so the frontend reads the active run id from
the authoritative backend store rather than minting or threading its own. The idle default snapshot carries runId: "".
The snapshot describes an in-flight run only. finish_run puts sync_progress back to the idle default as part of
ending a run — after the terminal frame has gone out, since every path emits or schedules it before reaching the
finally: finish_run(run_id) that lands there. (The per-unit error path schedules its emit as a task of its own, so
that one may run after the reset; it is safe because the coroutine binds the ERROR frame by value and the reset
rebinds the attribute rather than mutating the dict a pending task is holding.) It exists so a remounting QAM can pick
up a live run; a finished run's ending already reached the panel as an event, and a panel that merely finds a
terminal frame on a later mount is required to ignore it (#1019). Left in place, the frame answered every later mount
with a run that had ended — message and run id included — and the panel took it for the state of the run it was actually
watching: pressing Sync, then returning during the window before the new run's first frame, showed a green "Preview
ready" over a preview that was still going, and dropped the panel back to the idle buttons in the meantime. The reset is
not itself a frame: nothing emits it, so no running: false reaches the panel from it.
get_sync_status answers with the run lifecycle too. The payload carries an additive inFlight: bool —
box.is_in_flight(), the lifecycle state itself, not a re-reading of the frame. It rides this answer only, never an
emitted sync_progress event, and the two legitimately disagree: during a cancel drain the CANCELLED frame already
reads running: False while the run still owns the slot.
The panel needs it because a frame alone cannot separate two questions it must answer differently. When the module store
says a run is live and the answer says nothing is running, retracting is right if the run really ended (its terminal
frame was lost) and wrong if the backend simply has not heard of it yet — which is exactly the start window, where the
panel has written an optimistic frame and sync_preview has not yet claimed the run slot. So the seed retracts only
where the answer is evidence about that run: either it names it, or inFlight is explicitly false and the
store's run carries a backend-stamped id. An idle answer against the panel's own unstamped optimistic frame is refused,
and that cannot wedge, because the handler that wrote that frame always retracts it itself — from a dead instance too.
Inferring the lifecycle from the frame instead would wedge the panel: with the snapshot reset above, a finished run and
a run that never started are the same answer, so a lost terminal frame would leave every later mount believing a run is
live. Nothing recovers from that state — the start controls all live in the idle branch, "Cancel Sync" for an unknown
run is answered {"success": True, "message": "No sync in progress"} with no terminal to follow, and Data Management
and Library › Platforms gate removals on the same flag.
The same sync_plan capture point also clears the frontend's per-run cancel flag (_cancelRequested). The per-unit
handler resets that flag at its own start, but an incrementally-skipped unit never runs that handler, so a skip-only
run could otherwise carry a stale cancel from a prior cancelled run; resetting once per run on sync_plan is the
reliable reset.
Sync time estimate and live ETA (frontend)¶
The QAM's time readout is a two-stage design layered on the sync_progress stream. It is pure frontend logic
(frontend/src/utils/syncEstimate.ts and frontend/src/utils/syncEta.ts, both unit-tested); the backend supplies the
plan (per-unit weights + planned totals, via sync_plan) and the applying frames.
- The plan is skip-aware (#1382). At plan time the fetcher stamps every platform
WorkUnitwith two estimate-only riders read in one short UoW (_read_plan_estimates+ the puredomain/skip_prediction.py):predicted_skip— whether the wholesale incremental-skip gate is expected to skip the platform, replaying the gate's local conditions only (a completion stamp present that records a fetch start and an id checksum, rows carrying the stamp's fetch generation, bound rows, no sibling-group-key backfill pending for a row of that generation, no group holding more than one bound shortcut; the gate's one request,list_roms_newest_first, is deliberately not replayed — no network at plan time, so a platform RomM changed since the stamp is predicted as a skip the fetch then refuses) — andcollapsed_count, the persisted post-collapse shortcut count, mirroring the collapse (ADR-0021): one per sibling group, since a group with several bound duplicates converges to one (#2645), plus one per keyless row. Every collectionWorkUnitcarriespredicted_skiptoo (#2623), read in the collection riders' own short UoW (_read_collection_estimates):domain/skip_prediction.py::predict_collection_skipreplays the collection gate's conditions before its probe, in the gate's order and through the gate's own functions — the stamp can record the listing'supdated_at(a virtual collection's always, another's only when the listing carries one), a completion stamp exists, the listing matches it (membership_verdict) and it records a fetch start, and no game among its members still holds more than one bound shortcut on a platform the sync leaves off (holds_a_game_to_convergeover the stored rows). The probe is not replayed, so a collection whose member ROM RomM edited since the stamp is predicted as a skip the fetch then refuses, as on a platform. Both riders ride thesync_planpayload conditionally-present:predicted_skipis absent only on a failed read, whilecollapsed_countis absent on collections, never-synced platforms and failed reads, and is additionally gated on the platform's completion stamp (#1412) — a never-synced platform holds only PARTIAL collection-sibling rows (ADR-0021), so an ungated count would weight the ETA below the true work, and without a stamp the frontend falls back to the rawrom_count. The payload'stotal_romsstays the raw pre-collapse total (backward compat); an additivetotal_estimated_itemssums0for predicted skips, elsecollapsed_count ?? rom_count. A third rider,bound_count(#1511), counts the unit's known ROMs that already carry ashortcut_app_id, and rides both unit kinds, aspredicted_skipdoes. On a platform it counts the persisted rows, read in the same short UoW the skip prediction already needed that count for, and is not stamp-gated: a bound row genuinely has a Steam shortcut whether or not the mirror is complete, and zero persisted rows honestly means "every planned item is a create". On a collection it counts the bound members of the completion stamp's storedmember_rom_ids(the same member set the skip replays), in one short read UoW covering every collection unit — no ROM fetch. The two sides are deliberately asymmetric on the empty case: a platform reports0, an unstamped collection is omitted and so priced as all creates. The collection count is read from the stamp's stored member set only, so without a stamp there is no count to report, and0would state one. Absent and0price identically today; the distinction keeps the field honest for later consumers, so do not collapse it into consistency. A collection's stored member set may be stale if membership changed since the stamp — accepted, since this is estimate-only. A fourth rider,new_shortcut_count(#1517), is the create-side complement ofcollapsed_count: the shortcuts the next apply must mint rather than update — sibling groups with no binding anywhere, unbound keyless rows, and every server ROM the local mirror holds no row for (rom_count − persisted rows, clamped at zero). It is platform-only — a collection's rows belong to their platform's unit, so counting creates on a collection too would price the same shortcuts twice — and likebound_countit is not stamp-gated: an unbound group genuinely has no shortcut and a ROM with no local row genuinely has to be created, whether or not the mirror is complete. The frontend takes it as its create term directly instead of deriving creates by subtractingbound_countfrom the unit's weight (see the composition-priced seed below). Hard constraint (ADR-0023): the prediction never feeds the actual skip decision —_try_unit_incremental_skipand_try_collection_incremental_skipat fetch time remain the sole skip authority, so a mis-prediction can only make the estimate read long or short, never mis-apply. A Force Full Sync needs no special case:clear_sync_cachedeletes every stamp before the run, so a forced plan predicts no skips and drops everycollapsed_count— the unit is then weighed at the full pre-collapserom_count, butbound_countandnew_shortcut_countboth survive the clear (they read the rows, not the stamp) and keep the forced re-apply priced by composition (#1517).get_platformscarried the same count as a per-platform garnish for the old toggle label; the Library page's list shows RomM's ownrom_count, so the garnish had no reader and the whole-table scan it cost — inside aBEGIN IMMEDIATE, on the read that gates the page's first paint — went with it (#1815). - Static walk-cost ceiling (pre-run seed). Before a run — in the preview, and again as the initial "up to X min" the
instant a skip-preview run starts — the estimate is a pure cost model over three independent terms (#1511),
because a run is three independent phases and one blended per-item rate cannot describe a mix of them:
created × NEW_ITEM_SEC+changed × UPDATED_ITEM_SEC+(created + cover_refresh_count) × COVER_DOWNLOAD_SEC+ a flat fixed-overhead allowance. Each constant is calibrated to its own measured mean with a ~15% ceiling margin (0.36 s per created shortcut against a measured 0.314; 0.13 s per update against 0.109; 0.15 s per cover download against 0.132 cold), and the allowance (45 s) covers the run's genuinely fixed cost — the one-time shortcut scan, the multi-page ROM/save fetches, the inter-chunk gaps, finalize (measured 17–24 s). So the seed still reads long, never short, but it now reads long by a margin rather than by a factor. Two consequences worth stating: an update carries no artwork cost (the apply loop gates cover application oncreated), and a cached cover is not a term — warm covers cost 0.0018 s, 73x cheaper than cold, and pricing them would need a backend cache probe in the preview path that is deliberately not added, so every create is priced as needing a cold download. - Composition-priced plan seed. The skip-preview seed prices the plan per unit by composition, not as one
blended item count (#1511): a predicted-skip unit costs nothing, the unit's already-bound ROMs (
bound_count) take the update rate, and the shortcuts that genuinely have to be minted (new_shortcut_count) take the create rate. Pricing every planned item as a create over-read by ~4x on the common case — any re-sync, and every Force Full Sync, which clears the completion stamps but unbinds nothing and is therefore an all-updates run. - Why the create term is read, not subtracted (#1517). The two rates are priced from independent counts; the
create term is
new_shortcut_countdirectly, neveritems − bound_count. The subtraction over-read a Force Full Sync of a sibling-heavy platform by ~2.5x: a forced run dropscollapsed_count, so the unit is weighed at its pre-collapserom_count, and on a platform carrying sibling groups (ADR-0021) that raw count exceeds the real shortcut count — one row per group is bound, its duplicates are not. Subtracting the bound rows priced every unbound duplicate as a phantom new shortcut, at the dear create rate plus a cover download it would never perform, even though a sibling duplicate never produces a shortcut.new_shortcut_countreports no creates there instead, because the clear takes the stamps and not the bindings, so no group is left without one. The symmetric hazard is a never- synced platform holding only PARTIAL collection-sibling rows (ADR-0021): counting creates from the known rows alone would price a handful of items for a whole platform — a short read, the one direction this estimate may not err in. Therom_count − persisted rowsterm prices those unmirrored ROMs as creates, so the seed stays at (or above) the full platform. A unit whosenew_shortcut_countis absent (collections, older backends) falls back to theitems − bound_countsubtraction, the pre-#1517 behaviour. - Collections still price by bound members. Platform and collection units are priced the same way on an ordinary
re-sync, so a collection-heavy library does not reinstate the over-read through its collections. A Force Full Sync
is the exception for collections:
clear_sync_cacheclearscollection_sync_statewholesale, and a collection's count is read from that stamp only, so every collection unit is unstamped for that run and reverts to create pricing. Platforms are unaffected — theirbound_countandnew_shortcut_countare deliberately not stamp-gated. This follows directly from theNone-not-0rule above and errs long, the safe direction. A unit whosebound_countis absent (older backend, unstamped collection) prices as all creates, the pre-#1511 behaviour. The live-countdown weights are unaffected and staypredicted_skip ? 0 : (collapsed_count ?? rom_count). - Measured live countdown (takes over within seconds). Once the apply is underway,
syncEta.tsmeasures the real rate from the applying frames — one throttled sample per second over a ~30 s sliding window — and projectsremaining = (planned_total − processed) / rate, rendered rounded up ("9 min left") so it never promises less time than it expects. It replaces the static seed as soon as the window spans enough real time to trust the slope (a couple of samples across a few seconds). Because it reflects the actual mix of cheap-update vs. full-create work, it is far closer to reality than the ceiling and ticks down as the run proceeds. - Estimator-owned sticky deadline. The estimator, not the UI, owns the displayed value: each fresh measurement
re-anchors an absolute wall-clock deadline, and the countdown renders
max(0, deadline − now). This is what keeps the readout smooth. The raw measurement re-arms to "not ready" between measurement segments, and a run's tail of small units each finishes inside the readiness window — so a raw snapshot would blink back to the static "up to X" seed for the whole tail; holding the last good deadline through those gaps keeps the countdown honestly ticking down instead. - Segment break across fetch gaps. Applying frames arrive roughly every second during real apply work, so a silence longer than ~10 s is always a unit or fetch boundary, never apply progress. Crossing it starts a fresh measurement segment (the prior samples are discarded), so the slope is never measured across the gap — pairing a pre-gap sample with a post-gap one would be a tiny item delta over a long span, an absurd rate that would briefly spike the countdown.
- Stalled-prefix trim (shorter stalls). A stall too short to break the segment used to poison the window head instead (#1511). The applying stage is entered — and its first frame emitted — before the one-time shortcut scan, which then blocks for ~9–14 s (it scales with the existing library, not the delta). That frame became sample #1 and paired with the first post-scan sample: a couple of items across the whole scan, a rate ~10x below the real one, which the sticky deadline then held until the 30 s window slid past — the reported "6 min → 2 min" collapse. Leading samples are therefore dropped when the head gap is both wide (> 3 s, several sampling cadences) and unproductive (fewer than one item per sampling interval, below anything the 50 ms-paced apply loop produces). Both conditions are required: a wide gap that carried real throughput is slow work, not a stall, and discarding it would measure only the fastest stretch. Trimming can legitimately leave a single sample — "no measurement yet" is the honest answer while a stall is all the window has seen, so the static seed keeps standing until clean samples accumulate.
The coarse QAM progress bar reuses the same run weights (weightedCoarseFraction, syncEta.ts): each unit's bar width
is its item-weight share — skip-aware at plan time, corrected to the real delta as units dispatch — with the within-unit
fill scaled by the running unit's share, so a predicted-skip platform occupies no width and a huge platform fills the
bar in proportion to its real work. It falls back to the old equal-per-unit index weighting when no plan is measured
(QAM opened mid-run before any sync_plan, an older backend) or the plan can't apportion (unit-count mismatch, all-zero
weights).
Two run-scoped latches keep the coarse fraction monotonic so the bar only ever moves forward. A run's leading
zero-weight units still refresh covers, so rather than pinning the bar to zero they each claim an equal 1/totalUnits
slice as a floor, with the weighted shares compressed into the band above it (#1506); that floor is held at its
high-water mark, so raising a mispredicted skip off zero can't shorten the prefix and retract it. On top of that, the
returned fraction itself is latched at the run's high-water mark by latchedCoarseFraction — the wrapper MainPage
actually calls, keeping weightedCoarseFraction a pure reader (#1509): when observeUnitTotal corrects a mispredicted
trailing skip up (0 → its real delta) it correctly grows the countdown's totalRoms (the run really is longer), but
that shrinks the bar's completed/total ratio and would retract width already shown, so the bar holds while the countdown
lengthens. Both latches touch only the bar output; the live ETA still reads the corrected total. The null fallbacks
(no run, unit-count mismatch, zero total weight) pass through the wrapper un-latched.
The within-unit fill is itself split into three monotonic sub-slices (#1407, withinUnitFraction in
frontend/src/utils/syncProgress.ts): fetch (FETCH_SHARE 15%) → covers (COVERS_SHARE 25%) → apply (APPLY_SHARE
60%). A unit is worked in that order — paginate the ROM list, download/refresh cover art, then create the shortcuts —
and each phase fills its own slice by its own current/total, with a later phase's floor sitting at the sum of the
earlier phases' shares. So the bar advances continuously through a unit's fetch and cover phases instead of resting
frozen at the unit floor until applying, and it never jumps backwards at a phase boundary even though each phase
restarts current/total from zero (each phase's frames land in a strictly-higher band than the phase before). The phase
is tagged on the sync_progress payload's additive subStage field (camelCase, matching the sibling totalSteps /
runId keys — the emit_progress Python kwarg is sub_stage, the emitted key is subStage): the fetcher's per-page
frames carry subStage: "fetch", the artwork cover-refresh and download frames carry subStage: "covers" (both
share the covers slice), and the frontend-driven apply frames are keyed on the applying stage alone — so a merged
apply frame still carrying a stale subStage is unaffected. Within the covers slice the fill is not strictly
monotonic: the covers phase runs two sequential passes — the cover-cache refresh first, then the download for the apply
set — each counting current/total from zero, so a mixed unit (changed covers to refresh and new covers to
download) can dip backwards within the covers band at the refresh→download handover, bounded to the covers share of
that unit's slice. The fetch→covers→apply phase handovers themselves stay monotonic (the common cases don't dip either:
a first full sync refreshes nothing, and a steady incremental sync with no cover changes runs neither pass). A frame
with no sub-stage (a platform unit's fetch anchor, or a pre-#1407 backend) rests at the unit floor, the old behaviour.
The frames of the fetch of every collection unit fill the first collection unit's fetch slice by the count of
collections fetched (its walks' page counts stay in the message), and a collection unit's own anchor then fills that
slice whole, since its fetch is done, so the first collection's row never steps back between the two. emit_progress
writes subStage into both the emitted event and the persisted get_sync_status snapshot, so it rides the QAM-remount
re-seed path too.
The whole thing is an approximation by design — though a narrow one since the plan went skip-aware: seed weights and the
applying frames usually both count post-collapse shortcuts now, the raw pre-collapse rom_count survives only as the
fallback where the backend doesn't know better, and a unit the plan mis-predicted re-corrects on its first
sync_apply_unit (observeUnitTotal). Estimate degradation is strictly one-directional: a wrong prediction or a stale
collapsed count makes the readout run long or short, never changes what the sync applies. The readout wording ("up to",
rounded-up countdown) keeps it an estimate, never a guarantee.
Save-sync serialization (device gate)¶
Every save-sync run on the device passes through a single device-level serialization gate (SaveSyncGate, in
services/saves/sync_engine/_gate.py). Only one save-sync run is in flight at a time per device: when a second
trigger fires while one is running, it queues — it waits for the in-flight run to finish rather than running
alongside it. An asyncio.Lock is the serializer/queue; the gate owns the acquire discipline around it and the record
of which trigger holds it (held_by, holder(), a GateHolder) — no run ids, no cancellation.
Three of the four trigger methods on SyncEngine bound their wait — post_exit_sync (60 s), sync_rom_saves (15 s),
sync_all_saves (60 s). If the gate can't be acquired within the budget the call refuses instead of blocking, and all
three raise the same busy refusal, SyncBusy: reason: "sync_busy", no offline detail. A busy gate is a local
wait — nothing on that path observed the server — so it never borrows a server-reachability slug and never sets the
offline flag (#1625); the session-end toast would otherwise announce "Server offline" about a reachable server. Each
caller routes the skip on success: False alone, and the session-end toast keys on the reason. The lock is never leaked
on timeout — a timed-out acquire releases any photo-finish hold before raising.
pre_launch_sync gives the gate no budget of its own: it waits until the gate is free, as it waits for the ROM's lock,
because whether to keep waiting is the user's choice at the start. While it waits, pre_launch_sync_state answers whom
for (the holder's GateHolder), and stop_pre_launch_sync ends the wait without a write; both, and the checks the sync
makes for its stop once it runs, are Pre-launch sync's.
The gate sits outside the per-ROM lock (SyncEngine.rom_lock(rom_id)): the device gate admits one run at a time
across the whole device, then each ROM still takes its own rom_lock for the read-mutate-write of its
RomSaveSyncState. The cheap stateless early-out (is_save_sync_enabled()) runs before the gate, so a
disabled-feature call never queues behind an in-flight run just to report it's off.
FirmwareService decomposition (services/firmware/)¶
The BIOS subsystem is a façade over its sub-services plus the demand reader they share. What the resolver says and what
the RomM library holds are two different questions, and the split is drawn where they meet: demand.py answers the
first and listing.py the second, and every surface below joins them rather than asking either source twice.
| Module | Role |
|---|---|
service.py |
FirmwareService façade — public use cases; wires the sub-services and delegates |
listing.py |
FirmwareListing — the firmware subsystem's only reader of RomM's platform list, and so of the library's firmware listing. It reads the platform list (/api/platforms carries each platform's firmware), files each file on the platform its platform_id names and leaves out the files RomM marks missing_from_fs, whose bytes are gone from the server (domain/firmware_platforms.py). It owns the TTL cache, in memory and in the SQLite firmware_cache table; a download and a delete both invalidate it through this one instance |
demand.py |
FirmwareDemand — the machine's side: the per-platform verified reading every status answer, Download required and "Replace…" are built from |
status.py |
FirmwareStatusReader — get_firmware_status (whether the RomM firmware listing could be read, and nothing about any platform), get_platform_firmware_status (one platform's whole entry, the call that pays the reading) and check_platform_bios (the per-game check), the aggregates they answer from, and the per-platform delete count |
emulator_answers.py |
What the resolver says about each emulator of a platform: the launching emulator's answer and the words a surface puts to it (emulator_fields), and one entry per emulator it answered for (emulator_tabs, #2190 D15) |
downloads.py |
FirmwareDownloader — the three download entry points: each fetches into the platform's store folder and then brings the platform's links up to date |
store.py |
FirmwareStore — Tender's firmware store: fetching a file into it, the links into the emulator sources, removing both, and what is recorded under Tender's folder (The firmware store below) |
rows.py |
One platform's rows: the resolver's requirements, what fills each, what lies at each place, and the library section |
replacement.py |
FirmwareReplacer — "Replace…", which asks every condition again off a fresh reading before anything moves |
deletion.py |
PlatformBiosDeleter — every BIOS delete: the store's files with Tender's own links to them, and the downloads made into an emulator source's folder before the store, at the path their records hold |
The two resolver seams, the scope that tells them apart, and the reasoning behind every answer they produce are in the FirmwareService notes — the live firmware resolver section below.
The firmware store¶
A BIOS file Tender downloads lands in its firmware store,
<Tender's folder>/bios/<the platform's folder in RomM>/<the
file's name in RomM>, where the platform is the one the
download was made for (#2191 D3). It is written to a .tmp
beside its store path and kept only when its MD5 is the one RomM states (#2191 D9); a file RomM states none for is not
downloaded. Every file is kept as it comes, two files with the same bytes included, and a file two platforms need lies
in each of their folders, so no link points into another platform's folder. A file already lying at a store place with
no record of Tender's is taken over where its MD5 matches and otherwise moved to the backup folder first (#2191 D25).
The store is the first store that records something under Tender's folder (#2385 D2): FirmwareStore answers
TenderFolderRecordReader, every write into the store goes through do_prepare_for_write, and every use case that
writes into the store or links into it names the folder rule.
The platform's folder in RomM is its fs_slug (#2219
D11, #2223 D8, D25). It names one platform where two share a
slug — RomM's built-in gc and gamecube folders are both ngc — so each keeps a store folder of its own, and Delete,
the platform page and a file of the same name stay with the platform RomM lists a file under. A platform's rows, its
Download required and its Replace draw only on the firmware RomM lists under its own folder
(domain/firmware_rows.py::platform_library, #2223 D48), as RomM keeps a platform's firmware there: a file another
platform's folder holds fills no requirement here, and a requirement its own folder holds no file for reads as not in
the RomM library. Store and link records are keyed by that folder; a store record keeps the slug beside it, by which
Tender finds the platform's system and its emulator pick. Every platform use case takes both, and refuses
invalid_value without a folder rather than asking by the slug. The game page asks by the folder the last sync recorded
on the game (roms.platform_fs_slug, #2223 D45); until one is recorded its BIOS read answers as a check that could not
run.
An emulator reaches a store file through an absolute symlink at the place the resolver names for it (#2219 D4). The
link goes in for every emulator of the platform whose requirement the file fills and that its source does not state not
installed, in every enabled emulator source (D23, D36): the source's own launch reading (emulators_for, each entry's
availability) is asked beside its firmware reading, and only not-installed on every entry the source lists the
emulator under rules it out — one it cannot establish is linked for (D37). The launch reading mainly guards a carded
standalone emulator, whose card states what it wants whether or not it is installed. A sealed emulator catalogue
(EmuDeck's) is read for its places all the same, while its rows and verdicts stay withheld and its list never decides
that a link is unwanted (AtlasPlatformFirmwareAdapter.every_source, D37, D40). A link goes only where nothing lies
under that name: Tender places no link over a file, a folder or a link that is not its own, and changes nothing there
(D13). For a folder an emulator lists (LRPS2's pcsx2/bios) every store file the resolver recognises for it is linked
into it under its name in RomM (D26). A place under a root the resolver only assumed gets no link, nor does any place of
a source whose roots could not be asked, and nor does a place whose folder is not there — Tender creates none (D11,
2219 D6). For the first two the row says so, naming that source (D32). Which file fills which requirement is¶
domain/firmware_match.py: its MD5 against the checksum the resolver pins for the requirement, by its declared name
only for an archive or where no checksum is known (D2, D8, D16); where several fill one, the alphabetically first is
linked.
Two records. firmware_store holds one row per store file; firmware_links one row per link, keyed by its place,
naming the store path it points at rather than the store row (domain/firmware_store.py). A link is Tender's own when
it points at a store file Tender has recorded, wherever it lies (D24), or where Tender's record of a link at that place
names the path it points at — which is what keeps a link Tender could not remove with its store file its own (D20).
Bringing the links up to date (FirmwareStore.do_update_links, domain/firmware_places.py::plan_link_update)
places every link an emulator of the platform wants where the name is free, places again a link of Tender's that points
at nothing where a store copy exists, removes such a link where none does, wherever its place may be touched, removes a
link of Tender's that still points at its store file where no emulator of its source that is not stated not installed
wants its place any more — only in a source whose list it read whole, never a sealed one, and whose roots it may touch,
so such a link in a source switched off, not read or sealed stays (D36, D37, D40) — records a link of Tender's found at
a new place, and drops the record of a place it may touch where no link of Tender's to the recorded file lies any more;
it downloads nothing (D6). Beside it, "Replace…" places the one link at the place it cleared (do_place_link), and a
delete and "Forget what is recorded there" remove the links of the files they take (do_remove,
forget_records_under); nothing else places or removes a link. Every download ends with it; so does a replacement; and
the panel asks for it (update_bios_links) when a platform page or a game's page opens and after an emulator switch.
Before such a link goes, every other platform is asked whether it still wants the place (D42): every platform whose
ids are kept — each one RomM listed at the last sync, and any kept since, which PlatformSystems.listed_platforms
answers without reading RomM — and every platform Tender holds store files for. A listed platform is asked whether or
not Tender holds a file for it. Each platform's system is asked first and each system read once in every enabled source;
an emulator its source states is not installed wants nothing, and so does an installed one whose wants the resolver
could not read (unread_emulators), only because the resolver states no place for it (D43) — any emulator its source
does not state not installed wants every place the resolver states for it. A sealed list's places count as wanted, and a
folder declaration wants a place in its folder only for a copy it recognises (domain/firmware_rows.py::wants_place).
The asking stops once every link is known to be wanted. A link stays wherever the answer could not be had: no listing
was ever kept, a platform's system could not be asked, or a source could not be read for its system
(SourceCatalogues.unread) — a platform or source that could not be read is never counted as wanting nothing, and the
next update asks again. A platform with no system in the answering source wants nothing; one whose system is switched
off is asked by that system all the same.
A file the resolver reads as wrong is replaced only on the user's confirmation, and only where the library copy is
proven right by its checksum (D5): the copy is fetched and checked first, the file in the way goes to
<Tender's data folder>/backups/bios/<date and time>/<its original absolute path> (D17), the date and time on the
machine's own clock, written YYYY-MM-DD_HH-MM-SS (domain/firmware_store.py::backup_stamp, D38), and the link takes
its place. Where the link cannot be placed, the old file goes back only while its place is still free, and stays in the
backup folder otherwise; the refusal (replace_failed) says which (D33, D34). Neither the backup nor the put-back lands
over anything: the file adapter claims the new name by creating it — a link as itself, a file by a hardlink, or, where
none can be made, by a copy flushed to disk — before the old name goes (FirmwareFileAdapter.move_aside). "Download
required" never replaces anything.
Removing takes Tender's own files and links only: a row's Delete removes the store file and every link of Tender's to it, the platform's Delete every store file of the platform and their links (D3). A Delete is never refused because a place cannot be reached: a link under a root the resolver only assumed, or of a source whose roots could not be asked, stays with its record, the answer names it, and the next link update removes it or places it again once that root is read (D20). BIOS files Tender downloaded into an emulator source's folder before the store existed stay where they are with their download records, and their Delete removes exactly the recorded path, only inside the BIOS folder RetroDECK names now (D12, D29): a record whose file is gone is dropped without a word; while RetroDECK's settings cannot be read, RetroDECK is not installed, or it names no BIOS folder (D35), the file stays with its record and stays counted; a file outside that folder stays and its record goes. The answer names each file it left and why.
DownloadService notes¶
The package is split along its jobs, as services/library/ is, and bootstrap/ builds every part and hands it in,
so the façade builds none of them:
DownloadsStateBox(_state.py) holds what the façade and the transfer share: each download's in-progress claim, its task, byte reservation and control token, and the semaphore. They change only through its verbs: a start takes its claim and drops it when it ends without a download (claim,drop_claim); once its pre-flights pass it registers the task, the reservation and the token (admit); the task gives back the claim and all three (release), and only while its token is still the one registered, so a superseded task never releases a newer attempt's; the shutdown forgets the tasks it has ended (forget_tasks).DownloadQueue(queue.py) holds the entries and changes them — an admitted download's entry joins it (enqueue) in the same synchronous step as the box'sadmit, andprune,clear_completed_downloads,evictandcleartake entries out; the others read the entries and never change them. It sends everydownload_progressframe that reports one, the frames nobody awaits as tasks of its ownBackgroundTasks, whichDownloadService.shutdownawaits after the façade's own.DownloadTransfer(transfer.py) is an admitted download's task: the transfer, what the finished transfer becomes (throughDownloadUnpacker), the extracting progress, and how the download ended — completed, paused, cancelled or failed._names.pyholds the safe file and folder names and the partial file's size, as functions both the façade and the transfer call.DownloadService(service.py) is the one surface the endpoints and the other services call: admission (the occupancy gate, the sibling supersede, the disk-space pre-flight), control (cancel, pause, resume), the installed-ROM read and the shutdown.
The progress frame, the queue's answer and the reservation arithmetic are pure functions in domain/download_frames.py
and domain/disk_space.py.
A download fetches a game's own files, not its extras
(#2223 D27, D30). RomM files each of a ROM's files under a
category (RomFileSchema.category, RomM 5.4.1): a file directly in the game's folder is game unless RomM's own read
of the file says otherwise — its Switch title-id read, on by default, files a top-level update or DLC as update or
dlc — a file in a folder directly below the game's folder named after a category, singular or plural, takes that
category, and any other nested file takes none. A download fetches the game files and the files in no category — a
PS3, Wii U or DOS game's own content lies in no category folder — and leaves every other category on the server:
manuals, walkthroughs, soundtracks, screenshots, cheats, patches, translations, hacks, mods, demos, prototypes, updates
and DLC. Where that leaves nothing, every file is fetched. That split is game_files (domain/game_files.py), and the
fetch one download makes is plan_fetch (services/downloads/_fetch.py), which the admission and the transfer both
call so they read one answer from the same ROM detail. Only where something is left out does the content request name
the files, ?file_ids=… on GET /api/roms/{id}/content/{name}; RomM serves one remaining file as itself and several as
a zip with a .m3u it builds from exactly those files, unless the ROM holds a .m3u of its own. Where the list would
make the request line longer than 8 KiB (MAX_REQUEST_LINE_BYTES, the whole URL counted, which errs long: only a
forward proxy is sent the whole URL, while RomM or a proxy in front of it is sent the path alone; RomM's own nginx takes
16 KiB, a proxy in front of it may take less), or a fetched file has no id, the whole game is fetched instead, and the
unpack takes the left-out files back out of the folder, with every folder that held nothing else, and drops their lines
from the playlist RomM generated into the zip — removing it where no line is left; a .m3u that is one of the ROM's own
files is kept as it is (#2732 D4).
Every size Tender shows or checks is the sum of the files fetched (download_size_bytes): the free-space check and
the progress of a start, the unpack's size check (the zip's own central directory, which holds only what was asked for),
the size on the game page before a download — read from RomM's ROM detail once the page opens, through the same
plan_fetch (DownloadService.get_download_size) — and the size an install records (RomInstall.fetched_size_bytes),
which the Data Management page sums. Where nothing is left out it is RomM's own size of the whole game
(fs_size_bytes); in the 8 KiB case the free-space check, the progress and the game page count the whole game, which is
what comes down. Rom.fs_size_bytes stays RomM's size of the whole game. The adopt dialog shows the download's side as
the size of the files fetched, while the ranking of found copies and the content check still compare with the whole game
(#2732 D5).
A game keeps the shape RomM gives it (#2223 D31; lands_as_folder, domain/rom_files.py). RomM exposes three
file-layout flags on every ROM detail:
| RomM flag | RomM server layout | What fs_name is |
Local layout |
|---|---|---|---|
has_simple_single_file |
roms/<platform>/<file> — one file, flat |
the filename | flat in platform folder: roms/<platform>/<file> |
has_nested_single_file |
roms/<platform>/<folder>/<file> — one file in a per-game folder |
the folder name | in a per-game folder named after the launch file: roms/<platform>/<launch-file-name>/<file> |
has_multiple_files |
per-game folder with multiple files (multi-disc, BIN+CUE, etc.) | the ZIP/folder name | extracted into per-game subfolder named after the launch file: roms/<platform>/<launch-file-name>/... |
A game lies flat only where RomM states it is one file (has_simple_single_file). Every other game lies in its folder,
also where the download fetches one file of it, and also where RomM sets none of the three flags — a folder with no file
directly in it, such as a Wii U dump (code/, content/, meta/) or a PS3 dump with no file at its root; a payload
that does not state all three falls back to more than one file being a folder. So the shape follows RomM rather than
what was downloaded. A folder game whose fetch leaves one file is served as that file, and the transfer places it at its
path inside the game's folder (DownloadUnpacker.finish_single's rel_path), where it takes the same steps as an
unpacked folder — the SFB heal, the M3U and launch-file rules, the ES-DE collapse rename and the PS3 folder name —
except that a .zip among them is unpacked or kept by the single-zip rule below
(#2732 D3). The search for a copy already on the device
follows the same shape: for a game RomM holds as a folder it offers only a folder, and a flat copy is a namesake it
cannot use (#2732 D2).
has_nested_single_file quirk: fs_name is the parent folder name, not the filename. The actual filename with
extension is the fetched file lying directly in that folder, else the first fetched file — RomM sorts files by name,
so a dump's inner asset can come first (resolve_local_file_name); Tender names the download's own file after it (the
content URL and the queue entry), and the folder after the ROM's identity. A defensive helper falls back to fs_name
and warns if files is empty or missing.
Extract-vs-file gate counts the files fetched, not has_multiple_files: Tender decides ZIP-extract vs a single
served file with the is_multi_file_download helper (domain/rom_files.py), which returns whether more than one of the
files game_files fetches remains, and reads has_multiple_files only for a payload that lists no files. This mirrors
RomM's own download gate, which zips whenever it serves more than one file, counting every file it serves, nested ones
included. RomM computes has_multiple_files from top-level files only, so the two counts disagree for a nested
layout: a Switch game with its base file at the root plus update/ and dlc/ has one top-level file yet more than one
file, and RomM zips it where all of them are asked for. Since its updates and DLC stay on the server, it is served as
its one base file. The flag cannot stand in where files are listed: an update or DLC lying beside the base game at the
folder's root sets it, while RomM files that file under update or dlc, so one file is fetched and served as itself.
A single zip holding a disc is unpacked like a multi-file ROM. Many emulators cannot mount a multi-track disc from
inside a zip — RetroArch hands a core only the member it claims, so a Saturn .cue inside a zip finds none of its
.bin tracks — and Dolphin reads no zip at all. So once a single file named .zip is downloaded, its own central
directory is read (no decompression) and the zip is unpacked when a member at its top level is a disc image, the top
level of a zip that holds nothing but one folder being that folder's, and macOS Finder's __MACOSX/ folder ignored
(holds_disc_image and wrapper_prefix, domain/served_shape.py; the first also says why an .m3u and a disc image
in a subfolder do not count, and directory-only entries count on neither RomM's listing nor the local one); a file that
does not open as a zip is kept as it is. Every single zip is unpacked without __MACOSX/ and without a directory-only
entry named with a \ (which zipfile reads as an empty file), and a wrapped one without its wrapper folder too
(extract_zip's single_zip and strip_prefix, each member placed by unpacked_name and the ZIP-slip check judging
that name), so a wrapped zip lands — and collapses for ES-DE — as an unwrapped zip's would. From there it takes the
multi-file path step for step — DownloadUnpacker.unpack_and_record is the one implementation both call: bounded by the
system's own ROM folder, unpacked into <roms>/<system>/<resolve_extract_dir_name>, PS3-healed, the M3U and launch-file
rules, the ES-DE collapse rename, recorded with file_path=<launch file> and rom_dir=<folder>, the zip removed. Only
the multi-file path URL-decodes the unpacked names: RomM writes them encoded into the zip it builds, while a user's own
zip holds the names its maker chose. Whatever sat at the zip's own path — a replaced file, or this ROM's earlier copy —
goes only once that install is recorded.
The question "what does this game become on disk" is asked of one function, served_shapes (domain/served_shape.py),
and every site that needs the answer asks it: the download's target gate claims both paths for a single zip — the zip
and the folder — the candidate search offers a namesake of either kind, adoption's target follows the kind of the
content being adopted, and the replace of a file at the zip's own path keeps it for the finalize. Where both paths are
occupied, Replace removes only what the user was shown: each occupied path is refused with a target_occupied dialog of
its own, one after the other, the re-sent start_download names every path answered so far (confirmed_paths, kept
with a paused download), and nothing is cleared until every occupied path is named. On a resume the folder is never
covered again and the zip only where the start confirmed it. That rule is replace_covers (domain/served_shape.py);
every other home of it names only its own part. Adopt from such a dialog names the path it showed, and "Download
Instead" over a namesake of a ROM that may unpack carries no saves, because the launch file they would be renamed after
is not known until the zip is unpacked. A folder adopted for a single zip is held to RomM's archive_members as loose
files, less a wrapper folder (unpacked_manifest, domain/served_shape.py); with none listed it cannot be confirmed.
Where RomM listed members that named no disc but the downloaded zip does hold one, the gate never asked about the
folder, so the zip is kept as it is if something already sits there.
Space for unpacking is checked twice. Before the download, a single zip whose listed members name a disc image at
the top level holds_disc_image reads is counted as itself plus their sizes (disk_space_verdict's unpacked_size); a
zip whose members RomM does not list, or lists without a disc image, is counted alone. After the download, before
anything is written, the uncompressed total the zip's own central directory states — for a single zip and a multi-file
download alike — is weighed against free space less what other downloads have claimed and not yet written, with the same
headroom (unpack_space_verdict, domain/disk_space.py; what another download has written is its transfer since it was
admitted plus what it has unpacked so far, written_since_admitted in the same module). Too little raises
NotEnoughSpaceToUnpack: the download fails with "Not enough space to unpack", nothing is recorded, and the downloaded
.tmp / .zip.tmp is kept. Before every transfer DownloadTransfer asks whether that file is already whole
(kept_download_complete, off the event loop) — it opens as a zip, and a single .tmp is exactly RomM's stated size of
the file it fetches — and a whole one skips the transfer and goes straight to the unpack, its pre-flight discounting the
kept bytes. Skipping is what spares a resume of a whole file the range past its end that a server answers with 416; the
size check keeps a whole zip of something else under this name from being unpacked as this game. The start-up sweep
(LeftoverTmpCleanupService) still removes every .tmp and .zip.tmp, so a kept file does not survive a backend
restart. A member whose CRC-32 does not match fails the unpack (zipfile raises when the member is read to its end);
the unpack removes the folder it created and the archive is removed with the other partial files.
ES-DE directory-collapse rename: a multi-file ROM is extracted into a staging folder named after the ROM's
identity — fs_name_no_ext, falling back to splitext(fs_name) (resolve_extract_dir_name in domain/rom_files.py)
— never after files[0]. That distinction matters for a has_nested_single_file folder game served as a ZIP (a PS3
title whose files RomM lists by name, an inner asset first): the local filename is a file's name, but the extract
directory must carry the game's identity or the whole install — launch file_path, rom_dir, and the folder a
folder-boot start is asked with — inherits the wrong name. The staging folder is then renamed after the detected
launch file including its extension (e.g. Example Quest - Second Journey (USA).m3u/ containing
Example Quest - Second Journey (USA).m3u). ES-DE only collapses a directory into a single game entry when the folder
name matches the launch file's full name with extension; without the rename a multi-disc game shows in ES-DE as a folder
plus loose disc files. The launch file is only known after extraction (an .m3u may be auto-generated — see below), so
the rename happens last, after launch-file detection, via es_de_collapse_rename (domain/rom_files.py) + the
DownloadFileStore.move_dir whole-directory move. On a name collision (target already exists) the rename is skipped and
the staging folder is kept — never clobbered or merged. Existing installs from before this feature keep their old folder
layout until re-downloaded.
Folder-boot launch target (PS3): for a PS3 folder game detect_launch_file picks the nested
…/PS3_GAME/USRDIR/EBOOT.BIN as the install's file_path — the correct launch file identity (save path, core, and
displayed filename all derive from it). But RPCS3's directory-boot wants the game folder, not the EBOOT, so the path
a start is asked with becomes the game folder (folder_boot_root, domain/rom_files.py, in the DiscLaunchResolver
seam), without rewriting file_path; the resolver's command for that folder is run as it answers it, like any other
(#2221 D7). file_path stays the EBOOT anchor
(ADR-0019, see
Core and Emulator Selection, which also says what the folder
itself is named). The multi-file download path also suppresses M3U generation and heals a .txt-suffixed
PS3_DISC.SFB for a folder-boot dump (domain folder_boot_layout_root + DownloadUnpacker.heal_ps3_sfb).
M3U generation rule (needs_m3u in domain/rom_files.py): a game-named <fs_name_no_ext>.m3u is auto-generated
(when no .m3u already exists) for multi-disc ROMs — two or more disc images of any format in
DISC_IMAGE_EXTENSIONS — so the emulator can switch discs, and for single-disc bin/cue ROMs — exactly one
.cue — so the extract dir is renamed after a game-named playlist rather than a generically-named cue (disc1.cue/).
Every other single disc gets no playlist: bin/cue systems read a one-entry playlist, iso-based GameCube/Wii do not
launch from one reliably, and whether a single .gdi, .ccd, .toc or .mds does is unmeasured. A disc made of
tracks launches from its descriptor: detect_launch_file takes a .cue before the WiiU, PS3 and 3DS rules, and .gdi,
.ccd, .toc, .mds, in that order, after them — those extensions also name files inside a game's own data, such as a
PS3 dump's levels.toc — and all of them ahead of the largest-file fallback, which would otherwise pick a track.
M3U is platform-gated on ES-DE's own extension list (ADR-0013).
The file-count rule above only runs when the ROM's system actually supports .m3u. RomM bundles a platform-blind .m3u
into the ZIP for every multi-file game, including cartridge systems (Switch .nsp, Xbox 360 .iso) whose emulators
have no playlist concept — so an extension-only heuristic wrongly produced a <Game>.m3u/ folder that never collapsed.
Tender now asks whether ES-DE lists .m3u as a supported extension for that system, read from the same es_systems.xml
ES-DE uses to decide directory-collapse, via AtlasCatalogueAdapter.system_supports_m3u(system) exposed through the
SystemM3uSupportFn Protocol (services/protocols/) and threaded into DownloadUnpacker from bootstrap. When the
answer is False, no .m3u is generated and the bundled one is never chosen as the launch file
(detect_launch_file skips its .m3u preference), so selection falls through to the real game file and the folder is
named <Game>.nsp/ / <Game>.iso/ instead. The capability crosses the service/domain seam as a plain bool — the
domain functions (needs_m3u, detect_launch_file) take m3u_supported, never a system name or an adapter. The
bundled .m3u is left inert on disk, never deleted. When es_systems.xml cannot be found the answer defaults to
False (safe: a missing playlist only degrades disc-switching, a wrong one breaks the launch).
Launch-target validation¶
detect_launch_file ends in "largest file by size", so a download none of its format rules recognise still yields a
file_path — a PS3 title distributed as .pkg + .rap
(#1582), where the game is still sealed inside the package,
or a disc rip whose only sizeable file is a raw track. Whether the system starts such a file is the resolver's answer
(launchable), and Tender reads it rather than judging it
(#2195 D2, D13).
Asked for one game, when it is read, and stored nowhere. The catalogue's reading module asks it
(AtlasCatalogueAdapter.launchability, read into domain/launchability.py::Launchability) with the label of the entry
Tender starts the game through, so the verdict is about Tender's emulator rather than ES-DE's; a game with no entry
chosen — no default and no pick — is asked nothing, since its start is refused for that already. The path asked about is
the one the shortcut starts (the disc pick folded over the install), except for a game kept as a folder, whose folder is
asked about as ES-DE scans it. Two endpoints ask it, each for one game: CoreService.get_platform_core_info carries the
chosen entry's verdict as launch — the game page, a core switch and Steam's own Play read it there — and
CoreService.get_emulator_launches answers every entry's for the emulator menu, which opens at once and shows each
answer as it lands (D22, D31). Nothing asks it for a list of games: an entry's first answer may probe its core, which
costs seconds (D31). Each answer carries the name the game page shows the game's platform under (platform_name, read
through domain/platform_names.py::platform_display_name as game detail reads it, by the game's fs_slug, the slug
where the cache names none or no fs_slug is recorded), and the sentences name the platform by it rather than by the
system the resolver was asked as (D36).
Nothing is written for an installed game. A shortcut's launch options are its appId whatever the verdict, and nothing asks for a list of games' verdicts: the start asks for the command of the one game it starts (Core and Emulator Selection).
Every code the resolver states on the answer is read (D40): where the entry does not take the file, the verdict
carries the resolver's reason in Tender's word (reason — a folder the entry takes none of, nothing to boot in it, no
disc image or game program, a disc that cannot be mounted, an encrypted image without its key), and where the file
starts but the resolver's reading of it stopped short — RPCS3's own boot read as far as it goes, a loader that takes the
extension and then reads content the resolver does not, an extension the emulator's builds disagree on — the verdict
still starts and reason names what was not established.
Where the resolver reads the files a disc file opens, each failure of that open carries the course the resolver
states for it (effect), and Tender reads it by that alone
(#2698 D5, D7): the failure it states as the stop is the
refusal's reason, a file the emulator goes on without is said as one it goes on without — never as not established — and
a failure whose course is not established is said as not established, as is anything else that reading could not follow.
Genesis Plus GX not reading every disc a playlist lists is said beside the start the same way.
Every statement is said (#2698 D8): beside a start, statements carries each one Tender words, in the resolver's
caveat order — the format not established (#2195 D41) and the archive whose contents were not read among them; beside a
refusal, whose stop is reason, it carries each file the emulator would have gone on without, what else it is
established to do, and each thing the resolver could not establish, in the same order. On a start reason holds the
first statement but the format and archive notes; what a start says is statements. Which code is which is
adapters/atlas_catalogue.py's (_REFUSED_BECAUSE, _OPEN_FAILED_BECAUSE, _STOPPED_SHORT_BECAUSE,
_STARTS_WITH_A_NOTE_BECAUSE, _STARTS_ALSO_SAYS), and so is which statement names a file. The resolver's boot facts
themselves (boots) are not read until the direct start does (D38), and the list of files a disc file opens (opens)
is not read at all. Every recorded launch machine of the resolver is run through this reading by
tests/adapters/test_launch_vectors.py (#2698 D6), which holds Tender's judgement to the resolver's verdict and stated
effect.
What each verdict does (D2): which verdicts refuse a start, and each one's words, are
frontend/src/utils/gameLaunch.ts's. A verdict that refuses disables Play with its own sentence, and Steam's own Play
on the shortcut is refused with the same words, asked fresh at its press (emulatorRefusalForRom); one that lets the
start go ahead leaves Play on, and its sentence, where it has one, stands under the play row. The launch gate holds no
step of its own for it: Tender's Play is never pressed where the page's read refuses the start.
While the page's check runs, and where it fails (D32, D33). The core read is the page's check of whether the game
can start, bounded at the local call limit. While it runs, Play is grey and disabled and reads "Checking..." beside the
throbber the other checking states show (romm-throbber, frontend/src/bigpicture/CustomPlayButton.tsx); once it
answers, Play follows the answer. Where the read fails or gets no answer in time, a notification says what went wrong,
nothing is said under Play, and Play is grey but can be pressed: a press asks again (recheckStart, the game page's
store). Its answer is the page's check succeeding and is shown as one — where something stands in the way, Play turns to
the cannot-start state with the sentence below it, a notification says it too, and nothing starts; where the start then
throws, a notification says that; a start that succeeds is left alone. The state is startCheck on the game page's
store (frontend/src/utils/gameDetailStore.ts), set back to checking whenever the page reads the game's core again: a
load, a version switch, an emulator or a source switch.
A PS3 folder installed before its name came in is renamed once at start-up (D27). What a PS3 folder is named, and
why, is Core and Emulator Selection's. Ps3FolderNamingService
renames each such folder, its install record following. Which installs it renames is fixed at its first pass and
recorded in kv_config as ps3_folders_to_name, so a folder adopted later is left as it stands and gets the resolver's
answer for its name. The step is stamped done (ps3_folders_named) only after a pass that left none of those it could
still rename: a folder on a drive that is not mounted, or whose move or record failed, is asked about again at the next
start, and a folder already moved whose record still names its old place gets its record moved after it. The step stays
in every later release.
A PS3 game folder one level inside its install's folder is lifted once at start-up (D39), by a second one-time step
of the same service, run right after the naming step with its own subject (ps3_folders_to_lift) and stamp
(ps3_folders_lifted) and the same rules. The game folder moves out to a sibling named <folder>.tender-lift, the
emptied folder is removed, and the game folder takes its place, its record following; it moves only where it is the
folder's only entry, so anything beside it keeps the install as it is. A pass stopped part way is finished at the next
start: a game folder waiting in its sibling is put in place, and one already in place whose record still names the
folder inside gets its record moved.
The files are always kept. A download whose file does not start is a real install: the row is written, the ROM is uninstallable through the normal path, and nothing is deleted — installing a package by hand in the emulator is the documented RetroDECK procedure, and the files are what it needs.
Filesystem writes go through DownloadFileAdapter. ZIP extraction is ZIP-slip protected and streamed: extract_zip
copies each member in chunks and reports byte progress through an optional callback, so an unpacked download — a
multi-file ROM, or a single zip that holds a disc image — emits download_progress frames with status: "extracting"
(bytes_downloaded/total_bytes over the uncompressed total, resumable: false) after the transfer hits 100%. The
frontend reuses the same event — no new event name — to switch the download button and QAM queue into the
non-cancellable Extracting… phase. A single file kept as it is never emits it. No download_progress frame follows
a download's terminal frame (download_complete, download_failed, or the cancelled frame): a progress tick or
resumability verdict the worker queued before the end can reach the loop after it, and DownloadQueue._live_entry drops
it once the entry has been evicted or its status is terminal.
Bounded concurrency + reserved-bytes pre-flight: at most two ROMs transfer at once, gated by an
asyncio.Semaphore(2) around the transfer + post-IO critical section. start_download enters the queue with status
queued and reserves the download's required bytes in the state box; DownloadTransfer.run flips the status to
downloading only once it acquires the semaphore (emitting a download_progress status: "queued" frame first if it
has to wait), and releases the reservation in its finally. The disk pre-flight subtracts what other downloads have
reserved and not yet written (DownloadsStateBox.bytes_reserved_by_others, the same figure the check before unpacking
uses; free space already lacks what they wrote) so two concurrent downloads that each fit alone but not together can't
both pass — the second is refused with insufficient_space.
Cancel reaches the UI and never destroys a live install: cancelling a download emits a terminal download_progress
status: "cancelled" frame so the frontend resets the button out of its downloading state (the cancel path used to be
silent). Because executor threads run to completion regardless of cancellation, a cancel that loses the race to a
just-committed install is reconciled (DownloadTransfer._reconcile_post_io awaits the in-flight post-IO future): if the
install committed, the download is surfaced as completed (download_complete emitted) rather than torn down.
DownloadUnpacker.remove_partial_download removes only the transient transfer artifacts (.zip.tmp / .tmp) and,
for a multi-file ROM that did not commit, the extract dir(s) this download created — it never deletes the bare
target_path, so a re-download that fails mid-stream (or a cancel that lost the race) cannot destroy a pre-existing or
just-committed install. An unpacked single zip's folder is removed by the unpack itself when it fails, because only the
unpack knows it created one. A download that failed with NotEnoughSpaceToUnpack removes nothing, so the kept file is
there for the retry.
Sibling supersede — one downloaded version per group
(ADR-0021
§4 amendment, #1298): before a download begins, start_download (and resume_download) strips any other installed
member of the ROM's sibling group so a game keeps at most one copy on disk. _conflicting_sibling_install_ids reads the
group in one short UoW (indexed iter_by_group_key), and supersede_siblings (domain/sibling_supersede.py) picks the
members that are installed and either unbound or bound to the same shortcut — a member bound to a different
shortcut (a second shortcut of the same game, which a game keeps only where the sync has not converged it yet or never
reaches it — Sibling groups, "Convergence") is exempt and never removed, because that shortcut still launches it. An
adoption runs the same supersede and passes the path it is about to record as adopted_path: a picked member whose
install is recorded at that path is handed over instead of removed — its rom_installs row is deleted in that same
UoW and its files are left as they are, because they are the content being adopted. A download passes no path, since
what sits at its target is either refused by the occupancy gate or, on an explicit replace, overwritten, so a member
recorded there is superseded like any other. Each superseded install goes through the canonical
RomRemovalService.remove_rom_unchecked (files + rom_installs row; saves untouched per ADR-0007), the membership
UoW closed first so the removal's own UoW does not nest (ADR-0006); a removal refused with NotInstalled raced clean
and is skipped, any other refusal propagates and aborts the download so the invariant stays honest. A superseded or
handed-over sibling's paused queue entry is evicted (DownloadQueue.evict_if_paused) so a stale .tmp can't later
resume into a second install. resume_download additionally refuses (superseded, entry dropped) when a switch has
moved the group's binding to a different member since the pause — it won't re-download a version the picker has already
left. The in-progress claim is taken before the ROM detail is fetched and given back on every way out but a started
download, a cancelled start included, so a concurrent start_download for the same rom is rejected rather than racing
past it and a failed start never leaves the ROM "Already downloading". Of what the start's steps raise, an OSError (a
disk that cannot be read or prepared) is refused there with download_start_failed; a RomM error and a refusal pass on
to Endpoints unchanged — such as the platform's system refusal (no_platform_system, platform_system_off, or the
RomM classifier's reason where the platform listing could not be read) and RetroDECK's folder refusal — and anything
else is a bug.
ConnectionService notes¶
The minimum-version gate is SemVer-aware on SYSTEM.VERSION. domain.version.meets_min_version compares the
numeric core against MIN_ROMM_VERSION (domain/identity.py); when the core equals the floor, a -alpha / -beta
suffix (case-insensitive, optional .N build number) ranks below the release and is rejected — so 5.3.0-beta.1
fails at floor 5.3.0 while 5.3.1-beta passes. development and a missing version bypass the gate. The floor was
raised to 5.3 by ADR-0040, and to 5.4.1 by
#2223 D29, when a download began to read the category RomM
files each of a game's files under.
A Client API Token is bound to the server it was minted against. When the token is minted, the canonical origin of
romm_url (full scheme://host[:port], default ports folded out, path/query dropped — lib/url_host.normalize_origin)
is stored alongside it as romm_api_token_origin. RommHttpAdapter.auth_header() attaches the bearer only when
that stored origin matches the current romm_url origin; on a mismatch it raises TokenHostMismatchError instead of
sending the credential to a host the token was not minted for. The error is non-retryable and maps to a config_error
failure (Your saved RomM login is for a different server. Sign in again to continue.), so every data flow fails fast
until the user re-signs-in. https://h and http://h are deliberately different origins — a plaintext downgrade is
a different destination, not the same one. A legacy token minted before origin stamping carries
romm_api_token_origin =
None and is treated as un-bound: it is still attached (never blocked) so existing installs
keep working until their next sign-in stamps the origin.
An address carrying a user name or password is refused where it is entered, and, saved, at each sign-in and at the
connection check. No request to such an address reaches RomM; why is the comment above the two sentences in
lib/url_host.py. carries_userinfo beside them decides it, and each refusal is a config_error:
ENTERED_ADDRESS_WITH_USERINFO_MESSAGE when the URL is saved (SettingsService.save_server_url), and
STORED_ADDRESS_WITH_USERINFO_MESSAGE at each of the three sign-ins — the panel signs in with the URL it holds saved —
and at test_connection, before any request. The panel's Connection row names it Remove login from RomM URL. A stored
one is not refused anywhere else: a sync, a download or any other request to it is still attempted and fails, with
errors that carry no login (below, "what an error's text carries").
A stored token no HTTP header can carry is refused before it is used. auth_header() raises TokenUnsendableError
for a token holding a CR, an LF or a character past latin-1 (domain/header_value.py, the same rule the SteamGridDB key
is checked by), after the origin check. Like a host mismatch it is non-retryable and answers config_error
(Your saved RomM login cannot be used. Sign in again to continue.), so it never marks RomM unreachable.
Sign-in ordering: validate → probe → mint → persist (one atomic save). establish_token trims the entered URL and
rejects a non-http(s) value before any network call. It then holds the candidate URL in memory only — clearing the
stored token in memory first so the version probe never carries the old server's bearer to the candidate host — and
persists nothing until the mint succeeds. On any failure (probe unreachable, version too old, forbidden/error mint, no
usable token, or a disk error) the in-memory auth state is rolled back to the previous working URL + token, and because
disk was never touched the prior working credentials survive a failed sign-in. Only a successful mint commits
romm_url + SSL flag + token + id + origin to disk in a single save_settings() call. Every step of a sign-in up to
that save gives the previous state back when it fails, a bug or an interrupted call included, and nothing after a
successful save does, which would leave memory and file disagreeing. A settings file that cannot be written refuses the
sign-in with save_failed and the write's cause; any other exception from the write stays a transport error.
The old-token DELETE is origin-guarded and provenance-guarded. RomM scopes a Client API Token to the account, and
re-auth deletes the device's previous token. That DELETE is only fired when the old token's stored origin matches the
new URL's origin (same-server re-auth) — replaying it against a different server would delete an unrelated token there,
so the DELETE is skipped (and logged) when the origins differ or the old origin is unknown. It is also skipped
whenever the previous token's romm_api_token_source is "user": a token the user pasted belongs to the user, not to
this device, and must never be revoked by Tender (a user token also carries no romm_api_token_id, so there is nothing
to DELETE by anyway — the source check states the intent). The DELETE uses Basic auth from the one-time credentials,
unaffected by the cleared bearer.
A token's provenance is recorded in romm_api_token_source. The value is "minted" for a token Tender minted from
a username/password (establish_token) and "user" for a token the user pasted (establish_user_token). It is part of
the snapshot/restore auth-state set, so a failed sign-in rolls it back with the rest of the auth state.
Pasted-token sign-in (establish_user_token) — the OIDC path. OIDC / SSO accounts have no password to mint from, so
the user creates a Client API Token in RomM's web UI and pastes it. establish_user_token mirrors establish_token's
validate → probe → gate → validate → persist-on-success-only shape, but the credential is the pasted token rather than a
fresh mint, so there is no mint and no server-side DELETE of any prior token. The entered URL is trimmed and
rejected if non-http(s); a blank/whitespace token is rejected as config_error before any network call. The candidate
URL and the pasted token are held in memory only, with romm_api_token_origin stamped to the candidate origin and
romm_api_token_source = "user", so the auth-header guard attaches this token (not the old server's bearer) to the
validation probe. Validation is an authenticated GET /api/users/me: a 401 means the token is invalid or revoked, a 403
means it authenticates but lacks a required scope (Tender cannot introspect a token's granted scopes, since
/api/users/me's oauth_scopes reflects the user's role, not the token's grants, so this connect-time authenticated
probe is the only validation). Both map to the canonical auth_failed failure with a token-specific message. On any
failure the in-memory auth state is rolled back and disk is never touched; only a successful validation commits URL +
SSL flag + token + id = None + origin + source in a single save_settings(). The token value is never logged. The
device-forget-on-origin-change and playtime-scope-notice clear run on the success path exactly as they do for
establish_token. The /api/users/me validation + persist tail is shared with the paired-token path below via the
private _validate_and_persist_user_token helper (both hand Tender a token to validate and store with "user"
provenance), so their observable behaviour stays identical.
Pairing-code sign-in (establish_paired_token) — the OIDC path without pasting. The same OIDC accounts can sign in
by entering a short-lived RomM pairing code instead of pasting the token: Tender exchanges the code for the token over
the public, unauthenticated POST /api/client-tokens/exchange endpoint (the one-time code is itself the
credential). It mirrors establish_user_token's validate URL → probe version → gate → obtain-credential → validate via
/api/users/me → persist-on-success-only shape, but the credential is fetched by the exchange rather than pasted, and
the exchange runs with the token trio cleared in memory (like establish_token) so no old bearer leaks to the
candidate host during the unauthenticated call. The code is normalized the way RomM normalizes it — all whitespace and
- stripped, then uppercased ("ab-cd ef23" → "ABCDEF23") — and a code that is blank after normalization is a
config_error before any network call. The exchange is never auto-retried: a pairing code is single-use, and a
replay would burn both the code and RomM's per-client rate limit, so the transport's unauthenticated_post_json skips
with_retry. Each RomM rejection maps to a distinct, actionable message: an invalid/expired/used code, a
token-no-longer-exists 404, and a disabled-owner 403 all carry auth_failed; a 429 carries the bespoke rate_limited
reason. On success the freshly rotated raw_token (the exchange regenerates the token server-side, so any previously
copied raw value stops working) is host-bound in memory and run through the shared _validate_and_persist_user_token
tail, so it is persisted with "user" provenance and id = None exactly like a pasted token — no mint, no server-side
DELETE. The pairing code and the returned token are never logged.
The registered device id is forgotten only on a genuine origin change. A device registered with RomM
(POST /api/devices, its id stored in kv_config["device_id"]) is bound to the server it was registered against —
RomM's negotiate save-sync transport hard-404s a foreign device id. So a successful sign-in that genuinely changes
origin forgets the stored device id (SaveService.forget_device → DeviceRegistry.forget_device, wired as the
DeviceForgetFn injected into ConnectionService), and the sign-in then registers this device against the new server
(below). The decision is is_origin_change(old_token_origin, new_url) in lib/url_host — both origins are normalized
and the id is forgotten only when both are known (parseable) and differ. This is deliberately the opposite failure
posture to the old-token DELETE guard (same_origin, which fails closed): a same-server re-sign-in keeps the device
identity, including a token swap on the unchanged URL, URL-formatting variants, and a None/unstamped old origin (an
unknown old origin is treated as unknown, not different — never a change). Without this, a same-server token swap
dropped the identity and the next post-exit sync flagged a spurious conflict for a save this device itself synced
(#1437). The old_token_origin comparison input is captured from the pre-clear auth-state snapshot, so the leak-safety
in-memory token clear (which zeroes the origin) never poisons the comparison. The forget is local-only (the row on
the old server is left for RomM's machine-id dedup to reconcile on a later re-registration) and best-effort on the
success path — a failed local clear never turns a good sign-in into a failure, and a failed sign-in (in-memory
snapshot restored) keeps the still-current server's id. ensure_device_registered never deletes the id on any failure,
so a permission-degraded (403/timeout) session leaves the identity intact. This is Phase 0a of the RomM Device Sync
negotiate adoption (ADR-0016).
Every successful sign-in registers this device, whether save sync is on or off. After the origin forget above, each
of the three sign-ins calls SaveService.ensure_device_registered, wired as the DeviceRegisterFn injected into
ConnectionService, so the id the play-session upload and the save operations send exists from the sign-in on. It is
best-effort: a registration that fails is logged and the sign-in still succeeds, since every save operation registers
again before its first request to RomM.
The signed-in user's own id (romm_user_id) is bound to the token — stamped at sign-in, backfilled lazily, cleared on
sign-out. It drives the collection owner scope (build_work_queue + get_collections, described in the
LibraryService section above). Every sign-in path re-derives it from the freshly authenticated token so it can never
linger for a different user or a different server: the mint path (establish_token) probes GET /api/users/me after
host-binding the minted token (the mint response carries only the token id, not the user id); the pasted/paired paths
reuse the id from the /api/users/me validation probe they already run (no second call). In every case the id is set
in memory before the token-persist save, so it rides the same single atomic save_settings() — the sign-in write
shape is unchanged. It is part of the snapshot/restore auth-state set, so a failed sign-in restores the previous id, and
it is cleared in each path's in-memory auth clear so a probe failure or a malformed payload leaves it None rather than
stale — so the own scope drops nothing until the next backfill. Existing installs (a valid token minted before this
setting existed) carry no id; test_connection lazily backfills it — when a token is present and the id is missing,
the successful connection check probes /api/users/me and persists the id (its own save), so the scope takes effect
without a re-login. A known id needs no network on later checks. Every identity read is best-effort: a failure never
fails the sign-in or the connection check.
The no-sign-in URL change path (SettingsService.save_server_url) deliberately does not touch the token, so pointing
the URL at a different origin leaves the stored token's origin mismatched and the auth-header guard makes subsequent
data flows fail fast with config_error until the user signs in again. Because every data flow (including device
registration) is inert until that sign-in, a stale device id from this path cannot be used before the sign-in clears it.
Sign-out (sign_out) is a local forget — never a server-side delete. sign_out clears the token quad
(romm_api_token / romm_api_token_id / romm_api_token_origin / romm_api_token_source) in the in-memory settings
dict and persists them in a single save_settings() (the same atomic-write funnel _persist_token uses), keeping
romm_url and the SSL flag so the user need not re-enter them. It mirrors the sign-in paths' persist discipline: the
auth state is snapshotted first, and a failed save rolls the in-memory quad back to the snapshot and refuses the
sign-out with save_failed and the write's cause, so a disk error never strands the user with a half-forgotten but
still-valid token. Only on a successful save does it drop the cached RomM server version (set_version(None)) so no
stale value lingers. It is synchronous — no run_in_executor, no network — and idempotent (signing out when already
signed out still returns success). It never issues the server-side token DELETE: a minted token deliberately lacks
me.write (deleting it would require re-entering the password, which sign-out does not have), and a user-supplied token
belongs to the user. That is why the direct re-authentication path stays as Sign in again (not sign-out-then-in):
establish_token's same-origin minted-token cleanup (#1038) only fires while the old token id is still stored, so
signing out first would strand the old minted token on RomM (which caps tokens per user). After sign-out,
test_connection returns the canonical "Not signed in" config_error because romm_url survives but the token is
gone. Signing out (like re-signing-in) while a sync or download is in flight needs no guard: once the token is gone the
in-flight operation simply fails authentication and surfaces its normal error — deliberate, not a race to defend.
Server-supplied paths are validated before any write: the firmware file_name and the post-extraction URL-decoded
ZIP member names are checked through lib/path_safety (safe_join for realpath containment, safe_path_component for
single-component names) and fail stop on a traversal (../, an absolute path, or a %2e%2e%2f-encoded ZIP member that
decodes to ../ after the pre-decode ZIP-slip check passes). A ROM's local file name is not refused but coerced to one
safe component (coerce_safe_component), falling back to the ROM's synthetic name; the ROM's folder is the resolver's
answer, so no platform slug is joined onto a root. A ZIP member that decodes to a traversal aborts the whole
extraction rather than skipping the offending entry: already-extracted members are cleaned up (no half-installed ROM)
and the download_failed event fires with the error's message, so the UI doesn't hang on "downloading". A firmware
download refuses with path_traversal, raised by the single download (FirmwareDownloader._download_one) and answered
by Endpoints; a batch download names that file among its failures and carries on.
StartupHealingService notes¶
Beyond the missing-download report and orphaned-SyncRun reconciliation, this service owns the launch-options
reconcile (#1043). Every bound shortcut's launch options are its appId
(Core and Emulator Selection): the
sync sets it on every shortcut it creates or rewrites, without confirming it at creation, and a shortcut an older
version wrote a command into, or one edited in Steam, carries something else until something sets it back.
get_bound_launch_options() is the read half: a 0-arg read whose items are one {app_id, launch_options} for every
bound ROM (its roms.shortcut_app_id is set) whose launch options in shortcuts.vdf are not its appId — or for
every bound ROM where the file cannot be read. It reads the bound appIds in one short read UoW and reads the file after
that UoW closes, since BEGIN IMMEDIATE takes the write lock even for a read and a file read inside the UoW would stall
every other writer for its duration (#1779). The endpoint is read-only and checks the prune rule but not the
migration rule; a non-empty answer carries an installed_reconcile lease for the frontend's writes.
The frontend pulls this on mount, once the backend is proven reachable (it reuses the app-id/metadata init's
retry/backoff, not a second loop), and again after every sync, and confirm-sets each entry via the existing
setLaunchOptionsConfirmed fire-then-poll. The pass is idempotent and appId-safe: a launch_options write does not
change the shortcut's appId, so artwork, collections, and the shortcut identity survive. A confirm Steam's report and
re-read do not settle falls back on what shortcuts.vdf holds for the shortcut, read once per batch of confirms through
get_shortcut_launch_options (ShortcutFileService); see setLaunchOptionsConfirmed in
Steam shortcuts. A confirm that still fails logs a warning
naming what Steam's re-read of the shortcut saw — no details within its bound, or another value — and what the file
showed, with the value written beside any value either showed, all in full, ahead of the reconcile's own failure line.
UpdateCheckService notes¶
The service answers one question — is a newer Tender release out — and stays silent whenever it cannot say. It asks
through LatestReleaseFn, whose adapter reads GitHub's "latest release" route: drafts and pre-releases are excluded by
that route, a tag other than tender-v<version> is not a Tender release, and the release counts as available only
once its romm-tender-<V>.tar.gz asset is attached, GitHub states a valid sha256 digest for it (64 hex digits after
sha256:), AND the romm-tender-<V>.tar.gz.sha256 asset the installer verifies against is attached beside it: a
tarball that cannot be verified before an install makes the release not available, and a stored answer without a valid
digest or without the checksum file's address is dropped when it is read back. The assets job uploads both files minutes
after the release is published, so for that window the release is read, answered and passed over: the last available
release the checks found stays standing, and a check that reached nothing at all does the same.
- At most once a day — asked when the panel loads, and while the backend runs. The last answer lives in
kv_configunderupdate_check_last_seen— its version, the tarball'sbrowser_download_url, the lowercase sha256 hex of itsdigestand the checksum file'sbrowser_download_url(checksum_url), which name one release and stay together, plus the time of the last attempt. The stamp records the ATTEMPT, so an offline start does not pay the ten-second timeout at every panel load; a stamp dated in the future is due at once. - The user's two keys are in
settings.jsonand are written only through theSettingsPersister:update_check_enabled(absent means on) andupdate_notice_dismissed_version, which holds a version rather than a flag so the next release raises the card again. The switch governs only what the program asks by itself: with it off neither the panel-load read nor the running check requests anything, andget_update_noticereports the stored answer however old it is — card, version and all. What a check found is offered to install whatever the switch says. - Check now (
check_for_update_now) skips the throttle and forgets the dismissal whatever the switch says — a press is the user asking — and addsreachedto the answer, so the Settings section can tell "nothing newer" from "nothing found out". - The "is available" toast is owed once per version (
toast_owedon the notice, the pushed one included): while the release isavailable, the switch is on, the version is not the one stored as told underupdate_available_toasted_versioninkv_config— observed state, kept across restarts because the toast is owed once per version rather than once per process — and the release was not seen (below). Two things record a version as told: the panel's acknowledgement once it raised the toast (acknowledge_update_available_toast), which is refused withversion_changedfor any version but the stored release so a late acknowledgement cannot take a newer release's toast with it, and a Check now whose read answered with a newer release, which the user has just read in Settings. A told version that cannot be read is a warning and counts as none, so the toast is owed once more; one Check now cannot record is a warning too, and Check now still answers, its own answer owing no toast. What else holds the toast back is the panel's (QAM panel, Notices and homes). - A release is seen once per version (
seenon the notice, the pushed one included): the version is the one stored underupdate_available_seen_versioninkv_config, observed state like the told version and kept across restarts for the same reason. The panel records it (mark_update_available_seen) once Settings › Updates has shown the release for about a second, and it is refused withversion_changedfor any version but the stored release, as the toast's acknowledgement is. Seen takes the panel's dots and owes no toast; it leavesavailable, and so the card, alone. A seen version that cannot be read is a warning and counts as none, so the dots show once more. - While the backend runs, it checks by itself (
run_due_checks, started with the background tasks and cancelled inApplication.shutdown). It wakes once an hour on the injectedSleeperand asks for the notice exactly as the panel load does, so the stamp still decides whether GitHub is asked and a check that reached nothing stays silent. A notice different from the one it saw last is emitted asupdate_notice, carrying theget_update_noticeanswer, and the panel's notice store takes it, so the card and the Settings section appear without a reload. With the switch off it asks nothing and emits nothing. - The stored release has one reader.
last_seen_releaseanswers what the last check stored, whatever the switch says, and asks GitHub nothing; the install reads it there (UpdateInstallService notes) rather than from thekv_configrow. - What the request carries is the program's User-Agent (
romm-tender/<version>, fromdomain/identity.py) and GitHub's JSONAcceptheader; GitHub sees the machine's IP address like any other request, and nothing else about the user.
Two answers come from the environment rather than from the service, and the entry point resolves both once
(domain/update_release.py::resolve_update_source) next to the directories. TENDER_RELEASE_API is the installer's own
test seam, read under the same name with the same default, so a unit drop-in points both at one fake server.
installed_program is True only where TENDER_CODE_DIR names the directory the running code actually sits in: only
the installed unit sets that variable, and requiring it to name this process's own directory keeps a checkout from
claiming to be the install because a shell exported the variable for an installer test. A run from a checkout checks and
shows the card like any other, and the Settings section shows a line naming this a development build.
UpdateOutcomeService notes¶
The service tells the user what the last update did, in one of two ways, and never touches the installer's files.
- A version that moved is announced, with the way it moved — one toast, and a card until dismissed. At every start
(
note_start, a start-up step ahead of the others) the runningVERSIONis compared withlast_run_versioninkv_config, which is then set to the running version. A stored version the running one is later than is an update — logged at INFO asupdated from <old> to <new>,announce_direction"updated"— and one it is earlier than is a return to that release —back on <new> after <old>,"back"; which is later isdomain/version.py::is_newer_version's answer. The announcement is owed to the panel in this process's memory only, as two things held apart: a toast and a card. The panel reads it at load (get_update_outcome'sannounce_version,announce_directionandtoast_owed), raises the toast whiletoast_owedis true and acknowledges it (acknowledge_update_toast), so a panel a Steam restart reloads does not raise it again — the acknowledgement leaves the announcement itself standing, and the card with it. The card goes when the user dismisses it (dismiss_update_announcement), which drops the announcement for the rest of the process, sotoast_owedis false from then on too. The next start compares equal and owes neither, so a backend restart on the same version takes a card nobody dismissed down as well. The first start that records a version announces nothing, and so does a stored version that differs while neither is the later — one of them unreadable, or two pre-releases of one release, whichis_newer_versiondoes not rank: the version is still recorded, and nothing is logged or announced rather than a direction nobody established. - A rollback is never announced, not even as
back. The installer restores the databaselast_run_versionlives in, so a rolled-back start normally finds its own version stored already. The record below is the second witness:domain/update_outcome.py::announced_updatealso announces nothing where the record says this start is back on the running version after trying the stored one — the rolled-back notice tells that start's story. - A rolled-back update is the installer's record, read, never written.
update-failure.jsonin the state root (adapters/update_failure.py) is read at start — one WARNING,the update to <X> was rolled back at <T>; back on <Y> — what <X> logged when it tried to start is earlier in this log, or in journalctl --user -u romm-tender if it failed before logging— and again on everyget_update_outcome, so the card goes as soon as the next update that answers removes the file. A record that is missing, unreadable or short of any of its three keys is no record. - The record has kinds, and a record without
kindis a rollback. The pre-install check refuses a new version that cannot be built, or that crashes it, before anything is stopped or replaced (Running an installed one), and an update records that refusal in the same file with"kind": "check"—restored_versionthe version still running,rolled_back_atwhen. The keys keep their names so that a backend from before the check still reads such a record, and tells it as a rollback. This one logs it as its own WARNING,the pre-install check refused <X> at <T>: it does not start, so nothing was changed and Tender is still on <Y> — what the check said is in journalctl --user -u romm-tender-update, or in the terminal the installer ran in, and handskindto the panel infailure, which words the line under the card by it. Akindit does not know — a later installer's — is still an update that did not go through: it reads as"unknown"(domain/update_outcome.py::decode_update_failure), logsthe update to <X> did not go through at <T>; Tender is still on <Y> — what the installer said is in journalctl --user -u romm-tender-update, or in the terminal the installer ran in, and the panel names no cause for it, only where the installer's output is. - A record stands only while the running version is the one it restored
(
domain/update_outcome.py::standing_update_failure). One that names anotherrestored_version— the installer's removal did not happen, and a later update went through anyway — is a leftover: no WARNING,failureisNone, and so there is no card and no failure block under Settings › Updates. The file is left where it is, since removing it is the installer's. - Dismiss is per record.
update_failure_dismissed_atinsettings.jsonholds the dismissed record'srolled_back_at, written only through theSettingsPersister, so the next record the installer writes raises the card again. - A record's toast is owed once, for good. A record stands across every start until a later update goes through, so
unlike the announcement its toast cannot be owed per process:
get_update_outcome'sfailure_toast_owedis true while the standing record'srolled_back_atis not the oneupdate_failure_toasted_atinkv_configholds, and the panel writes that stamp once it raised the toast (acknowledge_update_failure_toast). It is observed program state, not user intent, which is why it is inkv_configbesidelast_run_versionwhile the dismissal is insettings.json. A dismissed record owes no toast, and the next record owes its own. - Which card stands where both apply is the panel's rule (
failureTakesThePlaceOf, QAM panel).
UpdateInstallService notes¶
The service installs the release the check stored, from the panel, on the installed program only. It never replaces
anything itself: it hands a verified tarball to the same install.sh a user runs by hand, which stops this process,
swaps the tree and rolls back what does not answer
(Running an installed one).
- What is offered.
get_update_install_stateoffers an install where this process is the installed program and the stored release (LastSeenReleaseReader.last_seen_release) is strictly newer than the running version, whatever the check's switch says. A run from a checkout is offered nothing. - What a press waits for is listed as
wait_reasons, each a discriminant the panel words: an app Steam lists whose display status holds the wait (app_running, with the names; those whose status could not be read, which hold it too, apart inapps_status_unread— the backend's reading of the list, A panel an earlier backend left behind, step 1), a running-apps reading that could not be taken (running_apps_unknown— never read as "nothing running"; withTENDER_INJECT=offthe injector never attaches, so no reading is ever taken), a library sync, ROM downloads in flight or queued, a save sync (the device gate, pre-launch and post-exit included, and every save operation outside it — a conflict resolved, a slot switched or deleted, a save copied or rolled back, local saves deleted), firmware downloads, a save directory being followed, a removed-game cleanup, a RetroDECK migration that is moving files, any other claim of work held on the prune conflicts (other_work— an uninstall, a shortcut removal, a version switch, an adoption, a lease the frontend holds for its Steam writes), Steam's interface reload limit (interface_reload_limit, withfrees_at, when the oldest recorded takedown leaves the window), and a reading of that limit that could not be taken (interface_reload_limit_unknown— never read as a limit that lets one more through, since the restarted backend replaces the panel only where the limit lets it). A migration that is only pending does not wait: the stored question survives the restart and is asked again. Paused ROM downloads do not wait either — they are counted aspaused_downloads, because the queue lives in memory and a start removes their partial files, so the restart cancels them. The running apps and the reload limit are the host's (SteamInterfaceReader, filled in byhost/runtime.pyonce the injector exists), and the claims are the prune conflicts' (HeldClaimsFn, which sweeps expired leases first); every other reader is the owning service's own. A claim counts toward the reason that names its work —domain/update_install.pymaps each endpoint and lease key to one, the save use cases to the save sync among them and the rest of the work toother_work— except a claim whose own work only reads, or writes nothing a restart can cut short (READ_ONLY_CLAIMSthere, whose comment is the one list of the writes they may make), which makes no press wait; a lease such a read hands out is a claim of its own. A test reads every claim name the source takes and fails on one in neither list, and a claim neither names still counts asother_work, so work nobody classified makes a press wait rather than slipping through. - The press (
install_update, naming the version it means) is refused while an attempt holds the rule (update_in_progress), where nothing is offered (not_offered), for a version that is not the stored one (version_changed), and while any reason holds (update_waiting, carryingwait_reasons). The reasons are asked again at the press with one fresh reading of Steam's running apps and of the reload limit, taken first, and the rule is taken in the same loop turn as the check that passed, so no work of this process starts between the two. Otherwise the answer is{"success": True}and the attempt runs on by itself. - The attempt:
<cache root>/update/is started afresh; the tarball is downloaded toromm-tender-<V>.tar.gzthere (the name its checksum file names) with byte progress; its sha256 is compared with thedigestGitHub stated; the.sha256file is downloaded beside it for the installer's own check; onlyromm-tender/install.shis copied out of it; Steam's running apps are read once more, and a game started during the download — or a reading that could not be taken — fails the attempt there asgame_started(orrunning_apps_unknown), before anything changed; a line in the log names the unit to follow (journalctl --user -u romm-tender-update); and/bin/bash install.sh --from <tarball> --yesis started withsystemd-run --user --unit romm-tender-update --collect. The unit gets the sixTENDER_*directories as this process resolved them,TENDER_PYTHONset to this process's interpreter (the installer writes it into the service'sExecStartand would fall back to/usr/bin/python3), andXDG_CONFIG_HOME,TENDER_RELEASE_API,TENDER_DOWNLOAD_BASEandTENDER_UPDATE_WAITwhere this process has them. A transient unit starts from the user manager's environment, so nothing else of this process reaches it. Why the installer is unpacked into the cache root rather than beside the program isadapters/update_staging.py's. - Progress goes out as
update_install_progress, carrying the attempt —version,step(downloading,verifying,installer_started,failed),bytes_done,bytes_total,failure— once per step and at most every half second while bytes arrive, the last tick always. Afterinstaller_startedthis process sees nothing more succeed: the installer stops it. - Starting the installer. A
systemd-runthat refuses — a name still loaded among the reasons — fails the attempt asinstaller_not_started. One that gives no answer within thirty seconds is not taken for a refusal, because the unit may have started behind it: the user manager is asked, and only a unit it still says is not running one watch interval later fails the attempt — a unit not created yet reads as ended too; otherwise the attempt goes on as started and the watch finds out. - Watching the installer. Every three seconds the unit is asked whether it still runs. It ending while this process
still runs means the installer refused or failed before it stopped the service. Where the installer's record says its
pre-install check refused this attempt —
kindcheck, naming this version, leaving the running one in place, written no earlier than the attempt record'sstarted_at(domain/update_install.py::refused_by_the_check) — that record is pushed to the panel asupdate_failure_recorded, in the shapeget_update_outcomeanswers it in, so the notice on Main shows it at once, and the attempt fails asnew_version_does_not_start. Otherwise — a record that cannot be read included — it fails asinstaller_stopped. What the installer said is in its journal either way. A user manager that cannot be asked is not taken for the unit ending: the first such reading is a WARNING, later ones go to the debug log, and the watch goes on with the update rule held, for as long as this process lives — it is never given back on a guess. The panel says what that looks like after seven minutes (QAM panel). - Something unforeseen — an exception the steps above do not name — is logged with its trace. Before the installer
was asked to start it fails the attempt, as
download_failedwhile downloading and asinstaller_not_startedafter; from that moment on it leaves the rule held, since the installer may be running. A running-apps reader that raises is not unforeseen: it is a reading that could not be taken,running_apps_unknown, at the press and before the installer alike; nor is a reload-limit reader that raises, which isinterface_reload_limit_unknownat the press. - Shutting down cancels the attempt and a judgement of the attempt record still waiting for the installer's unit, and a download still running on its thread ends at its next block of bytes rather than holding the process's exit up until it finishes.
- A failed attempt —
download_failed,checksum_mismatch,installer_not_started(no installer in the tarball, or a unit that would not start, a name still loaded among them),installer_stopped,game_started,running_apps_unknown,new_version_does_not_start— removes what it staged, gives the update rule back and is reported; nothing is retried by itself.try_againis then true for that version, and it is also true where the installer's record of an update that did not go through (read throughUpdateFailureFn, never written) stands and names the offered version. - The update rule. From the press that starts an attempt until it fails,
is_update_in_progressanswers true, andConflictRuleSetrefuses withblocked_by_updateevery use case that names the migration rule, and the migration itself (migrate_retrodeck_files) — ahead of every other rule — as do the save engine's pre-launch and post-exit backstops, its save-directory follow, and the post-exit sync inSessionLifecycleService, which ask the migration outside the rules.holdasks the update rule once more after it registers a use case's operation, since a press that came while it waited for the lock saw no operation; and the save engine's manual syncs (sync_rom_saves,sync_all_saves) ask it again once they hold the device gate, as defense in depth. A successful install ends it with the process. The rule and where it holds: invariants. - Leftovers. The update directory is removed at every start (
remove_update_leftovers); the installer never removes a tarball it was given. - The attempt record. An installer that stops without updating after it stopped this process can tell nobody: the
next start runs the same version and looks like any other. So right before the installer starts, the service writes
update-attempt.jsonin the state root (UpdateAttemptStore,adapters/update_attempt.py) — the attempted version, the running one, and when — the backend being its only writer and the installer never touching it, unlikeupdate-failure.json. A failure this process reports itself removes it (the panel already showed it), and so does the next attempt. At start (note_update_attempt),domain/update_install.py::stopped_attemptjudges it: running the attempted version, the update went through; a standing record of the installer's that it rolled this attempt back, or that its pre-install check refused it — one naming this attempt's version, written no earlier than the attempt started — is that record's story; running neither version, the version moved since — each removes it. Running the version that started it, with no such record from the installer, is an installer that stopped without updating: a WARNING, the attempt reported asinstaller_stoppedfor Try again, andget_stopped_update_attemptanswering it for the notice on Main untildismiss_stopped_update_attemptremoves the record or a new attempt starts. The dismissal does nothing where no such attempt stands — a card left on screen from before a new press must not take the new attempt's record away. The installer starts this program itself — the new version, the one it rolled back to, or the same one again after it gave up — so the start that looks at the record may run inside the installer's unit.note_update_attempttherefore only starts a task, which reads the record and asks the unit off the loop, so a user manager slow to answer holds up no other start step; only once that unit reads ended is the record judged. While it runs, or the user manager cannot say, the unit is asked again every three seconds; a stopped attempt found is pushed to the panel asupdate_attempt_stopped(the panel reads the answer once, at load, which may come before the judgement or after it); and a press in the meantime ends the question. A record that cannot be written is a WARNING and does not hold the install up — should that installer then stop without updating, the next start cannot say so; one that cannot be removed is judged again at the next start. - A stopped attempt's toast is owed once, for good. The same attempt is judged again at every start until it is
dismissed or superseded, so the answer and the push carry
toast_owed: true while itsstarted_atis not the oneupdate_stopped_toasted_atinkv_configholds, read once the judgement is made, and the panel writes that stamp once it raised the toast (acknowledge_stopped_update_attempt_toast). - A failed attempt's toast is owed until acknowledged, in memory. A toast raised off the failed progress frame alone
would be lost with a frame no panel received, so every failure of an attempt of this process owes one —
new_version_does_not_startapart, whose record's toast tells it. Each accepted press is numbered, andget_update_attempt_toastanswers{attempt, version, failure}for the latest one while its toast is owed; the owed number is set before the failed frame is emitted and ended by the next press. The panel asks at load and whenever a frame turns failed, andacknowledge_update_attempt_toast(attempt)ends it; a number that is not the owed one changes nothing. The acknowledgement also stores what outlives this process, so no later start raises a second toast for the same failure: where the attempt's record was written, itsstarted_atunderupdate_stopped_toasted_at— the record's removal may have failed, and the next start would judge it stopped — and forinstaller_stopped, where the installer's record reads as the pre-install check refusing this attempt (it could not be read when the attempt ended), that record's toast as raised, throughFailureToastAcknowledgeFn(UpdateOutcomeService.acknowledge_update_failure_toast). - The failed installer's start (
failed_installer_started_at) isstarted_atof the latest attempt where it failed after its installer ran —installer_stoppedornew_version_does_not_start, a stopped attempt an earlier start found among them, and still after that one is dismissed, for as long as the attempt is the latest — andNoneotherwise.UpdateOutputServicereads it to pick the installer's run.
UpdateOutputService notes¶
The service answers get_update_output: what the installer printed for one failed update, read back from the journal
through JournalEntriesFn (adapters/journal.py, journalctl --user --output=json --all, run off the loop; without
--all the JSON output answers a field over 4096 bytes as null). It only reads the journal; its own log lines reach
it as every backend line does. What is shown of a run is domain/update_output.py's.
- Which failure. The argument is the installer's record's
rolled_back_at, answered only while that record stands, orNonefor this process's latest attempt, answered only while it failed after its installer ran (failed_installer_started_at). Anything else isnot_found, logged at INFO with what was asked, and an argument of another typeinvalid_value, logged as a WARNING — a panel asks only about a failure it shows. - Which run — never merely the latest. A run is one
_SYSTEMD_INVOCATION_ID(the unit's own lines) orUSER_INVOCATION_ID(the user manager's lines about it); a line with neither belongs to no run and is dropped. A line a short-lived process of the installer's wrote often carries neither — why is the comment ininstall.sh'ssay_what_the_check_said— so the installer prints every line meant to be read from its own shell (TestTheInstallerSaysItsOwnLines,tests/scripts/test_install_sh.py). For an attempt, the run is the first run ofromm-tender-updatethat began at or after the attempt's installer was started. For a record, it is the run going on at the record's stamp — begun no later than that second and still printing at it, since the installer writes the record partway through — looked for an hour either side. - After a rollback, a second part. The installer's own output says only that the new version did not answer. The
failed version's reason is in the
romm-tenderunit's journal between the installer's run beginning and the record's stamp, which the installer writes before it starts the version it went back to. The run already going when the installer began — the version it replaced, which it stops in that window — is the run of the unit's newest line before that moment that belongs to one (of its last 50,REPLACED_RUN_LINES), and its lines are left out. A failed version that printed nothing leaves no second part. - Nothing to show is a reason, not an empty log. Where no such run is there,
missingsays why. Where the journal holds no line at or before the failure's moment — the record's stamp, the attempt's installer start — the lines have rotated out:rotated. Where it does, it reaches back that far: for a record the installer did not run as a unit —terminal— and for an attempt, which this program started as a unit, that unit left nothing there —empty. A journal that cannot be read, or gives no answer in ten seconds, isjournal_unreadable; whatjournalctlsays on stderr while succeeding — a journal file it skipped — is logged at debug. - What is shown of a run. The last 300 lines, each cut at 500 characters, with how many earlier ones were left out
(
earlier), and every admission-token value replaced by[hidden]before a line leaves the backend — why, and the spellings that catches, is the register's journal-line entry (invariants). - Each of the installer's rows in its last state. With no terminal,
install.shprints a row again on every step —[..] Checking, then[..] Checking python 3.13, and so on to[ok] Checking …— so the installer's part (installer_section) keeps only the last line of each row, where that line stood: a finished row's[ok],[!!]or[--], and the[..]of a row that never finished, which is where the installer stopped. A row line is one of those four marks, a space, the label padded to twelve columns, a space and the detail — or, while a row has no detail yet, the bare label with its padding dropped ([..] Checking) — asprint_rowandsay_progressprint it, and the row is its label; every other line stays, in order. The folding comes before the 300-line cut, soearliercounts only lines the folding kept. The new version's part is not folded.
Adapters (backend/adapters/)¶
Adapters own all I/O, and implement the Protocols defined in services/protocols/ wherever a service puts the question
through a seam. Selected adapters:
| Module | Role |
|---|---|
romm/http.py |
RommHttpAdapter — HTTP transport: auth, SSL, User-Agent, configured proxy headers, error translation |
romm/retry.py |
RetryLadder — the attempt policy behind that transport: retry ladder, backoff, known-unreachable state |
romm/romm_api.py |
RommApiAdapter — RomM REST surface (saves, ROMs, platforms, firmware, devices, play-sessions) over the HTTP transport |
steam_config.py |
SteamConfigAdapter — Steam VDF read/write, grid dir, shortcut icon write, Steam Input config |
steamgriddb.py |
SteamGridDbAdapter — SteamGridDB REST client; every failure to reach SteamGridDB or to read its answer raises SgdbApiError, its status_code None where there is no HTTP status, and an image download that fails answers False; a key no HTTP header can carry raises SgdbKeyUnsendableError before any request, which the shortcut artwork job answers as a refused key |
github_releases.py |
GithubReleaseAdapter — LatestReleaseFn: GitHub's "latest release" route, the tender-v tag and the release's romm-tender-<V>.tar.gz asset with its digest and its .sha256 asset; every failure answers None. And ReleaseAssetDownloadFn: one asset downloaded with byte progress, written beside the destination and renamed into place; a failure raises |
update_failure.py |
UpdateFailureFileAdapter — UpdateFailureFn: the installer's update-failure.json under the state root; a record that is missing, unreadable or malformed answers None |
update_staging.py |
UpdateStagingAdapter — UpdateStagingStore: <cache root>/update/, removed whole; the tarball under its asset name, its sha256, and only romm-tender/install.sh copied out of it, as a regular file, under a name chosen here |
journal.py |
JournalctlAdapter — JournalEntriesFn: journalctl --user --output=json --all over one unit's entries or every unit's, bounded by time and count, each line decoded by domain/update_output.py, and what it says on stderr while succeeding logged at debug; reads only |
transient_unit.py |
SystemdRunAdapter — TransientUnitControl: systemd-run --user --unit <name> --collect --setenv=… and systemctl --user show --property=ActiveState; --collect so a unit that failed does not keep its name loaded |
bounded_run.py |
run_bounded — a short command run to its end for journal.py and transient_unit.py, output decoded as UTF-8 with bad bytes replaced; on a timeout the command is killed and waited for at most two seconds more, where subprocess.run would wait without a bound |
sgdb_artwork_cache.py |
SgdbArtworkCacheAdapter — on-disk SGDB artwork cache |
cover_art_file_store.py |
CoverArtFileStoreAdapter — RomM cover art I/O across the per-ROM cover cache and the Steam grid dir (download, copy_file publish/seed, read, prune) |
persistence.py |
PersistenceAdapter + per-domain persister adapters — settings.json read/write |
repositories/ |
SqliteUnitOfWork + per-aggregate repository adapters — SQLite I/O (the live persistence path; see Database Design) |
sqlite_migrations.py |
apply_migrations — schema migration runner (db/migrations/NNN_*.sql, PRAGMA user_version) |
download_file.py |
DownloadFileAdapter — download filesystem |
firmware_file.py / migration_file.py / rom_files.py / save_file.py |
per-subtree filesystem adapters (BIOS, RetroDECK migration, ROM removal, local saves) |
retrodeck_folders.py |
RetroDeckFoldersAdapter — RetroDECK's folders, each the resolver's answer through RetroDECK's handle: a system's ROM folder, which also bounds the removal of that system's games, the ROM root, BIOS folder and saves root, the move code's roots, and the refusal where a download or a removal may not use one (Config source parsers) |
recovery_bundle.py |
RecoveryBundleAdapter — recovery root recovery/ inside Tender's folder as the settings name it, which it never creates, exact source measurement, verified copies, checksums/human manifests, fsync, staging cleanup, and atomic bundle sealing |
tender_folder.py |
TenderFolderAdapter — the folder's place read from the live settings, its condition (drive mounted, present, writable), creating it with its .tender-folder id file, removing one that holds only that file, and the writable drives mounted under /run/media/<user>/, each with its free space and its kind — SD card, USB drive or other — read off the block device behind the mount (/sys/dev/block/<major>:<minor>) |
prune_artifacts.py |
PruneArtifactAdapter — per-ROM cover/validator and SteamGridDB cache discovery/removal |
steam_recovery.py |
SteamRecoveryAdapter — per-shortcut Steam grid, Steam Input, and controller-setting recovery inventory plus post-confirmation cleanup |
emulator_sources.py |
EmulatorSourcesAdapter — the one place the resolver's installations are detected (per reading, never at construction) and held, with the one RealMachine every detection is handed; arranges them by the stored order and switches and names the answering source. Its DetectedSourcesReading keeps the answers a reading was asked, for the call or the run that took it. It also answers every detected source's ROM folder and system folders, resolved (rom_folders, each source's system folders asked in one question, rom_locations; where detecting the sources or a question to one of them raises, it answers nothing, and the place is refused with sources_unreadable), which a place for Tender's folder is judged against |
atlas_catalogue.py |
AtlasCatalogueAdapter — the ES-DE emulator catalogue through the vendored emu-atlas resolver (the picker's list, the system-layer default, the libretro active core, the per-system accept-list), and whether each entry's emulator is installed and where its launcher lies. Sorts the resolver's effective order back to the declared one, so no gamelist selection moves the default (ADR-0030) |
atlas_launch.py |
AtlasLaunchAdapter — one game's launch command, asked of the answering source with the entry Tender starts the game through and the game's own path, read into Tender's view (domain/start_command.py): the program and its arguments as Tender runs them, with the folder grants, --nosocket=wayland and the environment written in, or why the resolver gives no complete command; the reading module for the resolver's launch-command codes (Core and Emulator Selection) |
start_handover.py |
StartHandoverAdapter — writes one start's command into <runtime dir>/start/<appId> for bin/tender-rom-launcher to claim: the folder it starts in, then the program and its arguments, each ended by a NUL, written whole through a staging file and readable by the user only; built only where XDG_RUNTIME_DIR is set |
atlas_platforms.py |
AtlasPlatformSystemsAdapter — a platform's system in one emulator source, asked through the vendored resolver (systems_for_platform) with the platform's ids in turn, of the answering source or of the source a caller names; the reading module for the resolver's platform vocabularies and statuses (Core and Emulator Selection) |
atlas_saves.py |
AtlasSaveLocationAdapter — where one ROM's save lives and what it consists of, asked of the catalogue entry Tender resolved for it and always with the ROM's own content path, since the answer turns on the content file's extension (ADR-0034). Caches no answer and holds no installation: every call takes a fresh reading of the sources. Also describe_core_probe_interpreter(), the start-up log line naming the interpreter the resolver's core probe would run under |
atlas_zstd.py |
register_zstd_codec() — hands the resolver the vendored backports.zstd where the interpreter has no compression.zstd, so EmuDeck's catalogue sealed in its AppImage can be read; answers the start-up log line naming the provider |
gavel_native.py |
GavelNativeAdapter — loads the compiled romm-gavel core (backend/native/libgavel-x86_64-linux.so) via ctypes; is itself the ResolveUploadConflictFn seam and provides the ComputeSyncActionFn seam, the two save-sync decisions (no Python fallback) |
system_clock.py / system_uuid_gen.py / asyncio_sleeper.py |
concrete Clock / UuidGen / Sleeper seams |
hostname.py / path_probe.py / debug_logger.py |
hostname, the generic path seams (exists, symlink-resolve), settings-aware debug logger. The program's own name and version are not read from anywhere — they are constants in domain/identity.py |
renderer_rss.py / renderer_gc.py |
RendererRssFn — max steamwebhelper VmRSS from /proc; RendererGcFn (HeapProfiler.collectGarbage) over the CEF debugger. The session-budget measure + settle seams (ADR-0024). The "free memory" action is a frontend SteamClient.User.StartRestart, not a backend adapter |
steam_process_tree.py |
GameProcessControl — finds the reaper Steam started a shortcut under (SteamLaunch AppId=<appId>) and walks its /proc tree, deepest first, and signals those processes; and SteamLaunchReader — walks from a process up its parents to the reaper it runs under and answers that launch's app id. Direct reads + os.kill, no subprocess; fail-soft on every read |
running_steam.py |
RunningSteamReader — the Steam this user runs: its pid file, executable, start time, whether gamescope started it, and the X display and authority file of the session it was started from, read off /proc; fail-soft on every read |
x11_focus.py |
FocusedWindowReader — one connection to the session's X server through the system's libxcb over ctypes, loaded on the first connection: the root's focus changes, and the focused window's STEAM_GAME, STEAM_INPUT_FOCUS and X-Resource pid |
RommHttpAdapter notes: one retry ladder per call stack¶
How often a request is attempted is not the transport's own question: RetryLadder (romm/retry.py) owns the ladder,
its re-entrancy scope and the reachability state, and RommHttpAdapter holds one and delegates with_retry /
is_retryable to it — the two methods that make the adapter a RetryStrategy.
with_retry runs up to 3 attempts with a base_delay * 3^attempt backoff (1s then 3s — a 3-attempt ladder has only two
gaps) and only retries what is_retryable calls transient — never a 429, and never a download's destination this device
cannot open, write or close (is_retryable refuses both by name and says why). The adapter raises that one as
DeviceWriteError, an OSError rather than a RommApiError, carrying the system's words without the file's path; as
an OSError it would otherwise be retried, and as no RommApiError it leaves the reachability state below alone. Where
a feature has an answer of its own — a firmware download, a cover refresh, the slot-setup migration, a copy into a slot,
a restore — it words the fault as one on this device; elsewhere the system's words are the message. with_retry is
reached from two levels: most of the adapter's own request methods wrap themselves in it, and services wrap adapter
calls again through the RetryStrategy protocol (satisfied by RommHttpAdapter itself). The outermost ladder wins
— a re-entrant call runs its function straight through, guarded by a thread-local flag on the ladder, because the
blocking work runs one call per executor thread. Without that guard the two levels multiply into 9 HTTP attempts with
both backoffs stacked, which is what made a single game-detail page take ~30 s to fill in against an unreachable server.
The guard lives in with_retry rather than in the ~20 service call sites so that a twenty-first site cannot
re-introduce the nesting, and because the service-level wrap is the only ladder for the adapter methods that
deliberately have none (upload_multipart, request_once, unauthenticated_post_json, basic_auth_request) and for
the coarse wraps whose callee paginates or chains several requests — those keep being retried as the whole unit their
caller meant.
The ladder also remembers one bit of transport state: the server is known unreachable. A ladder that gives up on an
exception that means the server could not be reached sets it; while it is set every ladder shrinks to a single attempt
with no backoff, and a ladder already sleeping cuts its backoff short (the lanes a game-detail page opens start
simultaneously, so this is what makes the first load after an outage fast, not only the second). The degraded ladder
still really performs its call — it is never skipped — which is what makes the state self-healing and what makes the
UI's Retry button work with no path of its own. Any successful response clears it, including on the ladder-bypassing
paths, because every request funnels through one _urlopen choke point; the reachability probe and the 30 s heartbeat
therefore clear it for free. A new request method that opens its own connection would silently stop clearing it and
leave Tender degraded, so scripts/check_urlopen_choke_point.py confines urllib.request.urlopen to _urlopen. An
HTTPError deliberately does not clear it either: a 5xx answered by a proxy in front of a dead origin is one of the
shapes that counts as unreachable in the first place. A 5xx RomM answered itself is told apart by the first half of the
proof a 404 needs to be an entity verdict, without its blocklist
(what makes a 404 an entity verdict) — a body announced as
JSON, the only one read, whose detail is a non-blank string: it becomes RommReportedServerError, is still retried,
and answers its own server_error reason with the first line of RomM's detail, so it never sets the state. Any other
5xx — a proxy's 502 for a stopped container, an HTML or empty body, and every 5xx from the third-party cover CDN — stays
RommServerError and server_unreachable.
An error carrying a 4xx status code never sets it — the server answered, whatever it answered. Every HTTP error RomM
or a proxy answered carries its status, except the unproven 404 below: a 4xx Tender has no class of its own for (400,
405, 413, …) is RommClientError, and classify_error answers it — and the 409 and 422 that reach it unhandled —
unknown with "RomM or a proxy in front of it refused the request (HTTP 413 Request Entity Too Large)", the status and
phrase of the answer and never the request's address. classify_error gives every error carrying a 4xx a reason other
than server_unreachable, but the peel still runs before it: that is a user-messaging classifier whose last branch
folds every status-less RommApiError onto server_unreachable so a display string always exists, and the rule here
does not rest on its branch order. The 409 is the case that makes this load-bearing rather than tidy — every automatic
save upload POSTs overwrite=false precisely so RomM can reject a stale head with one, and that upload runs inside a
ladder, so a routine conflict would otherwise mark the whole server unreachable. The 404 is peeled for the same reason
and matters for a second one: it is deletion authority downstream (next section). An unproven 404 does still set it,
because it degrades to a plain RommApiError carrying no status code — nothing proved RomM answered, which is the same
fail-open reading that denies it deletion authority.
The state is about the configured RomM server only. download_external fetches a ROM's url_cover from a third-party
metadata CDN, so its ladder is entered with romm_origin=False and takes no part in either direction: a dead cover CDN
must not degrade every RomM call, and reaching the CDN is no evidence that RomM came back. It also keeps its full ladder
while RomM is down, because it is a different host.
RommHttpAdapter notes: the headers every RomM-origin request carries¶
_apply_origin_headers is the one place that attaches what a request carries by virtue of its DESTINATION rather than
its purpose: Tender's User-Agent and the user's configured proxy headers (settings.json romm_custom_headers, #1822
— for a RomM server behind Pangolin, Cloudflare Access, Authelia or Authentik forward-auth, which rejects every request
before RomM sees it). _apply_default_headers is that helper plus Authorization, and the two sign-in paths that
deliberately omit the bearer — unauthenticated_post_json (the pairing-code exchange) and basic_auth_request (the
Client API Token mint) — call the helper directly. All three attachment points are deliberate: an authenticating
proxy sits in front of sign-in as well, so a client that added the headers only to authenticated calls would leave a
user unable to authenticate in the first place.
The one deliberate exclusion is download_external, which fetches a ROM's url_cover from a third-party metadata CDN
and keeps the bare User-Agent it always had. A proxy access token is a credential for the user's own front door and
has no business reaching a foreign host — the same reasoning that keeps the RomM bearer off that request.
A configured header may never displace one the adapter sets itself, and two mechanisms hold that — unevenly.
The first is validation: the names are refused case-insensitively (domain/custom_headers.py, RESERVED_NAMES), with
authorization carrying a message of its own, because a proxy's own documentation suggests that header and it is
exactly the one the RomM bearer occupies. It is reached from two places — resolve_custom_headers for what arrives over
the wire, and stored_custom_headers, the single reading of the persisted list, which re-runs it on every request so a
hand-edited settings.json cannot put a reserved name on the wire either. Those two gates are one mechanism read
twice, not two: both consult the same frozenset, so removing a name from it opens both in a single edit.
The second is ordering: the configured headers go on FIRST, so a later req.add_header wins the same name whatever came
before it. This one is partial, and the gap is worth knowing. It covers only the names the adapter re-adds after
_apply_origin_headers — User-Agent, Authorization, Content-Type, and conditionally Accept-Encoding (on the
JSON GETs only), Range, If-None-Match and If-Modified-Since. It does nothing for Host or Content-Length, which
the adapter never sets: http.client supplies both itself and skips its own when the caller already did. So for host
— reserved precisely because http.client._send_request suppresses its derived Host when the caller supplied one,
which would retarget every request's virtual host — the frozenset is the only defence that exists.
Nothing pins the ordering leg. test_a_stored_authorization_never_displaces_the_bearer and
test_a_stored_host_never_retargets_the_request read as though they do, but both pass because stored_custom_headers
has already dropped the entry before add_header is reached; reverse the order inside _apply_default_headers and both
stay green.
One thing the attachment rule does not reach at all: what Tender attaches is not the same as what arrives. _urlopen
uses urllib's default opener, whose redirect handler copies every header except content-length / content-type onto
the followed request with no same-origin test, so a 30x from the RomM origin to a foreign host delivers the configured
header — and the RomM bearer with it. That is the transport's long-standing behaviour rather than anything this feature
introduced, and it is tracked in #1889.
The values are read from the live settings dict at call time, the way romm_url is: a value captured in __init__
would go stale the moment the user edits it. They never reach a log line — CustomHeader.__repr__ prints the name
alone, and no refusal message quotes a value.
RommHttpAdapter notes: what an error's text carries¶
Where an error the adapter raises names a request, it names its method and URL, path and query included — never a
header, and never the address's user name or password: translate_http_error and _translate_with_detail write the URL
through lib/url_host.without_userinfo, and RommApiError stores its url the same way. A password holding an
unescaped /, ? or # is not recognised as one (_userinfo_end says why).
A request urllib or http.client cannot build raises an exception whose text holds the material it refused —
InvalidURL for an address it cannot read as a host and port, a ValueError from urllib.request.Request for an
address without a scheme or from a header value, UnicodeEncodeError among them. For those alone
(_could_not_be_built) _translate_unwrapped keeps only the type (Unexpected error: InvalidURL), and the error is
raised without its cause, so no traceback shows the original either; _new_request does the same for the Request
every method builds before its translated try. An answer that could not be read — json.JSONDecodeError,
UnicodeDecodeError, both ValueError too — keeps its text and its cause. Either way the reason stays what it was: a
status-less RommApiError, which classify_error answers server_unreachable.
tests/adapters/romm/test_http.py::TestNoCredentialReachesAnErrorOrALog holds every public request method, failing
every way, to that, traceback included.
RommHttpAdapter notes: what makes a 404 an entity verdict¶
RommNotFoundError means "RomM's entity layer says this entity does not exist", and downstream that reading is
authority: the removed-game cleanup deletes on it, the version picker marks a version vanished on it, save-sync drops a
stale device registration on it. A bare HTTP 404 does not carry that meaning on its own — FastAPI answers a
misconfigured path prefix with the same status and a generic {"detail": "Not Found"} body, and a reverse proxy in
front of RomM (Cloudflare Tunnel, Traefik) answers a misroute with an HTML or empty one. So on the API routes the
adapter raises RommNotFoundError only for a response that proves it came from RomM's entity layer: a JSON content type
whose body parses to an object carrying a detail string that is neither blank nor FastAPI's stock Not Found (matched
case-insensitively — a real entity answer always names the entity, so it is never the bare phrase). The requested id is
deliberately not parsed back out of that detail — its wording moves between RomM releases while the generic default
stays put, so blocklisting the default is the robust test. Every other 404 shape degrades to a plain RommApiError,
which classify_error maps to server_unreachable, so infrastructure can no longer authorize a deletion and each
caller fails open on it instead.
The byte-stream routes — download, download_conditional, download_external — opt out via
translate_http_error(..., asset_route=True) and keep the plain status mapping. Their 404 answers about a file, not
an entity, so it is never deletion-authority grade; and it has to keep raising RommNotFoundError, because RomM serves
its cover resources from a static mount where a genuinely missing cover answers with exactly that generic route-404
body, and the sole consumer reacts by refetching from the ROM's url_cover (the fallback above) — non-destructive and
self-correcting rather than a deletion. Each of the three call sites carries that constraint as a comment: a future
unification of the two classes would silently disarm the fallback, so both sides are pinned by tests.
Every deletion-authority probe reaches the network through the JSON-API entry points, never a byte-stream one:
get_rom_once (request_once, on /api/roms/{id}/simple) behind the removed-game cleanup's exact-id proof and the
version picker's liveness probes, get_rom_simple (request, the same route) behind the picker's bound read, get_rom
(request, /api/roms/{id}) behind a server-only switch target's fetch, list_saves (request) behind the
save-status, copies and slot-setup reads, and update_device (put_json) behind the device re-registration. The one
consumer that branches on a byte-stream 404 is the url_cover fallback.
The entity-answer shape is captured from RomM 5.1.0, and RomM 5.3.0 raises the same one — a FastAPI
HTTPException(status_code=404, detail=…) naming the entity — for a missing ROM (exceptions/endpoint_exceptions.py),
save (endpoints/saves.py) and device (endpoints/device.py). If a future release's entity-404 ever carries a
different body shape (no JSON content type, no detail string), every entity-404 on that server degrades to
server_unreachable: the cleanup never confirms a ROM gone, no version is marked vanished, no stale device registration
is dropped. That is the fail-open direction by design — if a user on such a server reports exactly that symptom set,
this paragraph is the explanation, and the fix is a version-aware entity check, never a return to trusting the bare
status.
PersistenceAdapter notes¶
- File locking: write methods acquire an exclusive
fcntl.flockbefore touching the file, preventing concurrent writes from corrupting state. - Schema versioning:
settings.jsoncarries aversionfield. A file this release does not read is not converted: one older thanOLDEST_SETTINGS_VERSION(13, the version every release from 0.30.0 on writes), one whoseversionis missing or not a whole number, and one that is not a JSON object (domain/state_migrations.unreadable_settings). The adapter logs a warning naming what it found and returns the defaults, which the start's save writes over the file — no backup and no notice, because everything such a file held can be entered again. A file it reads goes throughmigrate_settings, whose one step (version 14) turns each bareplatform_coreslabel into{label, source}, a bare label becoming RetroDECK's (#2195 D20). - Crash-safe atomic writes:
settings.jsonis written with the durable write-tmp →fsync(tmp)→os.replace()→fsync(dir)recipe. The temp file's bytes are forced to disk before the rename, and the directory entry the rename creates is forced to disk after it. This closes the power-loss window on the Steam Deck's ext4: without the fsyncs, a crash after the rename but before the kernel flushed could leave a truncated or emptysettings.json— which boot rewrites every run, so the window recurred. The directory fsync is best-effort: on the rare filesystem that rejects it, the error is logged at debug and swallowed (the content is already durable via the temp-file fsync). - Corrupt-file quarantine (never silently factory-reset): a
FileNotFoundErroron read is a legitimate first run — defaults are returned silently, no backup, no flag. An unparseablesettings.json(aJSONDecodeError, e.g. a truncated file from a prior crash) is the data-loss hazard: returning defaults silently would let the immediate bootstrap save overwrite the corrupt file, wiping the user's RomM URL, API token, SGDB key, and platform/collection selections with no trace. Instead the adapter logs the corruption loudly at error level, renames the unparseable file aside tosettings.json.corrupt-<ts>(the<ts>is the injectedClock's epoch seconds — filesystem-safe), restricts the backup to0600like the live file — it still holds the token and the SGDB key, and a rename keeps whatever mode the corrupt file had — and sets a transient in-memorycorrupt_resetflag before returning defaults. If the backup rename itself fails (e.g. permissions), the error is logged and defaults are still returned so boot never crashes; a failedchmodis logged as well and leaves the backup and the flag standing. Bootstrap reads that transient flag after migration and — before the immediate save — folds it into the settings dict as a persistent_settings_reset_noticemarker ({"backed_up_to": <basename>}), so it survives a backend restart. The frontend reads it via the non-consumingget_settings_reset_noticeendpoint (SettingsService, as the dismissal is) and surfaces a persistent notice — a QAMPanelSectionbanner (with a Dismiss button) plus a game-detailWarningCard(informational; its copy points the user to the QAM to dismiss) — not a toast — telling the user their settings were reset and where the backup landed so they can re-enter the server URL and sign in. The marker is cleared only by an explicit user acknowledgement: the QAM Dismiss button callsdismiss_settings_reset_notice, which pops_settings_reset_noticeand persists; the frontend clears the shared store on success so the banner and every game-detail card disappear at once. Sign-in does not clear the notice — the user decides when they have read it. - Version never down-stamps: on write, the
versionfield is stamped tomax(stored_version, _SETTINGS_VERSION). A file written by a newer version of Tender (stored version > current) is preserved as-is rather than down-stamped, so a later re-upgrade does not re-run migrations against down-stamped data. An absent or older version is stamped up to the current_SETTINGS_VERSION.
GavelNativeAdapter notes — the compiled save-sync core¶
Both save-sync decisions run through the compiled romm-gavel core rather than in-tree Python:
- the full per-
(rom, filename, slot)sync action (Skip/Upload/Download/Conflict— the matrix in Save File Sync Architecture), and - the upload-409 resolution (the decision, on a
409from RomM'sadd_save, to eitherdownloadthe server head or surface aconflict).
GavelNativeAdapter loads backend/native/libgavel-x86_64-linux.so via ctypes at construction and binds both
symbols. It is itself the ResolveUploadConflictFn seam, and its compute_sync_action method is the
ComputeSyncActionFn seam (both in services/protocols/infra.py); both are injected through SaveServiceConfig →
SyncEngineConfig → MatrixExecutor, which calls the decision once per file in iter_matrix_outcomes and the 409
backstop in _handle_upload_409 (services/saves/sync_engine/matrix.py).
- What crosses the boundary: the adapter marshals the caller's raw dict shapes onto the core's caller-owned structs.
Two conversions are the adapter's, not the core's, and both exist so an "unknown" stays a flag instead of becoming a
value: a server save's ISO-8601
updated_atbecomes epoch seconds plus ahas_updated_atflag (an unparseable timestamp arrives as a cleared flag, never as a substitute instant, so it cannot win head selection), and an absent size or content size becomes a clearedhas_*flag (a raw size of0is what the guard refuses outright, and a content size of0is a file of fill only, so neither can double as "no size recorded"). A non-integral size raises rather than rounding, and so does asize_fixedthat is not abool: an absent orNoneone is unknown, and anything else would be read as yes or no by a guess. Presence of the local file rides on the pointer alone — a file that exists but could not be measured is a real case, distinct from a missing one. The core answers with the chosenserver_save_id, which the adapter resolves back to the caller's own save dict soDownload,Conflictand an adoptingSkipcarry the full record their consumers read; an adoptingSkipalso carriesacknowledge, and a skip that adopts nothing names no head. - What ships:
backend/native/libgavel-x86_64-linux.so(romm-gavelv2.0.0, a freestanding build with zero library dependencies — it loads on any x86_64 Linux), vendored verbatim from the upstream release with a pinned SHA-256 checksum. The C ABI has been part of upstream's promise sincev1.0.0: struct layouts, signatures and enumerator values cost a major bump to change, which is what makes pinning a compiled artifact meaningful — and why the binary and thectypesstructs inadapters/gavel_native.pymove together. Provenance and the update procedure live innative/README.md. The checksum is re-verified by CI, so a swapped binary fails the pipeline; nothing asserts the.soreaches a built package, because no build here produces one. - No fallback: if the library cannot load,
GavelNativeLoadErrorpropagates sobootstrap()aborts and Tender stays inert — the same "fatal until the environment is fixed" posture as the SQLite migration gate. There is no Python implementation of either decision to fall back to:domain/sync_action.pyholds only theSyncActionvocabulary the core answers in. What holds the shipped binary to the contract is the vendored gavel vectors — the ladder family intests/adapters/test_gavel_native.py, the decision table intests/adapters/test_gavel_native_table_vectors.py— alongside the hand-enumerated cases intests/adapters/test_gavel_native_decision_table.py, the property tier intests/adapters/test_gavel_native_property.py, and the boundary cases that pin the two marshalling conversions above.
FirmwareService notes — the live firmware resolver¶
Which firmware files an emulator wants is read live off the machine through two seams — FirmwarePlatformResolver
for one platform, FirmwareResolver for the whole machine — both implemented by adapters/atlas_firmware.py over the
vendored emu-atlas copy in backend/_vendor/atlas/ (provenance in
_vendor/README.md). It replaced
defaults/bios_registry.json, a frozen snapshot that no longer exists upstream and could never be refreshed again.
- The seam exists because of a contract, not a preference.
domain/may not import_vendor(thedomain-stdlib-onlyimport-linter contract), so the answer has to arrive through an adapter. That adapter is the one module that reads it (Resolver-reading gate under Boundary Enforcement): whatdomain/firmware_wants.pyand the modules after it still read of the resolver's vocabulary stands on that gate's list of today's readers (READERS_TODAY), which only shrinks. - Never cached. A firmware answer is about files the user is actively downloading and deleting, so it is read per query.
- Failure is "unknown", never "nothing needed". The resolver raises on its own invariant violations and promises
nothing about not raising, so the adapter wraps every call and answers an unresolved catalogue. Downstream that
classifies every file
unknown— a BIOS warning is never cleared on ignorance (#1693). - Caveats are the only degradation channel — the resolver never logs. The adapter carries their stable
codevalues (never the humanmessage, which may change freely) and traces them through the injectedDebugLogger.
The question is asked per platform, and the scope is what makes the two seams different.
firmware_for_system(<system>, verify=True) answers for every emulator ES-DE offers for one system — libretro and
standalone alike — and it is what every platform-scoped answer reads. firmware_inventory() sweeps the machine instead
— the installed .so files, one entry each, plus (since the resolver's 0.19.0) the standalone emulators a packaged card
covers — and it is asked unverified, so a card that identifies its image by content names no file there at all. It
is the right question only where the caller has no platform to name, which is one place: the RetroDECK-home migration's
untracked-BIOS sweep, which asks it for placement — where a file lies — never for readiness.
The per-platform reading is verified, and that is a contract rather than a tuning choice. Two answers cannot be had
without reading bytes. A packaged rule card may identify its image by CONTENT, so unverified it names no file while
still answering declaration="packaged" — DuckStation comes back with an empty requirement list and no system
recording, which a reader that counts the list renders as a green "nothing required" over a console that does not boot
without an image. And a folder declaration is satisfied by a file INSIDE the folder, which no stat can settle. Measured
on the reference machine at emu-atlas 0.22, the median of five runs each: 59–173 ms per system verified, against 172 ms
for one unverified whole-machine sweep, because the per-system read sweeps that system's own scope rather than the whole
BIOS root — it performs no unclaimed sweep at all. The overview therefore pays one reading per platform it renders; that
is the cost of answering for a platform whose emulators are all standalone.
One emulator is one identity. The resolver states it under emulator, in the spelling the launch command uses — a
libretro entry's core file basename (dolphin_libretro.so), a standalone entry's own command name (DUCKSTATION,
PCSX2) — and both a catalogue entry and a firmware answer carry it under that one name, which is what lets a pick made
in the emulator picker be matched against the firmware answer it should be judged by. adapters/atlas_identity.py is
the one place that reads it. Three neighbouring fields look like they would answer and must not be used for it: label
is presentation (ES-DE lists one pcsx2_libretro.so as both LRPS2 and PCSX2), core_so is None for every
standalone emulator, and a caveat's token is the resolver's own packaged-card vocabulary, which upstream states is a
different vocabulary again. An identity of None is a real state — the resolver could not identify the emulator behind
a row — and nothing may be scoped to it.
A catalogue lists rows, and two rows can be one emulator. ES-DE declares pcsx2_libretro.so twice for ps2, and
EmuDeck declares Cemu natively and under Proton; 15 of ES-DE's 172 systems carry at least one repeated identity. So a
file's declaring set is deduplicated by identity, the first row winning — which is safe because the rows differ in the
command that launches the emulator, never in the declaration they read.
Matching is per platform; completeness is per LAUNCH. They are different questions and collapsing them forces a
false choice. A file is needed / optional when any emulator in the platform's answer declares it. Whether an
absence may be read as "nothing wants it" is a property of the reading and is scoped to the one emulator the game
will launch with: read → not_needed; unread, unidentified, or the resolver unable to answer → unknown. Collapsing
not_needed and unknown is the defect the four-valued answer removes — a file nobody wants and a file nothing could
be asked about are different answers, and the old boolean called both "not required".
Scoping the doubt to the launching emulator alone is deliberate. An emulator ES-DE also offers, that nothing could be read for, says nothing about a launch that does not use it — under the old platform-wide scope one unreadable core took the whole platform's answer grey whichever emulator the game ran. What it costs is that switching emulators can move a platform from a finished answer to a withheld one, which is the truth about the new launch rather than a regression in the old one.
A declaration that established nothing is unread, whatever its state says. read with an empty requirement list is
the one pairing that means "this emulator needs no firmware". packaged with an empty list means a card exists and this
query established nothing — the same silence a missing .info leaves where atlas packages no declaration for the core —
so it counts as unread; PCSX2 answers exactly that when its own ini names no BIOS file. A libretro core installed
without its .info answers packaged too where atlas carries a declaration for it, with core-info-packaged beside
it; Tender reads the state by the rule above, and the statement changes nothing
(#2644 D3). unsupported is an emulator the resolver has no
source for, and a refused declaration counts as unread too: the emulator does want something the resolver would not
follow to a destination.
One platform, one emulator. Which emulator a platform's BIOS answers are ABOUT is resolved once, by
domain/emulator_commands.py's resolve_firmware_option — the per-platform override (settings.json platform_cores)
while its label still names an entry, else the entry the source ships first, whatever its state (#2195 D7, D23) — and
every platform-scoped answer takes a projection of that one pick: the pane's active_core_label is its .label, the
BIOS filter's key is its .emulator, and download_required_firmware asks the same function so the button fetches the
set the pane called required. It is the read-path precedence ActiveCoreResolver applies minus the per-game layer,
which is what lets check_platform_bios answer launching_emulator=None from it — that is the platform-level callers'
signal — and a second resolution behind it would put the two surfaces on two emulators. Which is what a device pass
found: with the PlayStation's emulator set to PCSX ReARMed the pane displayed that name and judged the platform by the
system default beside it, so one platform read ok on the game page and missing on the pane. The per-game path passes
the ROM's own chosen entry (ActiveCoreReader.resolve_rom(...).choice.option), whether or not it can start, whose
identity names a standalone emulator as readily as a libretro core; active_core_for_rom cannot be used there, because
it answers None for a standalone pick and would send the page back to the platform's.
Standalone emulators are in the scope, and the answer is only as good as upstream's cards. They are identified and
scoped like any other emulator now. What they want is another matter: _vendor/atlas/data/standalone_firmware.json
carries cards for five (CEMU, DUCKSTATION, MELONDS, PCSX2, XEMU) and those answer declaration="packaged"; every
standalone emulator without a card answers declaration="unsupported", which upstream documents as meaning unknown
rather than "needs nothing", and so reads as an unread emulator here. Measured against the RetroDECK release here,
es_systems.xml offers 20 distinct standalone emulators, leaving 15 uncovered; the count moves with each RetroDECK
release, the shape does not. A platform launching one of those 15 reads unknown — the honest answer, and the one
ADR-0020's deferral was reaching for by a cruder route.
A platform's file list is the union of two sources. The RomM library holds files nothing wants; an emulator can want
a file the library has never had. That third kind is shown like any other and marked on_server: False — the one field
every consumer reads, the row's id: None being an honest absence nothing consults. It counts in the launching
emulator's required counts (it is genuinely required and genuinely absent) and never towards a download button. It stays
out of server_count / local_count too: that ratio is a progress bar over a set the user can complete, and a stock
RetroDECK's SNES emulators declare 26 optional files, so folding them in would report 0 / 26 files, 26 missing for a
system no core requires anything from. known_count / unknown_count are scoped to the server's rows for the same
reason — they are weighed against server_count, and a beyond-server row always classifies needed/optional, so
counting one would cancel the unknown verdict for a platform whose every server file went unanswered. Readiness
therefore needs no server at all: ES-DE names the active emulator, the resolver says what it wants, and the resolver or
the filesystem says what is there (the split is the next paragraph but one). An unreachable RomM costs the files only it
knows about and the ability to fetch anything — not the answer.
A launch's rows are its own emulator's requirements. Two emulators routinely declare one file name, and not always
for one place: Kronos opens kronos/saturn_bios.bin where YabaSanshiro opens saturn_bios.bin. So the adapter keeps
one placement per emulator and name, never folding them by name, and every launch-scoped reader asks
FirmwareCatalogue.placements_for(identity): the launching emulator's own requirement wherever it declares the name —
its place, its presence, its verdict — and the first declarer's for a name it does not declare. The status rows and all
three platform downloads read it, so with YabaSanshiro picked the Saturn row reads not met over a file lying only under
kronos/, and "Download required" fetches the file into the store and links it where YabaSanshiro opens it — and, since
a download links for every emulator whose requirement the file fills and its source does not state is not installed,
under kronos/ for Kronos as well where Kronos is installed
(#2191 D3, D36). download_required_firmware takes an
emulator — the identity the page's emulator selector shows, None for the platform's pick — and fetches what THAT
emulator requires. An identity the answer holds no verdict for is refused with emulator_unknown rather than answered
with another emulator's requirement (_shown_emulator).
Where a link goes is the place the emulator opens, as the resolver names it. FirmwarePlacement.place is absolute:
the requirement's own path, inside the firmware root or outside it (Cemu's keys.txt under RetroDECK's data folder) —
except where the declared place is itself a link leading out of the root, for which the resolver states the requirement
at the link's target and names the link beside it (firmware-file-linked-outside-root), and the place is the link
(adapters/atlas_firmware.py::_links_out). relative_path keeps what the emulator spelled below the firmware root, the
folder a person placing the file by hand needs; it is read off declared, because the resolved path has followed every
symlink — RetroDECK points <bios>/pcsx2/bios at <bios>, so reconstructing LRPS2's location from the resolved path
yields . and loses the folder the core opens. A file requirement whose declared is a folder its core declares goes
in that folder under its own name — LRPS2's configured image; which requirements that is, and why a Beetle PSX spelling
is not one, is _declared_location's.
Presence is the resolver's wherever a file requirement's row has a place. Such a row's downloaded is the
resolver's present at its place, read the way the emulator reaches it, through a link of Tender's and the store file
behind it; a None leaves downloaded false and is no claim that anything is there. Tender's own look at a place
(FirmwareFileStore.look, no link followed) answers whose link lies there, whether a name is free and whether its
folder exists. The one row it answers is a file of a folder declaration: the resolver states a verdict per file only
where it says something about that file, so there downloaded is whether a file lies at the place, and the verdict is
the resolver's statement — or, with none, not in place where nothing lies there and not established where something
does. A library row has no place: its downloaded is whether this platform's store holds it. What the reading found at
a place travels per row — supplied_by (the distribution whose own copy sits there, named in the resolver's own display
form) and caveats (the resolver's stable codes for whatever else it found at or in that place, attributed as the
paragraph on folder words below sets out, and collapsed on the code within one place) — so both surfaces can say what a
row IS instead of describing every one of them as a gap in the library. declared_kind is what the emulator OPENS the
place at, a property of the declaration that survives an empty place.
A folder requirement is answered by what is inside it. LRPS2 declares pcsx2/bios — a folder, required, and always
present because RetroDECK links it onto the BIOS root. Reading presence as the verdict would say "All 2 files LRPS2
opens by name are in place" over a PS2 install with no BIOS file at all; reading absence would say red over a folder
that is plainly there. So BiosFileEntry.satisfied is the row's verdict — the resolver's own per-file verdict for the
launching emulator's requirement (#2190 D3), for a folder its
listing of what the folder holds — and the required counts a sentence words key off it and nothing else:
count_required counts rows answered True, count_required_withheld counts rows answered None, and a row answered
False is a requirement shown to be unmet — absent, the wrong image, refused on its size, or the wrong shape. All three
are scoped to required_by_active first, so an unmet row the launching core does not require moves none of them; the
library's own server_count / local_count ratio is a different axis again and keys off on_server / downloaded. A
library file no emulator declares is no requirement, so the resolver has no verdict for it, and its row has none
(domain/bios_status.py::_row_verdict).
A folder is shown as one row per file of it (#2191 D26):
each library file the resolver recognises for the folder — its MD5 filed under the folder's prefix in the packaged table
(FirmwarePlacement.folder_md5s) — with its download, its link and its Delete, and each file already lying in the
folder that is not Tender's and that the resolver makes a statement about, with its verdict over that file and no Delete
(FirmwarePlacement.folder_files, read off the per-image statements of the verified listing). A file the resolver
passes over because a valid image lies beside it gets no row. The folder itself is ONE requirement in the counts,
answered by its own verdict (count_required's folders), and none of its files is required on its own row. Where that
leaves no row — nothing lies in the folder and no library file is recognised for it — one row stands for the folder,
with its verdict and the codes the resolver states over it, and the line "No BIOS image here yet" (D28,
services/firmware/rows.py::_empty_folder_row).
A folder verdict costs a content read, and the per-platform reading pays for it once. There is no second, narrower
question any more: firmware_for_system(<system>, verify=True) settles the folder rows in the same answer the rest of
the platform comes from, so a row and the verdict over it can never come from two readings. Which rows a stat alone
could have settled is the resolver's own three-valued answer and not a list of shapes held here — today an absent
folder, a plain file sitting where the folder belongs, and a folder holding no file of a size the emulator would open
are all settled without reading a byte, and only a folder with candidates in it is a question about bytes. The seam
reads the disk, so _platform_firmware_resolver — the attribute every consumer binds it to — is registered in
scripts/check_uow_seam_nesting.py's IO_SEAM_METHODS and must never be called with a Unit of Work open; its
whole-machine sibling _firmware_resolver is listed beside it for the same reason.
A row hears what was found at its destination; only a folder row hears what a listing found inside it.
_caveats_by_destination builds two indexes — one keyed by the path a caveat is about, one by the dir a listing was
made in plus the parent of every path, which is how a verified read's per-image findings reach the folder row that
asked for them. A file declaration never draws on the second, which matters because on a linked root the listed folder
is the firmware root and so is the resolved destination of anything that collapses onto it.
A dir on a caveat is not always a folder declaration's own, and that is new with the per-system route. The
resolver states one for a standalone emulator's SEARCH directory too — DuckStation ranks its images in the BIOS root,
which is also where LRPS2's pcsx2/bios resolves — so keyed by place alone a folder row would word another emulator's
search as its own verdict's cause. _speaks_for drops a statement attributed to a different emulator, reading the
resolver's own attribution keys — which they are, and why a row is matched against its core's libretro short name beside
its identity, is adapters/atlas_firmware.py::_ATTRIBUTION_KEYS's; that is how LRPS2's
firmware-configured-image-missing reaches its pcsx2/bios row. The row carries that one as missing_configured_image
— the label of the emulator whose reading the row is and the file its pcsx2_bios option names — and it is never a
verdict: LRPS2 lists the folder instead, so the folder's verdict is what the launch rests on. Its sibling
firmware-configured-image-not-offered — the option names a file the core does not offer, so RetroArch hands the core
its default, the folder's listing — reaches the row the same way and is carried as configured_image_not_offered, no
verdict either (#2644 D4, D7). A caveat naming none of the
keys is a statement about the place with no owner — a listing that failed is the case that matters — and it stays on the
row, which is the permissive direction: the row keeps a cause it might not own rather than losing one it does.
The level is the resolver's verdict for the launching emulator, and no count decides it. compute_bios_level reads
requirements_met off the emulator's entry (CoreFirmwareVerdict): met is ok, not met is missing, not established
is unknown — and an emulator the answer holds no entry for is not read, so unknown
(#2190 D1, D4). The one exception is partial, the platform
page's amber, and it too is read rather than counted (region_decides, #2190 D10): the verdict is open, the resolver
stated that the disc's region decides it, a group of the emulator's has an image met and one not met and none not
established, every other group is met, and nothing else is open — no plain required file it could not judge, no refused
required declaration. Anything else open leaves it unknown. The counts beside it word it — "2 of 3 files melonDS opens
by name are in place" under a red verdict — and compute_bios_label words it as the play row's token. A required row
the resolver could not judge travels as required_withheld, which the shared utils/biosSummary.ts reads to say "One
file X opens by name could not be checked", in the same verb the counts take. The play-row badge reads none of these: it
rises on missing, the red verdict. How the emulator finds its firmware (locating, carried as the resolver spells it)
words the counts — "opens by name", "lists as required" — and for an emulator that searches its BIOS folder by content,
domain/bios_status.py::classify_content_search says what the search found, off the rows' verdicts, checked and
caveat codes and the emulator's own caveat codes (#1883) —
"found" only where the resolver states each file identified by its contents, and nothing at all where it states none of
the three answers; neither moves the level. A search that settled and reached no image is not among them: the resolver
states it as firmware-path-names-no-file with the folder it searched (dir) and how many files of an accepted size it
held (candidates), and adapters/atlas_firmware.py::_notes carries exactly that shape as the emulator's
search-found-nothing note — a required file not met is never read as one. What a withheld row SAYS comes from its
caveat codes and, for a declared FILE, from its checked — the resolver's own word for what became of the bytes, but
for rejected, which is Tender's (#2191 D41), carried from the same entry declaration is, because a file the emulator
read and does not recognise WAS checked while one whose bytes never came back was not, and one withheld verdict cannot
say which. The verdict is the answer alone and carries none of its causes; the verdict decides only which family of
codes can apply and what to say when none of them is recognised, which is the one sentence written off it —
frontend/src/utils/biosFileNote.ts is the one place both surfaces derive what a row says from — a sentence, the lines
under it, which is how a satisfied folder's images arrive as a list rather than folded into the row's own name, and the
description on its own line under the row (biosFileDescription). A row is headed by the file it declares on both
surfaces and never by its description: that is the packager's prose out of a core's .info, outside the resolver's
contract, and it routinely spells the row's own name into its words — so the shared rule takes the name back out and
answers null where nothing is left.
No unknown withdraws a download, because the two questions are independent. What the resolver could establish is the
emulator's DEMAND; what is fetchable is what the RomM library HOLDS, and neither answers the other. A platform nothing
could be read for still has a library behind it, and fetching from it is the one action that moves the platform along at
all — so every download the BIOS table offers (frontend/src/bigpicture/bios/BiosFiles.tsx, on a platform's pane and on
a game's BIOS tab) is built off the fetchable set (on_server && !downloaded && declared_kind !== "directory", plus the
offline server) and reads the verdict nowhere: not bios_level, not required_withheld. The two further inputs
those buttons do read are of the same two kinds — each row's fetch_for_required is its emulator's own demand and is
what Download required counts over the shown emulator's rows, and the library's own finished ratio is what stops
Download all offering a set already complete — so neither is a readiness gate either. fetch_for_required is
domain/firmware_groups.py's fetched_as_required, the one rule FirmwareDownloader.download_required_firmware
applies too: the files the emulator requires, and the options of its one-of groups that serve a region nothing in place
covers yet. A pick nothing could identify — a platform with no emulator listed, or one whose first listed emulator the
resolver names no identity for — is no emulator (#2190 D26): no row is its requirement or one it uses, nothing is
fetched for it, it has no entry in the emulator selector — so the platform page opens on the first other emulator and
offers that one's Download required, and none where there is no other (#2190 D34) — and the header reads that nothing
could be established — the empty required set is an absence of an answer, never "nothing required" (the verdict for no
identity is none, so emulator_read is false and the level unknown). Reading readiness there is what took the buttons
off PS2, GameCube and PSP the moment a BIOS answer was scoped to the emulator that actually launches: those launch
standalone emulators the resolver holds no card for, so the verdict declines over a library that still holds their
files.
What the narrowest decline does decide is wording. "Not read" (emulator_read false, reading_complete_for) means
the emulator stated nothing about what it wants — a read declaration with an empty list is read, and says it needs
nothing — not that it has no verdict. Where the launching emulator was not read at all the summary cannot name which
files to place, so the pane adds the one route it can still state — a file put in the BIOS folder by hand works
regardless — and nothingEstablished has that single consumer. A verdict the resolver left open over an emulator it DID
read is not that state. emulator_read is what separates the two on the wire — and only under a level that is
unknown: a packaged card that names no file reads as not read (adapters/atlas_firmware.py::_stated), and DuckStation
over a folder its search found nothing in is such a card with a verdict of not met, so the panel words the verdict there
and never "Nothing could be established". Neither condition is ever per file — a platform whose reading is complete for
its launching emulator (reading_complete_for) may hold plenty of not_needed files, with no placement in the
platform's catalogue, and trips neither, because "no emulator was found to ask for this" is an answer. Those files stay
fetchable like every other, since fetchability never reads the answer.
A one-of group is one requirement, judged region by region. emu-atlas states a demand a launch meets with exactly
one of several files as a group of options (FirmwareAlternatives), each serving the console regions whose launch opens
it — Beetle PSX's three region images, or the one image SwanStation's search found, serving every region the search
answered for. The adapter carries each group as FirmwareGroup on FirmwareCatalogue.groups, keyed on the emulator
stating it, together with the regions that entry names as unchecked or as booting nothing — which caveats those are, and
which one is not, is adapters/atlas_firmware.py::_UNCHECKED_REGION_CODES's. A region whose every name the resolver
refused arrives with neither an option nor such a caveat: the resolver names it — with every other region of the
emulator's route the group carries no option for — only under unstated on the group's core-mode-unestablished caveat
(reason region-decided-by-disc). The adapter reads that field, and nothing else, into FirmwareGroup.unstated_regions
(_unstated_regions, #2190 D11). Such a region is unchecked:
it keeps a covered group off met and the platform off the amber region rule, a game of that region reads grey, and the
verdict names it as unstated on the wire, unless the resolver also states that nothing boots for it, which makes it
missing. Every option's file is a placement like any other, and an option's need never reaches the row's required
(adapters/atlas_firmware.py::_wants) — the group is the requirement, so marking each option required would report
every option but the one in place as a missing prerequisite over a launch that needs one. domain/firmware_groups.py
judges a group into met / partial / unmet / unknown over its covered, missing and unchecked regions; the
launching emulator's verdicts ride the payload as one_of_groups, each row's membership as one_of, and the counts
take each group once — required_downloaded only where it is met, required_withheld for an unknown one. These
states word the group; the level is the resolver's verdict (above), so an unmet group sits under a missing level
because the resolver answers not met, and a partial one reaches the amber partial only under the region rule. The
game page narrows the groups to the ROM's own console regions (console_regions_of over Rom.regions), and there the
level is domain/bios_status.py::game_region_level's: everything outside the groups decides first — a plain required
file not met, or the console's own image shown absent, is missing; a plain required file not established, a refused
required declaration, or a verdict not met that nothing the resolver states explains is unknown — and then the image
serving the game's region: met ok, not met missing, not established or no option for that region unknown, the last
even under a met verdict, since the resolver leaves a region no option lists outside its verdict. Where no region maps
the platform's verdict stands. The token (compute_bios_label) counts the plain required files alone, never a group:
their ratio where one is not in place, Missing under a not-met verdict where none is short, OK otherwise and
Unknown under an open verdict. Nothing in the module knows which console it is judging.
The console's own need for an image is the resolver's to weigh, and reaches a surface only as words. A libretro
.info marks each file required or optional and can say nothing else: there is no way to say the console does not start
without one of them, so SwanStation marks all five of its PlayStation images optional. The resolver answers that half
from a packaged, source-cited table about the system (CoreFirmware.system_firmware, marked on the answer as
system-firmware-world-knowledge) and weighs it into requirements_met, so the level already carries it: with no image
in place SwanStation's verdict is not met, and the platform reads red. Tender keeps no reading of its own beside that
verdict (#2190 D13). What travels is console_needs_image,
the table's statement under its mark, for the sentence alone: "PlayStation does not start without a BIOS image, and none
of the five SwanStation knows is in place", worded only under a red verdict, with which image is in place read off the
row verdicts and never off presence. A system the table records nothing about (system_firmware: null) is an unasked
question and never "this console needs no firmware", for the same reason unknown is not not_needed one axis over.
Whether the console's image is shown absent is the resolver's too: system_firmware_met, its console-level verdict
folded into requirements_met, carried on CoreFirmwareVerdict beside it. Its false holds for every region, so the
game page reads it as red before any group is narrowed to the game's region (#2190 D14, D20), and never infers the
absence from a group or the rows. An emulator that names no image at all shows what that buys: DuckStation searches its
BIOS folder, and over one holding nothing of a size it accepts the answer states no requirement, a verdict of not met
and system_firmware_met false. Both reads send that answer though no row stands (check_platform_bios, and
get_platform_firmware_status, which answers every platform with its entry), so the platform reads red and the play row
warns. Its search statement names the folder and no count of candidates, and travels as the search-found-nothing note
with candidates: null; the sentence is the console's need without a count, then the search: "PlayStation does not
start without a BIOS image — DuckStation (Standalone) found no BIOS image in bios/".
No BIOS answer outlives the page that asked for it. get_cached_game_detail carries none and says so
(bios_status_unknown), and the live get_bios_status fills it in a moment later; there is deliberately no cached twin
of check_platform_bios for it to read. That is why BiosChecker has one method. The live answer's bios_level and
bios_label are derived once, in _bios_aggregates, where the reading state that decides them is known; every consumer
threads them through rather than re-deriving over the payload.
The BIOS delete's whole input is Tender's own records. PlatformBiosDeleter._delete_io removes two kinds of file
and nothing else. A file of the platform's firmware store goes through FirmwareStore.do_remove, which takes the path
the store record holds, removes every link of Tender's to it, and drops both records. A file Tender downloaded into an
emulator source's folder before the store existed goes by its downloaded_bios row: the row's own file_path is
unlinked, only inside the BIOS folder RetroDECK names now, and the row pruned. Both halves matter. Authority comes from
having placed the file, and the record is the only evidence of that, so a status listing cannot gate it — that gate hid
the button entirely for a platform whose downloads had all left the RomM library. And the record's path is where the
download actually wrote, where a status row's local_path is recomputed from today's placement: for a file fetched
before an emu-atlas bump moved it, the recomputed path names whatever now occupies the new destination. An older
download whose file is already gone is pruned without counting as a deletion or an error, and without a word.
get_platform_firmware_status ships deletable_count — every store file of the platform Tender has a record of that is
on disk, whichever row it fills or none, and the older downloads still on disk, distinct paths — so the button's number
is the same set the delete acts on; local_count is the library's progress ratio and is wrong for it in both
directions.
A delete is never refused because a place cannot be reached (#2191 D20). A link under a root the resolver only
assumed, or under none it could judge, stays where it is and keeps its record, and the link update removes it once the
root is read. An older download stays too where Tender may not remove it now (RetroDeckFolders.bios_folder,
is_inside_folder; D29): while RetroDECK's settings cannot be read, RetroDECK is not installed, or it names no BIOS
folder, the file keeps its record and stays counted, so its row keeps its Delete — nothing but a second press removes
it; outside the BIOS folder RetroDECK names now, the file stays and its record goes, so it is counted no more. The
answer names each under kept with its kind, place, folder and source — an older download also with its file name and
why (settings_unreadable, not_installed, no_bios_folder, outside_bios_folder) — and the status line words each
(D27, D29).
Two buttons reach that one loop, and they differ only in which names they select: the platform-wide
Delete downloads (N) (every record, #2190 D24) and a row's Delete (delete_bios_file, the records naming the row's
delete_name). A declared folder has no Delete of its own: each file of it Tender downloaded is a row of its own (#2191
D26). A name can only narrow that platform's own record set, so no caller can reach a file Tender did not place; a
second copy of the loop is what the register's BIOS-delete rule warns against, because the copies would drift in
silence. The same answer is stamped per ROW as deletable_count, which is what each row's button is offered on — never
downloaded, which is equally true of firmware RetroDECK ships.
A store record is keyed by the platform's folder in RomM and the file NAME
(PRIMARY KEY (platform_folder,
file_name)); an older download record is keyed by the platform's slug and the file name
(#2190 D27). One store file can serve several places, each with a link record of its own keyed by place, so a row's
Delete removes the store file and every link of Tender's to it, whichever emulator's place the row names.
Domain (backend/domain/)¶
Domain modules contain pure logic with no I/O. They take inputs and return outputs; anything stateless and I/O-free that
would otherwise sit in a service lives here. Domain is stdlib + self only — it imports no other internal layer (lib
and models included). Aggregate roots and the enforcement that keeps them honest are documented in
Database Design. Selected modules:
| Module | Role |
|---|---|
sync_action.py |
The SyncAction union (Skip / Upload / Download / Conflict) the whole vertical dispatches on and the compiled core answers in. The decisions themselves are made in the core, not here. See Save File Sync Architecture. |
sync_diff.py |
ROM classification and platform/collection diff computation for the sync preview |
cover_refresh.py |
cover-fingerprint compare kernel (#1386) — scan_cover_refresh_candidates, shared by the apply-path invalidation pass and the preview's cover-work count |
preview_delta.py |
PreviewDelta shape for the sync preview (the staged answer included) plus is_expired / preview_expires_at, the one expression of its TTL |
work_unit.py |
WorkUnit — the per-unit sync work item |
rom_save_sync_state.py |
RomSaveSyncState aggregate + FileSyncState value object — per-ROM save-sync state, backed by rom_save_sync_states + rom_save_files |
save_path.py / save_attribution.py / save_status*.py |
save path resolution, uploader attribution, status DTO building |
save_answer.py |
SaveAnswer / SaveComponent / SaveGroup and the rule that classifies one resolver reading into exactly one of the seven save states. Holds the progress-vs-configuration role rule, the download-name fold, the benign-skip reason the refusing states produce (ADR-0034), and which resolver findings refuse a save place as not read from the emulator's settings |
firmware_platforms.py / bios_status.py / firmware_wants.py |
Which platform a library firmware file belongs to (and where an older download record or per-platform emulator pick kept under another key — a slug or a server folder name — belongs, judged against one read of the listing, ListedPlatforms; and which records a platform's page offers), the status shape, and the resolver's vocabulary. bios_status.py holds the status dataclasses (BiosFileEntry, BiosStatus) and owns the unknown/ok/partial/missing LEVEL (compute_bios_level / compute_bios_label) — the single source of truth all surfaces read; phrasing + color stay UI-layer. firmware_wants.py holds only the words a firmware answer arrives in (FirmwareCatalogue / FirmwarePlacement, the four wanted values) — the decisions are the resolver's, not this layer's |
iso_time.py |
parse_iso / parse_iso_to_epoch — ISO-8601 timestamp parsing (stdlib only) |
achievements.py |
achievement progress computation |
shortcut_data.py |
shortcut data building (registry entries, shortcut dicts) |
steam_categories.py |
Steam collection name computation |
sgdb_artwork.py |
SGDB asset-type/endpoint maps and their queries, the image choice by stated size (choose_candidate), the artwork cache's file names, and to_signed_app_id |
installed_roms.py / rom_files.py |
installed-ROM detection, M3U generation, launch-file detection |
state_migrations.py |
unreadable_settings (which settings.json files are read) + migrate_settings (the version ladder) |
sync_state.py |
SyncState enum (idle, running, cancelling) |
emulator_tag.py / version.py |
emulator-tag formatting, version parsing, core-change detection |
identity.py |
DISPLAY_NAME — Tender's name where a person reads it: the RomM token label, the registered-device client, and the headline of each of the two files opened by hand (a recovery bundle's README and the recovery root's). The identifier romm-tender is deliberately not here and not in one place; which name a new string takes is GLOSSARY.md's "Display name vs identifier" entry; why its four homes stay apart is this module's docstring |
tender_folder.py |
Tender's folder: the stored place and its default, a typed place, the ROM-folder rule a place is judged by, when the folder counts as unreachable (only with something recorded under it), and the sentences its refusals are worded with |
Config-source parsers follow a dedicated domain+adapter template (pure parse in domain, I/O in adapter, callback Protocol into services). The full pattern, source catalog, and decisions log are on the Config Source Parsers page.
Models (backend/models/)¶
TypedDicts and dataclasses describing on-disk and in-flight data shapes (state.py, sync.py). Models import nothing
from the other layers.
Other¶
| File | Role |
|---|---|
main.py |
run() and build_backend(), the process entry, and Endpoints (one method marked @route per endpoint) |
check.py |
check(), the pre-install check's entry: builds the Application on copies of the live data and runs none of it |
bootstrap/ |
Composition root — build_application() composes bootstrap() and wire_services() into an Application |
lib/errors.py |
Exception hierarchy (RommApiError, Refused, classify_error) |
lib/background_tasks.py |
BackgroundTasks: the only caller of create_task outside host/ — spawn for a task nobody awaits (a failure logged as a warning under its name), start_awaited for one its starter awaits itself; each held until it ends |
lib/list_result.py |
ErrorCode and the canonical endpoint failure shape |
Where user data lives¶
Where this program's directories are is resolved once from the environment and handed in — it is never derived here
(ADR-0036). domain/app_directories.py is the ladder, pure, with the
environment passed as an argument so every rung is checkable against a table:
TENDER_CONFIG_DIR,TENDER_DATA_DIR,TENDER_CACHE_DIR,TENDER_STATE_DIR,TENDER_CODE_DIR— what the installer resolved once and wrote into the service unit. This is the rung a service runs on. The installer and the service do not share an environment: the installer runs in a login shell, the service in the user manager's, and anXDG_DATA_HOMEset in a shell profile never reaches the latter.XDG_CONFIG_HOME,XDG_DATA_HOME,XDG_CACHE_HOME,XDG_STATE_HOME,XDG_RUNTIME_DIR.- The built-in defaults below.
Rungs 2 and 3 exist for a start by hand. A variable set to the empty string counts as unset — an empty path would resolve to whatever directory the process happened to be started in.
| Root | Holds |
|---|---|
~/.config/romm-tender/ |
settings.json, plus its .tmp, .lock and .corrupt-<ts> siblings |
~/.local/share/romm-tender/ |
romm-tender.db, the single-instance lock; the installer's update-backup/ and rollback-backup/ |
~/.cache/romm-tender/ |
covers/, artwork/ — everything re-derivable from the server; update/, what an install from the panel staged, removed at every start |
~/.local/state/romm-tender/ |
the log file; the installer's update-failure.json |
$XDG_RUNTIME_DIR/romm-tender/ |
the port file |
The data/cache split is not filing tidiness. What lies under the cache root is re-derivable — covers and artwork are
fetched again if they are gone — and what lies under the data root is not: the database is the only copy of what the
user has installed, synced and chosen. A system that clears caches must be able to clear one and not the other. XDG
names no default for the runtime directory, so a start that has none falls back to the state directory; the port file is
a hint and connecting to it is the proof, so a stale one misleads nobody who checks. The entries marked as the
installer's are written by install.sh when it updates, rolls back or refuses a new version, never by the backend; what
they hold is in Running an installed one. The backend reads
one of them, update-failure.json (UpdateOutcomeService notes), and neither writes nor
removes it.
One place reads them. WiringConfig.directories is the only path any service reaches through, and RuntimeBundle
carries no directory at all. It used to carry two — the plugin folder and the loader's runtime directory — which is how
a question about a plugin loader's own layout came to sit beside a question about the user's data.
There is no start-up migration. Earlier releases copied a user's data forward from a directory a plugin loader had named, because that name followed the packaging and moved every user's data at 0.31.0. Nothing derives a directory from a folder name any more, so the question has no asker: the ladder above is the whole answer, and 1.0 is a breaking change that is installed rather than migrated into.
The launcher's home¶
<bin root>/tender-rom-launcher — ~/.local/bin/tender-rom-launcher by default — is the file every Steam shortcut's
exe names, and bootstrap() puts this release's copy there on every start
(ADR-0038). adapters/launcher_install.py owns the write;
domain/user_data_location.py owns where it goes, through two composers over one tuple — launcher_in_bin_dir for the
installed copy and launcher_path for the one the release ships beside the program — so the components that make up
/bin/tender-rom-launcher have one spelling, and the suffix ownership detection matches is derived from that same
tuple.
The bin root is the one directory on AppDirectories not named after this program: it is shared with every other
program the user installed for themselves, which is why the install creates it at the umask's mode rather than
owner-only, and why an uninstaller leaves it alone. The reason the launcher is there rather than beside the code is that
an update replaces the code directory, and a shortcut's exe must not name a file inside something that gets replaced.
The reason it is not under the data root is that the data root holds the only copy of the user's library and nothing
executable. The reason it is written on every start rather than once is that a launcher installed once would freeze at
whatever version the day of the move brought. The reason it is written through a staging file that is renamed on — never
in place — is that a game running right now is executing that file, and bash reads a script as it runs it.
It is unconditional. There was once an ordering to respect here — the install had to wait for a start-up migration's
data half to land, because writing into an empty data root would have settled that migration's first rung for the life
of the install. Nothing migrates now and the bin root is simply where this run was told it is, so the install runs on
every start and the only question left is whether the write succeeded. A start whose write failed creates nothing, and
ShortcutLauncher.path is then the copy that ships beside the program: a real file, so a sync in that state still
produces shortcuts that launch.
ShortcutLauncher carries the two answers apart on purpose. path is what a newly built shortcut names, and follows
the INSTALL rather than the intent: the home where this start actually got the launcher into it, the shipped copy
otherwise — including the start whose write failed, whose home is empty. The one case where even the shipped copy is not
a real file is a package shipped without its launcher, which is the same reason the install failed. at_home is the
narrower question — path is the home in the bin root (launcher_in_bin_dir(directories.bin_dir)), with this release's
launcher in it — and it is what repointing an EXISTING shortcut turns on: pointing one at a launcher nothing put there
stops its game from starting, and no part of Tender could put it back.
Repointing the shortcuts that already exist¶
services/shortcut_relocation.py answers which of Steam's non-Steam shortcuts are OURS and are not at the launcher's
home. Ownership is one ending, /bin/tender-rom-launcher, and nothing else — a shortcut naming any other launcher,
including one an earlier version of this program wrote, is foreign here and is left where it is. It reads them out of
shortcuts.vdf through SteamConfigStore.read_shortcut_exes — one parse for all of them, where the frontend's own
route to the same fact is a RegisterForAppDetails per shortcut — and hands the frontend a list of app IDs plus the
exe and start_dir to write. Two shapes in that file are matched carefully because both fail quietly: the keys
case-insensitively (Steam has written more than one case), and the app id converted out of the signed int32 form the
file stores, since every SteamClient API takes the unsigned one.
It is a one-time transition with a recorded completion (kv_config, shortcut_launcher_relocated), and the reading is
its only writer: a call that finds nothing of ours outside the launcher's home stamps it, and no later start reads the
file again. The frontend writes and reports; it records nothing, so a completed rewrite is stamped on the FOLLOWING
start — Steam writes its in-memory shortcuts to the file when it chooses, and the file is what the stamp rests on. Every
uncertainty answers blocked instead — the launcher is not at its home, or the file could not be read — and a blocked
answer is never stamped, so the next start asks again. The gap that leaves is named at get_shortcut_relocation:
nothing clears the stamp, so a shortcut of ours that turns up later away from the home stays there while the backend
reports done. The card that used to read the stamp is gone with the plugin loader; the stamp itself still decides
whether a later start re-reads Steam's file at all.
Composition Root (bootstrap/)¶
The composition root is a package of two halves, one per phase, the library service's wiring (library.py, under item
2), the Application that composes them, the wrapper each start-up repair runs through (startup.py), the copy of the
live data a pre-install check builds on (check.py), and an __init__.py that is namespace and re-exports only —
consumers write from bootstrap import … and never deep-import a submodule:
-
adapters.py— ownsbootstrap(), which is told where the directories are, and where releases are asked for, rather than deriving them, then builds every adapter, applies the SQLite schema migrations, and loads + migratessettings.jsonso the settings persister binds the live mutablesettingsdict at construction. Returns a typedBootstrapResultcarrying four bundles (adapters,stores,callbacks,runtime_adapters),focus_watch— the focus watcher's three readers, which reach it only where the entry point switches it on —directories— theAppDirectoriesthis run was handed, seven fields —launcher, where the shortcut launcher lives in the bin root and whether this start got it there, anduser_agent,<package name>/<version>composed once from the constants indomain/identity.py: the outgoing User-Agent, which is also the identity the host answers under. The bundle dataclasses are defined here too — they are the vocabulary the second half consumes. -
services.py— ownsWiringConfigandwire_services(), which takes the four bundles plusmin_required_version,directories,launcher,update_sourceandfocus_watcher(the focus watcher's seams,Nonefor off), and constructs every service, injecting each one's*ServiceConfig. Returns a frozenServicesBundleholding every service and the prune conflicts as typed fields. The library service's wiring is a module of its own,library.py:wire_library()takes aLibraryWiring(the adapters, runtime seams and peer services the parts are built from), builds the state box and every part, and answers them with the service asLibraryParts;wire_services()keeps the service, and the library tests callwire_library()with fakes. -
application.py— ownsApplicationandbuild_application().build_application()is synchronous: it callsbootstrap(), composes the runtime bundle from the loop, the event sink's emit and the host's reading of Steam the entry point hands it, builds the focus watcher's seams where the entry point hands it the switch on and the host's way of handing Steam a URL (SteamUrlControl), passesMIN_ROMM_VERSIONfromdomain/identity.pyasmin_required_version, callswire_services(), and answers anApplicationholding theServicesBundleas one field (services). It runs nothing. TheApplicationruns the start-up repairs (run_startup_repairs, which also starts the four background tasks it holds — the save-directory backfill, the running release check, the shortcut artwork job's wait for a panel and the focus watcher) andshutdown()— each when the entry point calls it. Its constructor takes a readyServicesBundle, which is the seam the contract harness uses: it swaps adapters betweenbootstrap()andwire_services()and constructs theApplicationitself.
The two-phase split exists because adapter instantiation and state loading happen first (bootstrap()), then
build_application() composes the runtime bundle (event loop, the event sink's emit, the host's reading of Steam) and
calls wire_services(). Services receive the settings dict (the only field on StateBundle) plus the SQLite
Unit-of-Work factory / repository handles for all relational state — no plural in-memory state dicts remain. Some
services are constructed before others to satisfy ordering constraints (e.g. MigrationService before SaveService so
save sync can gate on a pending RetroDECK home migration). Forward references between peers are threaded via
LateBinding.
bootstrap() also logs, at info rather than behind the debug toggle, which interpreter the vendored resolver would run
its core probe under: describe_core_probe_interpreter() (adapters/atlas_saves.py). The resolver derives it from the
running program — sys.executable is a real CPython here, the system one the service unit starts, or the venv's under
mise run dev — and nothing registers one over it, because a registered path could only be a guess about the machine.
Where it derives none it spawns nothing and answers unknown for every core it is asked about; the one question this
program puts that reaches the probe is a libretro entry's save answer, which then usually establishes nothing at all.
Nothing fails when that happens, which is why the answer is logged: the caveats the loss leaves (core-unqueryable,
core-generation-unestablished) do reach the debug log and the wire, but that line is the only place their cause is
named.
It also hands the resolver a zstd codec where the interpreter has none of its own: register_zstd_codec()
(adapters/atlas_zstd.py), once, and logs which provider a zstd image would go through. The codec has one reader —
EmuDeck's ES-DE catalogue, sealed inside its AppImage, which without one answers emulator-catalogue-sealed. From
Python 3.14 the standard library's compression.zstd is that codec and the resolver finds it by itself, so nothing is
registered; on 3.12 and 3.13 the backports.zstd copy vendored under _vendor/backports/ is registered, since the
resolver never tries that name. A registered provider is tried before the standard library, and the copy carries only
the cp312 and cp313 builds of its compiled extension, so it does not load from 3.14 on, which is why the
registration is skipped wherever compression.zstd imports. A copy that does not load — upstream refusing the
interpreter, or this machine's loader refusing the compiled extension — is no codec, never a failed start: the catalogue
stays sealed, and the line says why.
Per the process-boundary rule, adapter instantiation never happens in main.py, and no service wiring happens outside
bootstrap/. Every module in bootstrap/ is governed by the ~1000-LOC decomposition threshold
(scripts/check_module_size.py), and none is grandfathered.
Protocol Interfaces¶
Services depend on Protocols, never on concrete adapter implementations. The Protocols live in the services/protocols/
package, organised topically (consumers always deep-import from services.protocols import X):
transport— external system clients:RommApi(and its narrowed facetsRommSaveApi,RommRomReader,RommDeviceApi,RommFirmwareApi,RommPlaytimeApi,RommLibraryApi,RommConnectionApi,RommPlatformReader,RommAchievementsApi,RommSyncApi,RommVersion),SteamConfigStore,SteamGridDbApi.determinism—Clock/UuidGen/Sleepertest seams.persistence—SettingsPersister.paths—RetroDeckFolders,PlatformSystems,SourcePlatformSystems,CoreInfoProvider,SaveLocationReader(a game's save answer and its savestate directory),CoreResolverFn,PlatformCoreReader.infra— cross-cutting callable seams:EventEmitter,DebugLogger,PathExistsReader,ResolvedPathFn,HostnameReader,PendingSyncReader,DownloadQueueCleanup.files— filesystem seams:CoverArtFileStore,DownloadFileStore,FirmwareFileStore,MigrationFileStore,RomFileStore,SaveFileStore,SgdbArtworkCache.cross_service— narrowly-typed multi-method seams one service exposes to another so services stay independent:BiosChecker,AchievementsReader,ArtworkManager,ArtworkRemover,RetryStrategy,MigrationPendingFn,DeviceForgetFn,DeviceRegisterFn(device registration after a sign-in,SaveService.ensure_device_registered→ ConnectionService),DeviceIdProvider(server device id,SaveService.get_device_id→ PlaytimeService),PlaytimeScopeNoticeClearFn(PlaytimeService clears its re-sign-in notice on a fresh sign-in from ConnectionService), and theSession*seams.repositories— one repository Protocol per aggregate root, plusKvConfigRepositoryfor thekv_configkey-value surface.uow—UnitOfWork, which exposes those repositories as typed properties, andUnitOfWorkFactory, the call-shaped seam a service holds to open one.
Protocol names carry a suffix that signals shape (…Reader, …Provider/…Fn, …Store, …Cache, …Persister; bare
names for pervasive primitives like Clock).
RommApiAdapter implements RommApi over RommHttpAdapter.
Boundary Enforcement¶
The CI-gated checks below keep the dependency direction and the call-site rules from drifting. Aggregate-specific
enforcement (the @cosmic_aggregate decorator and the field-assignment check) is documented in
Database Design.
1. import-linter (CI-enforced)¶
.importlinter declares the layer contracts:
# Services must not import concrete adapter implementations (Protocols OK)
[importlinter:contract:no-adapter-impl-in-services]
type = forbidden
source_modules = services
forbidden_modules = adapters
# Adapters must not import services
[importlinter:contract:no-services-in-adapters]
type = forbidden
source_modules = adapters
forbidden_modules = services
# Utilities (lib/) must not import services, adapters, or domain
[importlinter:contract:utilities-independence]
type = forbidden
source_modules = lib
forbidden_modules = services, adapters, domain
# Domain is pure compute — no dependency on any other internal layer
[importlinter:contract:domain-independence]
type = forbidden
source_modules = domain
forbidden_modules = services, adapters, lib, models
# Domain is stdlib + self only — no vendored third-party packages
[importlinter:contract:domain-stdlib-only]
type = forbidden
source_modules = domain
forbidden_modules = _vendor
# Models must not import services, adapters, domain, or lib
[importlinter:contract:models-independence]
type = forbidden
source_modules = models
forbidden_modules = services, adapters, domain, lib
# The host is the process, not a layer: it reaches none of the code it hosts
[importlinter:contract:host-hosts-nothing-it-knows]
type = forbidden
source_modules = host
forbidden_modules = services, adapters, bootstrap, domain
# ...and only the entry point may reach the host. main.py is not a package,
# so nothing in this list can name it.
[importlinter:contract:nobody-imports-the-host]
type = forbidden
source_modules = services, adapters, bootstrap, domain, lib, models
forbidden_modules = host
# Services must not import stdlib I/O / non-deterministic primitives directly
[importlinter:contract:no-stdlib-io-in-services]
type = forbidden
source_modules = services
forbidden_modules = random, subprocess, threading, requests, time, uuid
# Services must be independent of each other
[importlinter:contract:service-independence]
type = independence
modules = services.library, services.saves, services.playtime, ...
Run with PYTHONPATH=backend lint-imports (or mise run lint). CI gates on this.
The service-independence modules list is hand-enumerated, so scripts/check_service_independence_contract.py
(bundled into mise run lint and gated in CI) derives the expected services from backend/services/ and fails if the
contract omits a service or carries a stale entry — keeping the list self-healing rather than silently rotting.
2. Cosmic Python call bans¶
scripts/check_cosmic_call_bans.sh (also bundled into mise run lint) complements the import-level guardrail at the
call site: services may not call datetime.now() / asyncio.sleep() / time.time() / time.monotonic() /
uuid.uuid4() / random.* directly — they inject the corresponding Clock / Sleeper / UuidGen Protocol instead.
3. Aggregate field-assignment check¶
scripts/check_aggregate_field_assignment.py (also bundled into mise run lint) is a small custom AST linter that
enforces the mutation-only-via-methods rule for aggregates — a rule no type checker can express directly. A trailing
# pragma: no aggregate-check exempts a value that only shares an aggregate's field name. What it flags and how:
Database Design.
4. Failure-shape dialect gate¶
scripts/check_failure_shape.py --check (also bundled into mise run lint) is a small custom AST linter that holds the
canonical failure shape to main.py: the wrapper around every endpoint builds {success: False, reason, message}
from a raised refusal, a returned PartialFailure or a raised RommApiError ("A refusal can be raised", above), and
every other module under backend/ but the upstream copies in _vendor/ and native/ fails the gate if it builds a
dict literal with a literal False, 0 or None success, calls error_response(...) or spreads a refusal helper's
answer — event payloads included — or does not parse. A result with a success field of its own that never reaches the
wire: a failed one is built through its type (MutationOutcome(success=False, …)) and passes. Two results compute a
failing success outside main.py, and the gate does not see them: the save-sync results — a completed sync with
per-file errors answers "success": len(errors) == 0 (#2205 D10), and Sync All answers false too where it passed a
game over for a missing saves folder (#2151 D4) — and the removed-game cleanup's prune_complete event; no third one is
added. The two documented carve-outs (discriminated-status unions — a status key with no success; and
partial-success payloads carrying an additive server_query_failed / recommended_action flag) carry no success key,
so the gate passes them. The routing slugs come from lib.list_result.ErrorCode (the Lean enum) plus bespoke
plain-string reasons for non-server-reachability guards, raised as the lib/list_result.py module docstring says. The
script's docstring names what it cannot see, a computed success first.
5. Background-task gate¶
scripts/check_background_tasks.py (also bundled into mise run lint) fails on a call named create_task or
ensure_future, on asyncio.Task(...) and on a bare Task(...) imported from asyncio, anywhere under backend/ but
lib/background_tasks.py, host/, _vendor/ and native/. The rule and why it exists are the invariant register's
entry (invariants); what the shutdown does with the tasks is the shutdown paragraph
above; what the check does not see is listed in its docstring.
6. Resolver-reading gate¶
Each kind of answer the resolver gives — firmware, saves, the catalogue, the folders and the sources — is read in one
module, the adapter that asks for it, which turns it into Tender's own view; every later layer, the panel included,
reads that view. This is the adapter of ports and adapters as Cosmic Python has it (chapters 2 and 3): it translates the
outside model, and nothing past it judges a fact the resolver states. scripts/check_resolver_reading.py (also bundled
into mise run lint) fails on a backend module outside those adapters that imports the resolver, on an import of a name
atlas.__all__ does not export, and, outside the adapters, on a string equal to one of the resolver's codes or to a
distinctive value of any other closed vocabulary it exports, or a read of one of its distinctive answer field names. The
panel is held by the ESLint rule tender/no-resolver-vocabulary over the same codes, values and names, which the script
writes to frontend/resolver-vocabulary.json, less a few values the panel speaks with a meaning of its own.
tests/adapters/test_resolver_codes.py fails at an update of the vendored resolver on a code it adds that no adapter
classifies. The rule and its lists are the invariant register's (invariants), and what the
script does not see is listed in its docstring.
7. Enforced: underscore prefix¶
All internal methods use a _ prefix; public ones — the endpoints on Endpoints and the use cases on a service — have
none. The endpoints in main.py delegate directly to the corresponding service method. An endpoint is reachable because
it is marked @route, not because it is async def; .claude/rules/endpoints.md owns when one is a def.
This is no longer just a convention — basedpyright enforces it with reportPrivateUsage = "error", so accessing a
_-prefixed name from outside its owning class is a hard type error. Tests are exempt via an executionEnvironments
override (white-box testing — inspecting and rebinding a system-under-test's private state — is an accepted pattern).
One corollary: a method one sub-service calls on a peer is part of that peer's public surface and carries no
underscore, which keeps reportPrivateUsage coherent with the saves-style peer-injection carve-out.
Service Dependency Summary¶
Every service receives its dependencies through a single *ServiceConfig dataclass. Cross-service dependencies are
Protocol-typed (services never import each other's concrete classes). Selected wiring:
| Service | Key injected dependencies |
|---|---|
| LibraryService | the parts it calls (LibrarySyncStateBox, LibraryFetcher, SyncProgress, SessionBudgetMonitor, SyncReporter, LibraryViews, SyncPreview, SyncOrchestrator), ConflictRules (the rules its use cases check at their entry); its parts, built by bootstrap/library.py, take RommLibraryApi, SteamConfigStore, ArtworkManager, Clock/UuidGen/Sleeper, SettingsPersister, UnitOfWorkFactory (roms / sync_runs / kv_config / rom_metadata), ConflictRules (the leases sync_complete and sync_stale carry), ShortcutArtworkJob (the placeholder icon's path, and a run requested at each sync's end) |
| MetadataService | UnitOfWorkFactory (reads rom_metadata / roms) |
| SaveService | RommApi, RetryStrategy, SaveFileStore, UnitOfWorkFactory (rom_save_sync_states / rom_save_files / answered_save_directories), Clock, RetroDeckFolders, SaveLocationReader, PlatformSystems, ActiveCoreReader, UpdateInProgressFn and MigrationPendingFn (the sync engine's own update and RetroDECK migration backstops), ConflictRules (the rules its use cases check at their entry), RunningRomIdsFn (the games a save path leaves alone while they run) |
| DownloadService | its parts (DownloadsStateBox, DownloadQueue, DownloadTransfer, DownloadUnpacker), RommApi, DownloadFileStore, RetroDeckFolders, PlatformSystems, UnitOfWorkFactory, DownloadTargetGateFn + RomRemoverProvider cross-service seams (the queue takes Clock; the unpacker takes RomInstallRecorder), ConflictRules (the rules a start and a resume check at their entry and the operation a download's task holds) |
| FirmwareService | RommApi, FirmwareFileStore, FirmwarePlatformResolver, FirmwareResolver, CoreInfoProvider, RetroDeckFolders, UnitOfWorkFactory (firmware_cache, downloaded_bios), the ListingRefiler each fresh read of RomM's platform listing is handed to, ConflictRules (the rule each download and delete checks at its entry) |
| SteamGridService | SteamGridDbApi, RommApi, SteamConfigStore, SgdbArtworkCache, Clock (an artwork slot's outcome, a keep-current check's time), UnitOfWorkFactory (sgdb_id, chosen_sgdb_id and artwork_checked_at on roms, artwork_slot_outcomes, artwork_slot_choices, artwork_slot_handovers), PendingSyncReader, CoverSlotReader (the cover's state and RomM's cover reload, from ArtworkService), ConflictRules (the rule the artwork fetch, the slot fetch and its report, the icon save, the resolution, the manual pick, the ARTWORK tab's five endpoints and keep_artwork_current check at their entry, and the lease an artwork, slot or tab answer carries) |
| ShortcutArtworkService | SteamConfigStore (the worklist, the grid directory's listing and Tender's two icons), ShortcutArtworkSource (the SteamGridDB service's icon and artwork half), UnitOfWorkFactory (bound shortcuts on roms, artwork_slot_outcomes), ConflictRules + EventEmitter (each shortcut_artwork batch under its lease), Sleeper/Clock, one WorkInFlightFn each for a sync, a removed-game cleanup and a shortcut removal, PanelConnectedFn, RendererAboveCeilingProvider, TenderLogoFn |
| MigrationService | MigrationFileStore, RetroDeckFolders, FirmwareResolver (which files in a pending home are firmware), SaveDirectoriesRecorderProvider (records the answered save directories afresh once a home migration has moved the files), ConflictRules (the rule the migration checks at its entry) |
| GameDetailService | BiosChecker, AchievementsReader (cross-service), Clock, UnitOfWorkFactory (one read UoW over roms / rom_installs / rom_save_sync_states / rom_metadata / kv_config), plus PathExistsReader + RetroDeckFolders + PlatformSystems for the single target-path stat |
| AchievementsService | RommAchievementsApi, Clock, DebugLogger, UnitOfWorkFactory (reads ra_id from roms) |
| SettingsService | SteamConfigStore, SettingsPersister, UnitOfWorkFactory (reads bound shortcut_app_ids from roms), ConflictRules (the rule the server-URL, custom-header and Steam Input writes check at their entry) |
| PlaytimeService | RommPlaytimeApi, RetryStrategy, DeviceIdProvider (server device id, satisfied by SaveService.get_device_id), Clock, UnitOfWorkFactory (reads/writes rom_playtime + rom_playtime_sessions, and the kv_config scope-notice flag). Exposes clear_scope_notice to ConnectionService (via PlaytimeScopeNoticeClearFn) so a fresh sign-in drops the re-sign-in notice, ConflictRules (the rules its use cases check at their entry, and the operation a session start's outbox flush holds) |
| RomAdoptionService | RommRomReader, DownloadFileStore (the stat, the top-level listing, the directory scan, the hashing), AdoptionMoveStore (the rename across the roms/saves/states trees), RetroDeckFolders, PlatformSystems, SystemM3uSupportFn, SystemSupportedExtensionsFn, SaveLocationReader (the save and the savestate directory of both launch paths of a rename, asked of the emulator ActiveCoreReader names), ActiveCoreReader, RomInstallRecorder peer, EventEmitter + Clock (throttled verify_progress frames), ConflictRules (the rules an adoption checks at its entry) |
| RomInstallRecorder | Clock, UnitOfWorkFactory (rom_installs upsert + the roms size write) |
| RomRemovalService | RomFileStore, RetroDeckFolders, DownloadQueueCleanup peer, UnitOfWorkFactory (reads/deletes rom_installs), Clock + EventEmitter (removal duration logging and uninstall_progress frames), ConflictRules (the rules its use cases check at their entry) |
| ShortcutRemovalService | SteamConfigStore, ArtworkRemover peer, UnitOfWorkFactory (unbinds via roms, offline name via kv_config), ConflictRules (the rules its use cases check at their entry, and the removal lease), ShortcutArtworkJob (stopped while a removal is under way, and asked to run at its end) |
| SessionLifecycleService | Session* cross-service seams (playtime / post-exit sync / achievement sync / migration reader), UpdateInProgressFn (which holds the post-exit sync off), ConflictRules (the rule the finalize checks at its entry) |
| GameProcessService | GameProcessControl (the shortcut's process tree + signals), UnitOfWorkFactory (the ROM's shortcut appId, on roms), Sleeper (the grace window — the service owns no clock) |
| ControllerFocusService | RunningSteamReader (which Steam runs, and how), FocusedWindowReader (the X connection), SteamLaunchReader (a window's process to its Steam launch), SteamUrlControl (the host's way of handing Steam a URL) — all four absent where the watcher is switched off — the loop (the X socket is watched with add_reader), Sleeper (the pace Steam is read at) |
| ConnectionService | RommConnectionApi, SettingsPersister, min_required_version, DeviceForgetFn (cross-service — forgets the device id on an origin change), DeviceRegisterFn (cross-service — registers the device after every successful sign-in), PlaytimeScopeNoticeClearFn (cross-service — clears the playtime re-sign-in notice on a fresh sign-in), ConflictRules (the rule the connection test, the sign-ins and the sign-out check at their entry) |
| VersionSwitchService | RommRomReader (live sibling_roms view + per-sibling detail), Clock, UnitOfWorkFactory (resolves a sibling group via iter_by_group_key and moves its binding on roms), SaveDriftProbeFn + ReachabilityProbeFn (the switch-away save-stranding soft block), ActiveDownloadRomIdsFn (refuse a switch while a group member is downloading), ConflictRules (the rules a switch checks at its entry, and its lease) |
| PruneLeaseService | ConflictRules (renews, releases and disowns the frontend's leases; checks no rule) |
| RunningGames | SteamInterfaceReader (the host's reading of the running apps), UnitOfWorkFactory (which ROM a listed app id is the shortcut of, on roms) |
| UpdateCheckService | LatestReleaseFn, Clock, Sleeper + EventEmitter (the check while the backend runs), SettingsPersister, UnitOfWorkFactory (kv_config update_check_last_seen, update_available_toasted_version and update_available_seen_version), the running VERSION, and whether this process is the installed program |
| UpdateOutcomeService | UpdateFailureFn, SettingsPersister, UnitOfWorkFactory (kv_config last_run_version and update_failure_toasted_at), the running VERSION |
| UpdateInstallService | LastSeenReleaseReader (the release check), SteamInterfaceReader (running apps and the reload limit, from the host), one WorkInFlightFn per kind of work a restart cuts short plus ActiveDownloadRomIdsFn and DownloadQueueFn, HeldClaimsFn (the prune conflicts), UpdateFailureFn, UpdateAttemptStore, ReleaseAssetDownloadFn, UpdateStagingStore, TransientUnitControl, UnitOfWorkFactory (kv_config update_stopped_toasted_at), FailureToastAcknowledgeFn (UpdateOutcomeService.acknowledge_update_failure_toast), Clock, Sleeper, EventEmitter, the installer's environment, the running VERSION, and whether this process is the installed program |
| UpdateOutputService | JournalEntriesFn, UpdateFailureFn, FailedInstallerStartFn (UpdateInstallService.failed_installer_started_at), the running VERSION |
Most services also receive the settings dict (StateBundle's only field), the runtime infrastructure (event loop,
logger, the DebugLogger Protocol), and the UnitOfWorkFactory for relational state through their config. The old
in-memory state / metadata_cache / save_sync_state / shortcut_registry dicts are gone — every relational
read/write goes through the Unit of Work.