✨ EMS-AUDIT: imports' checksums, deleted events in account exports, admin keys after commit, new tenants' realms (part 2)

[ereslibre]

Parent issue: https://github.com/sequentech/meta/issues/13781

Design package EMS-AUDIT, second part. #3902 gave each tenant a signed tenant log; this pull request closes the gaps it listed or left open, apart from the route-by-route entries (see "Split with #3928" below).

What changes

Area	Change
Imports	Importing an election event or a tenant configuration records the file's SHA-256, computed by the task from the file it read, and compares it with the checksum the admin declared (undeclared, matches or mismatch). The entry also counts what the file held: elections, contests, candidates, areas and areas per contest, or configuration sections. A failed commit of a tenant configuration import no longer records the import.
Entry popup	A new The imported file section shows the checksum verdict, the SHA-256, the counts and the document, instead of raw extra details. The deleted election event section names The log's system key. Translated into the portal's 8 languages.
Deleted events in exports	One account's export with "every election event" now also reads the logs of deleted events from their retired databases, merged by time; their rows name retired event <ID>. Each retired database is checked against the catalog, opened read-only under the event database lock, and closed to connections again, including when the read fails. The deletion entry keeps the retired log's system public key, since the event's keys are deleted with it.
Admin public keys	AdminPublicKey is posted to the tenant log only once the transaction that saved the key commits. A rollback leaves nothing behind (before, it could leave a public key in the log whose key was then lost). The delivery id is per account and key, so the key is posted once. The tenant log migration backfills keys created before keys went to the tenant log, and running it again does nothing. The outbox can't carry this statement (it only posts Audit bodies), so a small post-commit task does it instead.
New tenants	Creating a tenant prepares its Keycloak realm for the tenant log whatever the realm template: custom-event-listener, admin events with details, and the tenant-logs-read / tenant-logs-export roles granted to the admin group.
Monitoring	Every audit outbox worker pass writes {"target":"audit_log_outbox","pending_entries":N,"oldest_pending_seconds":S}, zeros included, so security monitoring can alert on entries waiting more than 5 minutes. This replaces the waiting/oldest_seconds names. It is documented in the developer page.
Permissions	/export-tenant-logs needs tenant-logs-read as well as tenant-logs-export (403 otherwise). The portal shows the tenant log's Export only to people who have both.
Fix	Tenant log exports could not be downloaded: fetch-document built a private document's S3 key from the event the caller named. The tenant log asks with an empty event, so the link pointed at tenant-…/event-/document-… (NoSuchKey). The key now comes from the document's own event.
Docs	Settings > Logs reference: the popup's new sections, deleted events in account exports, the export permissions. Developer page: the backlog line and its alert threshold. Auditor tutorial: retired logs, which have no system key in the manifest.

Split with #3928

#3928 (VOTE-AUDIT, meta#14259) closes the "routes write no AdminChange" gap from #3902 for every Harvest route, including messaging accounts, certificate authorities, countries and deleting a tenant. It does this by letting the audit_change() trigger see Step's own writes. To avoid logging each change twice, this pull request adds no per-route entries. The two pull requests touch different files.

Acceptance (meta#13781)

Ticked where #3902 (merged) and this pull request together meet the criterion. Items 2, 3 and 5 also need #3928.

• Signing in to the Admin Portal, failing (wrong password, wrong authenticator code, unknown account), being locked out, setting up or removing an authenticator, resetting or changing a password and signing out each write one KeycloakUserEvent entry to the tenant log, with the client, IP address and authentication used. An unknown username is recorded as unknown. Token refreshes and code exchanges are not logged. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; this PR adds the listener to new tenants' realms)
• Every tenant-level change writes one AdminChange entry to the tenant log with the person, the object, the action and each changed field's old and new value, and no secret values. This covers the Settings tabs, the tenant record, election types, trustees, templates, staff accounts, roles and permissions, countries, backup and restore, and creating, importing, archiving, unarchiving and deleting election events. (Imports now record the file's checksum and counts here. Messaging accounts, countries with old values and deleting a tenant are in ✨ Every admin change is logged, Harvest's and Windmill's too (VOTE-AUDIT, SW-F-0364) #3928.)
• Event-level create, update, delete, import, configure, approve and schedule operations write AdminChange entries with old and new values to their event's log (baseline). (Changes made through Hasura are covered since ✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; the Harvest routes are in ✨ Every admin change is logged, Harvest's and Windmill's too (VOTE-AUDIT, SW-F-0364) #3928.)
• A change made through Hasura to any table an admin role can write is logged in the same transaction. The coverage test fails when such a table has no trigger. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902)
• A change whose transaction rolls back leaves no entry. A committed change always gets exactly one entry, even when immudb or the worker is down for a while: its entries wait and are posted in order afterwards. (The outbox does this, and this PR moves AdminPublicKey to after commit. Certificate-authority routes still post their own entry after commit, on a best-effort basis; with ✨ Every admin change is logged, Harvest's and Windmill's too (VOTE-AUDIT, SW-F-0364) #3928 they also get an AdminChange through the outbox.)
• A change to the tenant realm made in Keycloak's console is logged as KeycloakAdminEvent, with the operator and the new value. Changes Step makes through its service account appear as SYSTEM entries next to the admin's AdminChange. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; new tenants' realms have admin events with details since this PR)
• Windmill puts an event without an election event in the tenant log. An event it can't place becomes an ERROR entry and the rest of its batch is still posted. An unreadable message goes to the dead-letter queue, and later messages are processed. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902)
• Deleting an election event writes its tenant log entry, with the event log's last entry and state hash, before anything is removed. The event's log database is unloaded, not deleted. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; the entry also keeps the retired log's system key since this PR. Checked on the local stack.)
• Settings shows the Logs tab only with tenant-logs-read, and its other tabs only with tenant-write: an Auditor sees only Logs, a Security Officer every tab, a Configuration Manager every tab but Logs, and an SBEI no Settings. Without tenant-logs-read, listElectoralLog and the exports refuse (403), including direct API calls. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; this PR adds the tenant-logs-read check to the tenant export)
• Settings > Logs has the same header, date range, filters, columns, details popup and export menu as Election Event > Logs, and differs only in its subtitle, timezone, statement kinds and export scopes. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902)
• Times show in the tenant log's timezone (PhST for COMELEC) and in my time. Filters by username, statement kind, event type, log type and created range (inclusive, to the minute, with PhST boundary cases verified) return exactly the matching entries in the tenant log and in event logs. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902)
• Clicking an entry in either list opens a popup (the shared Dialog, not a side drawer) with who, when (the log's zone, my time and UTC), what, the old and new values, the sign-in details or the Keycloak change, the log and the signing keys; Close and Escape dismiss it. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; this PR adds the imported file and the deleted log's system key)
• Exports reconcile exactly with the list for the same scope and range. They cover the tenant log (whole, or one account with or without its election event entries) and event logs (All Posts, one Post, one country, one account), over a date range, as CSV, SQL or PDF. Each export writes LogsExported with the file's SHA-256. The event export checks logs-export and the tenant export checks tenant-logs-export. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902. This PR adds deleted events' logs to one account's export and fixes the tenant export's download. CSV was checked on the local stack; PDF was not, because it does not render in the dev image.)
• The SQL export loads into PostgreSQL, and every entry's signatures verify with the exported public keys. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; test postgres_audit_log_outbox)
• Every posted entry appears as one JSON line in Windmill's output, without old or new values or secrets. The outbox's waiting time is available to VOTE-SECOPS's alerts. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; this PR writes the waiting time on every worker pass)
• The one-off task gives every existing tenant its log and key, and AdminPublicKey entries are written to the tenant log. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; this PR posts keys after commit and backfills keys made earlier)
• The second organization's tenant log works by configuration only (its timezone, accounts and labels); no code refers to COMELEC. (✨ Every administrative action logged, including a tenant-level log (EMS-AUDIT) #3902; nothing in this PR names a client)
• Apply every linked source requirement, preserving its exact targets, alternatives and lifecycle stage. This package estimate is shared and must not be summed per row. (A rule for the whole package, not something one pull request can tick)

Not in this pull request

• EMS-SBEI-QUORUM's signing entries still use their own signing_log_outbox.
• Tenant log checkpoints and an audit task for the tenant log (VOTE-AUDIT).
• Erasing or reopening a retired event database (OPS-RETENTION).
• Beyond's client-setup chart still configures only jboss-logging for the tenant realm. Existing realms are set up by the one-off task, and new tenants by this pull request.

Upgrade notes

• Monitoring that parsed the outbox line's waiting/oldest_seconds must read pending_entries/oldest_pending_seconds.
• Roles that exported the tenant log with only tenant-logs-export need tenant-logs-read too.

Tests

• electoral-log: 124 tests with the database tests (--include-ignored), including reading a retired database and closing it again.
• windmill: 1705 lib tests pass. decrypted_archives_use_private_owned_files_and_leave_no_plaintext needs an openssl binary on the host, and passes with one. postgres_audit_log_outbox (8, including the post-commit key announcement, the backfill and retired events in exports) and postgres_audit_change (18) pass.
• harvest: 561 pass, including the new 403 for an export without read permission.
• admin-portal: 2233 jest tests, LogEntryDialog stories (17, with a new import story), story inventory, typecheck:stories, and the journeys settings/logs, settings/access and events/operations (29).
• End to end on a local stack built from this branch (Keycloak, Hasura, Harvest, Windmill, PostgreSQL, MinIO, production portals), using the backend E2E bootstrap. All 11 UI journeys pass. The stack also checked, by hand: the import entry's checksum and counts; retired databases read and closed again; the account export's download and its retired rows; a new tenant's realm settings and roles; the backlog line. That run found the export download bug fixed here.

Not measured locally: the per-package coverage gates.

Demo

Video with voiceover, to upload to meta#13781: import, then an event edit, deletion, the account export with the retired log, a new tenant's realm, and the backlog line.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review: 2

• release/10.0
• feat/meta-12767/.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Repository: sequentech/step/.coderabbit.yaml
• Review profile: CHILL
• Plan: Advanced
• Run ID: 64c98b26-f591-46a1-b041-41658ce459e1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-17 00:30 UTC