Repository navigation
✨ Tenant log: activity outside election events is signed and shown in Settings > Logs (VOTE-AUDIT) - #3900
Merged
Conversation
… Settings > Logs SW-F-0364 (VOTE-AUDIT): log all system activity from start to finish. Each tenant gets a signed log next to its election events' logs, a board of the electoral-log base database with a key of its own. Keycloak events of the tenant realm and the Harvest requests that belong to no election event go to it through the electoral-log event queue. Settings > Logs lists it under the new tenant-logs-read permission. Parent issue: sequentech/meta#14259 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review: 2
Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
|
A batch stamped every Keycloak entry of the tenant's log with the time it was stored. The entry now carries the event's own time, as a request's entry does. Parent issue: sequentech/meta#14259 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Hasura metadata: declare the RestorePrivateKeyOutcome enum that AuthorizeTallyCustodyOutput uses. - sequent-core: the WASM boundary test signs with voting-crypto, as the crate does, instead of the removed ed25519-dalek. - orare: sequent-core is a dev-dependency of the proc-macro crate, so sequent-core and braid are built once and their rlibs no longer collide. - Windmill: the migrated sign-key-ceremony and sign-tally-key roles are described as the Admin Portal labels them; realm templates and docs follow. - Admin Portal: the trustee, key ceremony and start step stories and the /trustee route smoke follow the custody screens; Prettier. - Tooling: the Rust watch test reads the workers' target directory, the journey runner test sets the voting portal URL, ruff on the coverage summary test. - The committed sequent-core WASM package is regenerated. Parent issue: sequentech/meta#14259 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keycloak reports a sign-in code it sent as an event of the step that sent it, with the message as its body. In the tenant's log that looked like a second sign-in and carried the message. It is now a MESSAGE_SENT entry with no text. Parent issue: sequentech/meta#14259 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ation - Admin Portal journeys: the custody request is downloaded under the trustee's name, a new event is created with the WBRAID protocol, and the tally list's key restore state is mocked for trustees. - Coverage: the voting-crypto, protocol-board and trustee profiles declare their scope exceptions, which the runner requires. Parent issue: sequentech/meta#14259 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rvice's port The job's PostgreSQL service listens on 3322; WBRAID_BOARD_DATABASE_URL named 5432, so the protocol board archive test could not connect. Parent issue: sequentech/meta#14259 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test unpacked the bundled runtime into the home cache, which the coverage job's two measurements share: the second one found the files and covered the reuse path, so the head, measured first, always looked three lines behind the base. The test now has its own cache and builds twice, covering both paths in every run. Parent issue: sequentech/meta#14260 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With a cache directory of its own the test no longer took the fallback from XDG_CACHE_HOME to the home directory, which the base still covers. It now runs with a home directory of its own and no XDG_CACHE_HOME. Parent issue: sequentech/meta#14260 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ow the ballot box - Windmill: update_publish_ballot calls validate_published_ballot_styles again. The VoteSecure migration added the custody check to that function and dropped its only call, so a publication with a native joint key could be published before the trustees confirmed custody, and the decoding and acclamation checks no longer ran at publish. - Backend journeys: an imported publication is restored unpublished, as the import documents; the combined-contest journey reads the voter's stored votes from the voter status action, since Hasura has no cast_vote table. Parent issue: sequentech/meta#14260 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hell The PDF renderer starts Chrome with --single-process. Debian's Chromium, now 155, traps at startup with that flag (0 of 6 starts in the image), so in the development and backend journey stacks every PDF render retried for minutes and the post-tally task never finished: Windmill was killed at shutdown and the journeys' coverage step failed. On amd64 the image now installs the Chrome headless shell 153.0.8010.36 that .github/actions/setup-test-browser pins for the unit tests, ahead of chromium on the PATH (6 of 6 starts). Parent issue: sequentech/meta#14260 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The VoteSecure migration renamed RestorePrivateKeyOutput to
AuthorizeTallyCustodyOutput and removed the RestorePrivateKeyOutcome enum,
but left the type's outcome field. Declaring the enum again made the
metadata consistent and Hasura then refused Harvest's answer, which has no
outcome ("field outcome expected in webhook response"). No client selects
the field, so it is removed with the enum.
Parent issue: sequentech/meta#14260
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
send_bound_code delivers the code and marks the link CODE_SENT in a later transaction. The test read the link as soon as the fake provider listed the message, so on a slow runner it still found PENDING, which failed the harvest coverage measurement of the head or of the base. It now waits for the state, as it waits for the message. Parent issue: sequentech/meta#14260 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ereslibre
added a commit
that referenced
this pull request
Oct 14, 2026
#3900 merged a tenant log of its own into ovcs. EMS-AUDIT's replaces it: - One tenant log: the per-tenant database, outbox and trigger, listed by listElectoralLog with no election event. #3900's boards in the base database, listTenantLog, POST /tenant-log and its Settings > Logs go. - StatementBody::TenantActivity keeps Borsh tag 43 and its statement types 62-63; Audit moves to tag 44 and its types to 64-68. - Kept from #3900, written into the same log: Harvest's request entries (TenantAdminAction, with accepted, denied or failed), MESSAGE_SENT without the message's text, and entries stamped with the time their producer saw them. An event of an unknown tenant is dead-lettered instead of creating a log. - Keycloak listener: one realm parser (LogRealm); a message sent to a tenant account leaves Keycloak without its text. - #3900's repairs of ovcs's checks are taken as they are. - sequent-core WASM package regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ereslibre
added a commit
that referenced
this pull request
Oct 14, 2026
…MS-AUDIT) (#3902) Parent issue: sequentech/meta#13781 Design package [EMS-AUDIT](https://project-1.sequent.sh/#package/EMS-AUDIT). Every administrative action is logged, including the ones that happen outside an election event. Each tenant gets a signed log of its own, shown in Settings > Logs, and every change carries its old and new values, in the tenant log and in each event's log. > **Replaces #3900's tenant log (VOTE-AUDIT).** #3900 merged into `ovcs` first; `ovcs` is merged in here and this tenant log replaces its one. Gone: #3900's boards in the base log database, `listTenantLog`, `POST /tenant-log` and its Settings > Logs. Kept, and written into this tenant log: `StatementBody::TenantActivity` at Borsh tag 43 with statement types 62–63, so stored entries still decode (`Audit` is now tag 44, types 64–68); Harvest's request entries (`TenantAdminAction`: accepted, denied or failed); `MESSAGE_SENT` without the message's text; entries stamped with the time their producer saw them. ## What changes | Area | Change | | --- | --- | | electoral-log | `Audit` statement body with five kinds (`AdminChange`, `KeycloakAdminEvent`, `LogsExported`, `LogWriterStarted`, `LogWriterStopped`). A log database per tenant (`<base>_t<tenant>`). A deleted event's database is retired (renamed, closed to connections), not dropped. `event_type` and `log_type` columns so lists filter by them | | Database | `audit_log_outbox`, and the `audit_change()` trigger on the 26 tables an admin role can write through Hasura. It reads the Hasura session, diffs JSON key by key, redacts secrets and writes in the same transaction. A coverage test fails when such a table has no trigger | | Windmill | Outbox worker: signs each entry with the admin's key and the log's key and posts it exactly once, in order. Keycloak events of the tenant realm and admin events reach the tenant log; an event that cannot be placed becomes an ERROR entry instead of failing its batch. Exports by scope and range as CSV, SQL or PDF, each recorded as `LogsExported` with the file's SHA-256. One JSON line per posted entry for monitoring. One-off task for existing tenants (log, key, realm listener, the two roles) | | Harvest | `listElectoralLog` without an election event reads the tenant log (needs `tenant-logs-read`, 403 otherwise). `export_tenant_logs` (needs `tenant-logs-export`). The event export checks `logs-export` instead of `report-write`. Audit entries in the routes for staff accounts, roles, permissions, phone blacklist, realm attributes, password policy, voter authentication and the Google service account key | | Keycloak | `custom-event-listener` handles the tenant realm (sign-ins, failures, lockouts, credential changes; no token refreshes) and admin events, with secrets redacted. An unknown account is recorded without the name typed | | Admin Portal | Settings > Logs, for `tenant-logs-read`. Entry popup with who, when, what, old and new values, sign-in or Keycloak details and signatures, in both logs. Export scopes and SQL. "Timezone for tenant logs" in Languages. An auditor with only the log's permissions sees only Logs | | Docs | Manager reference for Settings > Logs, auditor tutorial for verifying an SQL export, developer page for the architecture | ## Not in this pull request - EMS-SBEI-QUORUM's signing entries still use their own `signing_log_outbox`. - Many event-level Harvest routes write no `AdminChange` yet (areas, elections, dates, ceremonies, tally, publications, scheduled events, reports). Changes made to those tables through Hasura are covered by the trigger. - Certificate-authority routes are unchanged. - No command erases or reopens a retired event database (OPS-RETENTION), and one account's export does not read retired events. - The tenant log has no checkpoints or audit task yet. - Beyond's client-setup chart is not changed here; existing realms are set up by the one-off task. ## Upgrade notes - Run `electoral-log-admin upgrade-event-databases` before the new Windmill and Harvest start: appends write the new `event_type` and `log_type` columns. - Roles that exported event logs with `report-write` need `logs-export`. ## Tests - electoral-log: the whole suite, including the database-backed tests (tenant databases, retiring, filters, backfill). - windmill: unit tests for entries, redaction, export and the Keycloak consumer; `postgres_audit_log_outbox` (6 tests: signing, exactly-once posting, ordering after an outage, rerouting, deletion, and an SQL export loaded into PostgreSQL with every signature verified); `postgres_audit_change` (18 tests of the trigger). - harvest: 544 pass. - hasura: migrations apply, roll back and reapply to an identical schema; 115 metadata tests pass. - custom-event-listener: 38 Java tests; spotless clean. - admin-portal: 2174 jest tests; the story and journey suites of the changed screens passed before the last portal fix (the role used to read the schema), which is covered by jest only. - Run end to end on a local stack (Keycloak, Hasura, Harvest, Windmill, PostgreSQL), which found and fixed seven defects. Failing on `ovcs` itself, not caused by this change: `migrate_realm_permissions::the_migration_adds_the_signing_and_messaging_permissions_with_their_labels` (portal labels and migration labels differ since #3885), journeys `events.spec.ts:215` and the `/trustee` route smoke, and Hasura metadata uses an undeclared enum `RestorePrivateKeyOutcome`. PDF export does not render in the dev image (its Chromium crashes with the flags `sequent-core` passes); CSV and SQL do. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
ereslibre
added a commit
that referenced
this pull request
Oct 17, 2026
…DIT, SW-F-0364) (#3928) Parent issue: sequentech/meta#14259 Requirement [SW-F-0364](https://project-1.sequent.sh/#requirement/SW-F-0364): log all system activity from start to finish. #3900 and #3902 log what a person changes through Hasura, accounts and roles, Keycloak events and the tenant-level actions. #3902's "Not in this pull request" left the rest open: most event-level routes (areas, elections, dates, voting status, ceremonies, tallies, publications, scheduled events, reports, applications, the approval matrix, monitoring, signing settings) write with Harvest's or Windmill's own SQL, which `audit_change()` skipped, so they wrote no `AdminChange`. This pull request closes that gap. > **Scope agreed with EMS-AUDIT (meta#13781):** this pull request carries the route coverage. The tenant deletion and country-limit entries come from that session's work; its own pull request keeps the rest of EMS-AUDIT. ## What changes | Area | Change | | --- | --- | | Database | Migration `1791800000000_audit_changes_step_makes`: `audit_change()` also logs the writes of a transaction whose admin Step names in the setting `sequent.audit_actor` (`{user_id, username, role, origin, tenant_id}`), with old and new values; a Hasura session still takes precedence. The trigger on the tables only Step writes for an admin: `keys_ceremony`, `messaging_account`, `messaging_event_config`, `monitoring_config`, `signing_checks`, `staff_certificate`, `tally_results_publication`, `tally_sheet_import`, `tally_sheet_import_item`. `webhook_key` is a secret (SQL and Rust rules) | | Windmill | `audit_log::act_for(&tx, &actor, origin, tenant_id)`. Results publication (publish, revoke, website policy), monitoring settings (`save_as`, `reset_to_preset_as`, `set_mode_as`) and a signing approval (as the signer whose signature runs the action) name their admin. `insert_tenant` takes the creating admin | | Harvest | `audit::act_for(&tx, &claims)` right after the transaction opens in every state-changing admin route: areas, elections, election dates and schedule recompute, keys and tally ceremonies, trustee custody, ballot publication, voting status, tally sheets and their imports, signing certificates, issuers, checks and rules, messaging accounts and settings, applications, scheduled events, lifecycle policies, uploads and previews, certificate authority import and delete | | Harvest, own entries | Where the trigger cannot see the change: area and candidate imports (one entry with the document and declared checksum), export requests (event, tally results, ballot publication, applications, tasks, trustees, certificate authorities, reports), the approval matrix (old and new version), a report's password set (named, never recorded), a document's password revealed, an electoral-log console query, a voter information letter (it resets the voter's password), the country limits before and after | | Windmill, own entries | Deleting a tenant: an `AdminChange` staged in the deleting transaction, in the log of the tenant whose admin deleted it (the route passes the admin as an optional task input) | | Guard | `harvest/tests/support/audited_routes.rs` fails when a route file commits a transaction without naming its admin or staging an entry. Exemptions, with their reasons, in `tests/fixtures/unaudited-committing-routes.txt` (voting portal routes, reads that commit, callbacks) | | Docs | Developer page "Tenant log and audit entries": sections 5, 6.1, 6.2 and 13 | Also fixed on the way: - `/encrypt-report` logged the report's password (`info!("body …")` and the tracing span). - `/miru/upload-signature` recorded its input, password included, in its tracing span. - `/import-candidates` wrote its task row before checking the permission; it now checks first (its `route_permissions` case is updated). - `/delete-election-event` passes the admin's role like every other route. ## Not in this pull request - What a Windmill task writes on its own after the route answered (ballot styles of a publication, a tally's execution, documents of reports and letters, reconciliation patches) has no admin; the route's entry records the decision. - Keycloak changes when an application is confirmed or rejected are recorded only by Keycloak's admin events. - `/fetch-document` downloads write no entry. ## Tests - windmill: the whole suite, 3088 pass, including `postgres_audit_change` (24 tests; 6 new: an admin named by Harvest or Windmill, the setting ending with the transaction, Hasura's precedence, the new tables, a keys ceremony, `webhook_key`). - harvest: 570 pass, including the route guard, `act_for` on the migrated test database, import and export entries, and the route permission inventory. - hasura: migrations apply, roll back and reapply to an identical schema; 115 metadata tests pass. - `reuse lint` passes. ## Demo A narrated video of the feature on a local stack is attached to meta#14259. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parent issue: https://github.com/sequentech/meta/issues/14259
Design package VOTE-AUDIT. This PR covers SW-F-0364, "Log all system activity from start to finish": activity that belongs to no election event now has a signed log, shown in the Admin Portal.
What changes
Tenant log. Each tenant gets a log next to its election events' logs, with a signing key of its own, in the electoral log's format (signed statements in a Merkle log). It is a board of the electoral-log server's base database, created with its key and a
TenantLogOpenedentry the first time the tenant has something to record.Sign-ins.
custom-event-listenerrecognizes the tenant realm and sends its events with no election event (KeycloakUserEvent), leaving out token housekeeping. The development realm and the COMELEC realm template list the listener.Administrator requests. Harvest records the requests that belong to no election event as
TenantAdminActionentries: election events (create, import, delete), staff accounts, roles and permissions, tenants, the tenant's configuration, templates and country limits. An entry holds the action, who asked, the outcome with its HTTP status and what the request named, inside the signed statement. Denied and failed requests are ERROR entries.One writer. Both producers enqueue on the electoral-log event queue. The batch task routes an event without an election event to its tenant's log instead of dead-lettering it.
Settings > Logs. A new tab lists the tenant's log with filters and details under Show More. It needs the new
tenant-logs-readpermission and nottenant-write, so a read-only role opens Settings and sees only Logs.API. Hasura action
listTenantLog, Harvest routePOST /tenant-log.Entries keep their own time. A sign-in is stamped with the time Keycloak saw it, a request with the time Harvest answered it, not with the time the batch was stored.
Sent messages. A message Keycloak sends to an account (a sign-in code) is recorded as
MESSAGE_SENT, without its text.Also in this PR: checks that were red on
ovcsovcshas had failing checks since #3885 (VoteSecure migration) and #3883. To get this PR green they are repaired here, in separate commits; the same repairs alone are in #3901 (meta#14260).windmill coverageandwrap-map-err coveragealso measure the base commit, whose own Windmill tests fail, so they stay red here until #3901 is merged intoovcs:AuthorizeTallyCustodyOutputloses a leftoveroutcomefield of a removed enum, which made every action inconsistent.voting-cryptoinstead of the removeded25519-dalek.sequent-coreis a dev-dependency of the proc-macro crate, sosequent-coreandbraidare built once and their rlibs no longer collide (thecontractsjobs).sign-key-ceremonyandsign-tally-keyroles are described as the Admin Portal labels them; realm templates and docs follow./trusteesmoke), and Prettier.validate_published_ballot_stylesagain, whose only call had been dropped; on amd64 the backend image installs the pinned Chrome headless shell, because Debian's Chromium 155 no longer starts with the PDF renderer's--single-process.scope_exceptionsof the voting-crypto, protocol-board and trustee coverage profiles.Not in this PR
Notes for review
StatementBody::TenantActivityand the statement typesTenantLogOpenedandTenantAdminActionare appended, so stored entries keep decoding; the discriminant tests pin them.GetElectoralLogBody.election_event_idnow defaults to empty, for the tenant listing.SettingsScreenopens withsettings-menuand eithertenant-writeortenant-logs-read; the existing tabs still needtenant-write.fail-requestpolicy it fails while the task-queue database is unreachable or not set up.tenant-logs-readrole andcustom-event-listenerin their realm's event listeners;docs/.../13-electoral-log/04-tenant-log.mdsays how. Nothing is migrated.Tests
SettingsLogs,SettingsScreen), journeys (settings/access.spec.ts).🤖 Generated with Claude Code