NOTE: AI-generated
Condensed security reference for fuz_app's auth stack. For design rationale and identity model, see ./identity.md. For error schemas, DB, and session internals, see ./architecture.md.
fuz_app's auth stack is designed to protect against:
- Network attackers without credentials — all sensitive routes require auth
- Password brute force — rate limiting + account enumeration prevention
- Credential theft — HttpOnly cookies, bearer token origin rejection
- Privilege escalation — credential type hierarchy, admin-grant-path enforcement
- Insider threats — best-effort audit trail for auth mutations (see Audit Logging — a failed audit write is logged and swallowed, never blocking the mutation), granted_by provenance
Current deployment target: single-process, single-node. See Known Limitations.
HTTPS is required in production. Session cookies are set with Secure, which
browsers silently ignore over plain HTTP — the cookie is never sent, and login
appears broken with no error. TLS termination at the reverse proxy (nginx) is
the expected configuration. The app server does not handle TLS directly.
The app server emits no backend-fingerprinting response headers — no
Server, X-Powered-By, or WWW-Authenticate on any response, including
401s and errors. Both spines converge on this: a prober cannot tell which
implementation or framework is serving the request from the response headers,
and a 401 carries no auth-scheme challenge to probe. The cross-impl
conformance suite enforces it as an always-on invariant over every case (the
FINGERPRINT_HEADERS floor in the conformance-table runner) plus an explicit
expect.headers row, so a framework upgrade or consumer middleware that adds
one of these to a single spine fails the suite rather than silently becoming a
fingerprinting oracle.
This is the application posture; server_tokens off (see nginx Static File
Serving) suppresses the reverse proxy's own
Server banner at the edge.
Three credential types with privilege ceilings enforced by credential type — not just by role_grant existence. A session cookie with a keeper role_grant cannot exercise keeper routes; only a daemon token can.
| Credential | How obtained | Max privilege |
|---|---|---|
| Session cookie | Login form (browser only) | admin |
| API token | account_token_create RPC (CLI/programmatic) |
admin |
| Daemon token | Filesystem (operator-only) | keeper |
Session cookies and API tokens can grant admin-level access. Only a daemon token — which requires local filesystem access — can reach keeper-level operations (role_grant management, audit, bootstrap recovery).
Nine endpoints declare credential_types: ['session'] on their auth
axis. The dispatcher rejects non-session credentials with 403
ERROR_CREDENTIAL_TYPE_REQUIRED + required_credential_types: ['session']
before the handler runs. Seven close a credential-minting, lockout, or
privilege-pivot threat (list below); the remaining two are gated for reasons
that aren't a threat of their own — POST /logout for forensic fidelity, and
GET /audit/stream because a long-lived admin feed is not handed to a
channel the deployment never chose (both noted after the list).
account_token_create— Bearer-spawn-bearer persistence — leaked API token mints siblings with innocuous names to outlive revocation.account_token_revoke— Sibling disruption — leaked bearer revokes the legitimate sibling token to disrupt the user.account_session_revoke— Lockout-by-composition — leaked bearer enumerates viaaccount_session_listthen revokes each session.account_session_revoke_all— Lockout — leaked bearer revokes every session in one call.POST /password(REST) — Lockout + credential reset — leaked bearer rotates the password to lock the legitimate user out.role_grant_offer_accept— Privilege pivot — leaked bearer accepts a pending admin offer, and the resulting role grant outlives revoking the token. The sibling offer verbs stay ungated:_createis the offering side,_decline/_retractdestroy a pending offer and confer nothing.self_service_role_set— Privilege pivot — same shape, self-targeted: a leaked account-wide token grants itself an eligible role. The verb exists to serve a UI affordance, so the bearer channel buys nothing.
Cookies are tied to a browser context (HttpOnly + SameSite=Strict + Secure)
— the right trust bar for "mint a long-lived credential / rotate password
/ revoke everything / move your own authority." Admin token/session revoke
specs in admin_action_specs.ts deliberately stay unrestricted because admin
scripting from CLI/bearer is legitimate operator workflow.
POST /logout — gated for forensic fidelity, not a threat. Logout
tears down access, so a leaked bearer "logging out" harms no one — it
holds no session to end. Without the gate the handler would return a
misleading 200 {ok: true} and emit a phantom logout audit row for an
event that never happened; the session-credential gate refuses such a
caller instead (403 ERROR_CREDENTIAL_TYPE_REQUIRED). Both spine impls
converge on this shape.
GET /audit/stream — gated because it is a long-lived admin feed. The
stream carries every audit row on the instance for as long as it stays open,
so it is not handed to a credential channel the deployment never chose: the
route admits sessions only. The gate is not what keeps a stream closable — the
handler registers a bearer's stream under its API token id and a token
revocation closes by that id, so a consumer that widens the channel loses no
close. The route's required_scope is the separate rule-3 declaration and is
unreachable behind this gate on the default mount; see §Token scoping.
Defense in depth — audit metadata. Every gated event records the
minting credential_type in audit metadata (token_create,
token_revoke, session_revoke, session_revoke_all, and all three
password_change outcomes). The field is typed against
BuiltinCredentialType. Forensics survive a future loosening or
bypass of the spec-level gate even though the prevention layer alone
already restricts the channel.
Argon2id with OWASP-recommended parameters. Two password schemas: Password
enforces PASSWORD_LENGTH_MIN to PASSWORD_LENGTH_MAX (300) on creation paths
(signup, bootstrap, password change). PasswordProvided uses min(1) on
login and current-password verification for forward-compatibility if length
requirements change.
Both login failure paths — account not found and wrong password — return identical
{error: 'invalid_credentials'} with status 401. verify_dummy() is called on
the "account not found" path to equalize Argon2id timing with the real
verify_password call. A regression test asserts byte-identity of both responses —
a change to either error message fails the test suite.
Cross-impl parity: the found-vs-not-found byte-identity is held on both the
TS spine and the Rust spine by the conformance suite's equivalence_group gate
(login_enumeration_shadow) — wrong-password and account-not-found must
produce a wire-identical {status, body} on each impl, so a prober hitting
either spine cannot distinguish them. (The timing floor stays a TS-internal
property; it isn't wire-observable for the runner to compare.)
Beyond the constant-time Argon2id baseline, every 401 response is floored to
DEFAULT_LOGIN_FAIL_FLOOR_MS (250ms) plus DEFAULT_LOGIN_FAIL_JITTER_MS
(±25ms) random jitter. The handler races real work against a sleep and awaits
both, so observed response time is max(work, delay) — found-vs-not-found and
rate-limit-skipped-vs-not paths all converge. 429 responses are intentionally
not padded: they carry no oracle (the attacker just chose to overflow the
bucket) and padding would let blocked attackers hold server resources open.
Both floor and jitter are configurable via login_fail_floor_ms /
login_fail_jitter_ms on AccountRouteOptions — tests set them to 0.
First-user setup uses a one-shot filesystem token, not first-signup-wins. Server
writes a secret token to a local file at startup. The operator enters it at
/bootstrap; the file is deleted after use, permanently deactivating the endpoint.
This prevents a network attacker from creating the admin account before the
legitimate operator.
Hardening layers:
- Atomic DB lock:
bootstrap_locksingle-row latch prevents TOCTOU races — the UPDATE-guarded latch is the single account-creation signal. An earlier belt-and-suspendersquery_account_has_anycheck inside the transaction was removed as redundant once both the production and test writers came to flip the same lock row; direct DB tampering withbootstrap_lockis out of scope for the write contract - Early in-memory check:
bootstrap_status.availableshort-circuits before any rate limiting, file reads, or crypto after bootstrap completes — and an unreadable token file flips it tofalse, closing the window rather than leaving the probe answering (and audit-writing) indefinitely - Token file deletion enforcement: If the token file cannot be deleted after
successful bootstrap, the handler throws after completing all success work
(session,
on_bootstrapcallback, audit log). The error response forces operator attention — delete the file manually and log in on_bootstraperror isolation: Callback failures are caught and logged without preventing the bootstrap success response- Input validation: Bootstrap username uses the
Usernameschema (same constraints as signup), not a weakermin(1)check
- Cookie attributes:
HttpOnly,Secure,SameSite=Strict,Path=/— regression-tested (a cookie attribute change fails the test suite) - Server-side sessions: Cookie contains a signed opaque ID (HMAC-SHA256); session data is DB-resident, stored as a blake3 hash
- Absolute lifetime: 30-day hard cap, set once at mint on both layers —
the DB row's
expires_atand the cookie's embedded expiry are fixed, and there is deliberately no touch/renewal on either (a sliding window renews a leaked cookie forever at one request per window; both spines converged on the hard cap). The only recovery from an aged-out session is a fresh login. Sessions carry no activity signal — there is nothing to touch or renew - Expiry and live connections: an expired session is refused on every
request, but a WebSocket or SSE stream it opened does not read it again.
The auth cleanup deletes expired session rows and closes those connections
— within one cleanup interval of the expiry when it is scheduled
(
auth_cleanuponcreate_app_server, off by default; see Connection Admission, "What remains") - Session limits: Per-account cap (default 5, configurable). Oldest session evicted on login when limit is reached, and the connections it opened are closed once the login commits
- Password change: Revokes all sessions and API tokens and clears the session cookie, so a compromised credential does not persist after rotation. The handler closes the account's WebSockets and SSE streams once the change commits (see Closing on Revocation)
SECRET_FUZ_COOKIE_KEYS supports key rotation via __-separated keys:
- Prepend the new key — it becomes the primary signer
- Old keys remain for verification — a cookie signed by a retired key keeps verifying as-is (it is never re-signed; any re-sign would extend the absolute lifetime)
- Remove old keys after the rotation window — exactly
SESSION_AGE_MAX(30 days), since every cookie a retired key signed has expired by then — or accept that sessions signed with removed keys will be invalidated - Emergency rotation — replace all keys at once. All active sessions are immediately invalidated; users must log in again
- Secret scanning prefix:
secret_fuz_token_triggers automatic secret scanner detection - Blake3-hashed server-side: Raw token is never stored — only the hash
- Required lifetime at mint:
account_token_createtakes a requiredlifetimeunion —{kind:'eternal'}or{kind:'ttl', days}— with deliberately no default, soexpires_at IS NULLalways means "deliberately eternal" and a bounded token needs no follow-up plumbing (the validate chokepoint has enforcedexpires_atall along, on both spines). A framework-level ceiling (max_token_ttl_days) is deferred untilfuzfgrows an expiry story (an expiry-aware token read + a 401 hint) - Browser context discard: Bearer tokens are silently discarded (not
rejected) when
OriginorRefererheaders are present — the middleware callsnext()without setting a context, so public actions still work. Browsers send these headers automatically; CLI tools don't. Prevents XSS from exploiting a token extracted via browser-side code - Soft-fail for invalid tokens: Bearer middleware never returns 401 or
error diagnostics. Invalid, expired, or empty tokens are treated as "no
credential" — downstream auth enforcement (the RPC dispatcher's
pre-authorization auth gate, or
require_authon REST) returns generic errors without leaking token-specific information (invalid_token,account_not_found). The bearer middleware has no hard-fail at all — it never returns a status of its own - Token limits: Per-account cap (default 10, configurable). Oldest token evicted on creation when limit is reached, and the connections it opened are closed once the creation commits
A token carries a scope naming what it may do, stored on
api_token.scope JSONB NOT NULL. Two variants and nothing else — full
(every RPC method, every spine surface) or methods (only the named RPC
methods). account_token_create takes a required scope, so full
authority is spelled at mint rather than defaulted into; sessions and daemon
tokens resolve to full by construction. parse_token_scope fails closed —
an unreadable document refuses the credential rather than widening it.
A narrowed token is RPC-only. methods denies every non-RPC spine
surface outright, whatever its method list says: the db-admin table browser,
the bare-hash fact read (GET /api/facts/:hash), the audit SSE stream
(GET /audit/stream), the surface explorer (GET /api/surface, when
mounted), and the WebSocket upgrade. This is the load-bearing
half of the design, and it is strictly more restrictive than naming each
surface. A method-name allowlist alone would have been a false promise — the
db-admin browser serves paginated rows of every browsable table plus row
DELETE (the credential tables sit behind NON_BROWSABLE_TABLES now, but
browsable rows can still carry secrets in data columns, and could be gated on
a global role a bearer satisfies), so a token whose UI badge read "scoped to
cell_get" could still bulk-read and delete browsable rows.
Enforcement sits at two kinds of site, both ahead of the role gate:
- Per method, inside
perform_action— between the credential gate and the role gate, so the order is credential → scope → role. Every transport that reaches the dispatcher (HTTP RPC, WebSocket) inherits it. - Declaratively on a route spec —
auth.required_scopenames the capability the caller's scope must admit, andfuz_auth_guard_resolvermounts the refusal as a pre-authorization guard. The WS upgrade calls the same shared decision directly (upgradeWebSocket's callback cannot produce a denial).
required_scope is the <section>:<id> capability string a denial reports
back, and it has two arms:
-
surface:<name>— a non-RPC surface, refused to every narrowed token whatever its method list says.surface:audit_stream,surface:fact_bare, andsurface:app_surface(the opt-in, admin-onlyGET /api/surfaceexplorer — TS spine only) are the spine's route-spec surfaces; the WS upgrade and the db-admin browser round out the set the spine knows./audit/streamcarries a channel gate as well as rule 3. A narrowed token is what rule 3 refuses; a full-scope bearer passes it. The audit route also declarescredential_types: ['session'], which refuses every bearer at the channel, ahead of the scope gate: the stream is a long-lived, instance-wide admin feed, and the default does not hand it to a channel the deployment never chose.required_scopestays declared beside it: it is the rule-3 statement the surface census matches on, and it still governs a consumer that deliberately widens the channel.Widening that gate loses no close. The stream handler registers a bearer's stream under its API token id as well as its account id, and a token revocation closes by that id — the handler's own close through
create_sse_connection_closer, and thetokenscopecreate_sse_auth_guarddispatches on. Which audit events close what is declared once, inaudit_event_revocation_scopes, and both revocation listeners read it; the Rust twin does the same with theRevocationScopecolumn onAUDIT_EVENT_SPECS. A consumer route with its ownSubscriberRegistryregisters the same identities to get the same closes. -
rpc:<method>— one action method, refused unless the token lists it. Whatcreate_action_route_specputs on a bridged route (below), and what the dispatcher's per-method gate reports.
The identifier half is deliberately open. A consumer names its own surface
— surface:file_store, surface:git_http — with no upstream release and no
registration. Nothing is lost by that: the RPC-only rule is all-or-nothing, so a
surface
name decides nothing and only labels which surface refused, and the sibling
rpc: arm has always carried arbitrary consumer method names on the same wire
field. Registration throws on a malformed capability (unknown section, empty or
non-[a-z_] surface name), on the field appearing on an ActionSpec (the
dispatcher derives it from the method there), and on an unrestricted route (the
same holder reaches it by dropping the credential) — each of the last two would
be a control that silently does nothing.
Denials are 403 with a flat {error: 'token_scope_required', required_scope}
body (-32002 in the JSON-RPC envelope for the per-method gate), identical on
both spines. The coarser credential gate outranks the scope gate when both
would fire: a channel that may never call the method learns that, not which
scope it is missing.
Consumers carry their own census. The spine gates the surfaces the spine
mounts. A consumer that mounts its own bearer-reachable surface — a file store,
git smart-HTTP — or that rewrites a spine route's auth to admit a credential
the shipped gate refused, has to make the same call for that surface. The
second case is the sharper one, and http/db_routes.ts is the live example:
the shipped table browser needs no surface gate because
credential_types: ['daemon_token'] already excludes every bearer, but a
consumer that re-auths it to a role a bearer satisfies loses both controls at
once and puts every browsable row — the credential tables are floored, but
data columns still carry consumer secrets — behind a scoped credential.
Declaring the capability is the whole of it — you get the refusal, in the right
position (ahead of the role gate, so a narrowed token hears about its scope
rather than about a role it also lacks), and a token_scope_required entry in
your generated attack surface:
const stream_route: RouteSpec = {
method: 'GET',
path: '/api/files/:id',
auth: {
account: 'required',
actor: 'required',
roles: ['admin'],
required_scope: 'surface:file_store',
},
// ...
};A surface that isn't a route spec — your own WebSocket upgrade, anything Hono's
upgradeWebSocket shape rules out — calls token_scope_surface_denial(c, 'surface:<name>')
directly instead, which is what the spine's own WS gate does. Mount it in the
position the resolver would have: after require_auth, before the role gate.
A bridged action gates itself. create_action_route_spec runs its own
handler through the REST pipeline and never reaches perform_action, so the
dispatcher's per-method check cannot fire on it — the bridge therefore derives
required_scope: 'rpc:<method>' onto the route it produces. A token minted for
that method reaches the bridged route; one that wasn't gets the 403 it would
have gotten over RPC. Bridge something with no request/response shape (an SSE
stream, a file download) and the whole-surface rule applies instead: pass
options.auth with your own surface:<name>.
What the spine still cannot do for you is know your surface exists.
src/test/auth/token_scope_surface_census.test.ts is the pattern: enumerate
every credential-consuming site and classify each as resolving the scope,
consulting it, or exempt with a stated reason, so an unreviewed surface fails
the suite instead of shipping ungated.
Rotating filesystem credential for keeper-level operations:
- The token file lives at
~/.{app}/run/daemon_tokenas JSON{"token": "<base64url>"}— wrapper leaves room for future fields (rotated_at, version) without changing every reader; matched by the Rust spine writer (fuz_testing::write_token_file) for wire parity - The producer is test-only on both spines. No production assembly mints
daemon tokens: the TS writer lives in
testing/daemon_token_rotation.tsbehind the dev-env guard (importing it in a production bundle throws), and the Rust writer lives infuz_testing, which a release audit keeps out of every production binary. The credential's only remaining role is the cross-process harness's keeper channel (_testing_reset&c.);auth/daemon_token_middleware.tskeeps only the consumer half. No caller in the ecosystem sendsX-Daemon-Tokenoutside that harness - Mode
0600is applied at creation. The test-only producer writes the file viawrite_file_atomicwith a unique exclusive temp and mode0600, so it never exists group/other-readable (the old optional-chmodpattern, under which a default umask landed it at0644, is gone) - Token rotated every 30 seconds (configurable); the previous token is also accepted to cover the rotation race window
- Both the REST guard composition (
require_credential_types(['daemon_token'])require_role(['keeper'])) and the RPC dispatcher's post-authorization auth gate (check_action_auth_post_authorization, JSON-RPC endpoints) check both: daemon token credential type AND an active keeper role_grant
- Browser context discard: like bearer tokens, daemon tokens are silently
discarded when
OriginorRefereris present — the middleware callsnext()without setting a context. Daemon tokens are loopback-only and never legitimately carry anOrigin, so a header-bearing one is dropped (and the request falls through tocredential_type_requireddownstream) - Compromising the web layer cannot escalate to keeper — filesystem access required
Cross-process integration tests need a few privileged operations the production
wire never exposes — wiping auth tables between tests, forging an expired
session to exercise the DB-row expiry gate, seeding a content-addressed fact
without a store handle, flooding a connection past its outbound budget. These
live as _testing_* RPC actions
(_testing_reset, _testing_mint_session, _testing_put_fact,
_testing_drain_effects, _testing_flood_notifications,
_testing_schema_snapshot, _testing_migration_tracker,
_testing_action_manifest)
that a consumer's test
binary appends to its RPC endpoint at assembly time (fuz_app's own spine
binary, like the Rust stub, also serves them on its WS endpoint, behind the
same per-message gate). They are a deliberate
backdoor, fenced on three independent axes:
- Daemon-token-gated. Every
_testing_*spec declaresauth: {account: 'required', actor: 'none', credential_types: ['daemon_token']}. A session cookie, an API token, or an anonymous caller is refused before the handler runs (401 for no credential, 403credential_type_requiredfor a non-daemon one) — the same credential-type ceiling that gates keeper operations. Only the operator with filesystem access to the rotating daemon token can fire them. - Off the declared surface. They are never registered on a
surface-generating registry, so they appear in no
AppSurface, no committed*_attack_surface.jsonsnapshot, and no generated docs — a published binary's attack surface stays the authoritative "what the server exposes" map. - Excluded from production builds. Every runtime-reachable module in the
testing tree begins with a load-time
assert_dev_envguard (DEVfromesm-env) that throws if imported underDEV=false(the lone exemption is a build-config helper that never reaches a runtime bundle); a coverage test enforces the guard is present on each module. The Rust twin keeps thefuz_testingcrate out of production dependency graphs (cargo xtask check-release).
_testing_mint_session is further constrained: its expires_in_seconds is
required negative, so it can only mint an already-expired session row, never
a usable session for an arbitrary account. The constraint is a
make-impossible-states floor under the gates above — even a misuse cannot forge
a live credential.
_testing_flood_notifications is bounded the same way: at most 65 536
notifications of at most 16 MiB each, and 64 MiB between them, all onto the
caller's own connection — so one call holds the event loop for a bounded
burst and can overflow nothing but the socket that asked.
SSE (Server-Sent Events) streams are long-lived HTTP connections. Auth is
checked at connection time via route-level guards (e.g., require_role('admin')
for the audit log stream), and re-read once the stream is registered (see
Connection Admission). Because the connection
persists, permission changes during the connection lifetime require active
enforcement:
- Identity slots: a
SubscriberRegistrysubscription carries ascope(single, capped identity — typically the session hash) andgroups(any number of uncapped identities — typically[account_id]).close_by_identity()matches either slot, so coarse close (account-wide) and fine close (session-specific) share the same API. It reaches a registration still pending admission as well as an open stream. - Per-scope cap:
max_per_scopebounds concurrent streams sharing a single scope. Audit log SSE defaults toAUDIT_LOG_SSE_MAX_PER_SCOPE = 10(i.e., up to 10 tabs per session). Groups are never capped, so an account's total streams is bounded transitively bymax_sessions × max_per_scope. Overflow closes the oldest FIFO. The cap counts admitted streams and is applied at admission, so a request that is refused never closes another stream. - Outbound budget: what one stream holds unread for its client is bounded
by
SSE_OUTBOUND_QUEUE_BYTES(1 MiB) — see Outbound Bounds. A frame past it ends the stream. - Handler close: every revocation handler closes the connections of what
it revoked through
deps.connection_closer, after its transaction commits (see Closing on Revocation). A registry reaches that closer throughcreate_sse_connection_closer(registry)— oneclose_by_identityper session hash, API token id, or account id. - SSE auth guard:
create_sse_auth_guard(registry, role, log)returns anon_audit_eventcallback that repeats those closes when the audit row is announced, by theRevocationScopethe event declares (audit_event_revocation_scopes):session(session_revoke) — closes only the stream whosescopematches the revoked session's hashtoken(token_revoke) — closes only the stream registered under the revoked API token's idaccount(session_revoke_all,token_revoke_all,password_change,logout,account_delete,account_purge) — closes the target account's streamsrole(role_grant_revoke) — closes the target account's streams when the revoked role is the one the stream requires. The guard is the only closer for this one: no handler close exists for a role, because the WebSocket half deliberately keeps its sockets (their next message is re-authorized)
- Failure guard: Events with
outcome='failure'are ignored. Failed revoke attempts carry attacker-submitted identifiers (e.g. guessed session hashes) in metadata — acting on them would let any authenticated user close another user's SSE stream by guessing or leaking a hash. - No polling: Disconnection is reactive — triggered by the revocation itself. No periodic role_grant refresh is needed.
- Client disconnect:
create_sse_responsecloses its stream when the client disconnects, including one that leaves before the handler returns — a runtime cancels only a body it has started sending, so that case would otherwise leave a registered stream nothing delivers to. On@hono/node-serverthe request'ssignalmisses such a disconnect (it is created on first read and aborted only if it existed when the connection closed), so the Node stream behind the response is watched there — the response object over HTTP/1, itsHttp2Streamover HTTP/2. The audit route registers the stream's close listener before admitting it, so a client gone by then takes no slot and evicts no other stream. - Factory-managed:
audit_log_sse: trueoncreate_app_serverhandles all wiring (registry, guard, closer, broadcaster,on_audit_eventcomposition, event specs).create_audit_log_sse({log, connection_closer})remains for manual control.
The audit log SSE route (/audit/stream) registers each stream with
scope = session_hash and groups = [account_id] (plus the API token id for
a bearer, on a route widened to admit one), so session_revoke closes only
that session's streams, while a role_grant_revoke of the stream's role and
the account-wide revocations (session_revoke_all, token_revoke_all,
password_change, logout, an account delete or purge) close every stream
for the account.
BackendWebsocketTransport bounds concurrent WebSocket connections per
account: max_connections_per_account, default
DEFAULT_MAX_CONNECTIONS_PER_ACCOUNT = 50 (set it on the transport, or on
WsEndpointSpec for the transport create_app_server creates; null
disables). It counts an account's connections across every credential type —
session, API token, and daemon token, none exempt — and is per transport, so
endpoints sharing one transport share one count.
- Evict-oldest, never refuse: a connection past the cap is always admitted and the account's oldest is closed. Refusing the newcomer would let half-open sockets lock a user out with their own dead connections, and would let a stolen credential fill every slot ahead of the real user. Revocation is the answer to a stolen credential, and evict-oldest never stands in its way.
- After every upgrade gate: the cap is applied when the connection is admitted, once the origin check, authentication, the token-scope gate, and any role gate have passed and the credential has been re-read (see Connection Admission) — an unauthenticated, forbidden, or just-revoked request can't close anyone's socket. A connection still pending admission neither counts toward the cap nor is evicted by it.
- Its own close code: the evicted socket closes with
WS_CLOSE_CONNECTION_LIMIT(4004, reasonconnection limit), not the revocation codeWS_CLOSE_SESSION_REVOKED(4001). The credential is still good, soFrontendWebsocketClientdoesn't enter itsrevokedstate — but it doesn't auto-reconnect either, since a reconnect would close a newer socket in turn and clients past the cap would close each other in a loop. It stays closed withsupersededset until the app callsconnect().
The Rust spine applies the same cap, policy, close code, and reason.
Requests on one socket dispatch concurrently — @hono/node-ws does not await
onMessage, so nothing in the adapter serializes them. register_action_ws
bounds them itself, the same way the Rust spine does:
- In-flight ceiling: at most
MAX_INBOUND_DISPATCHES_PER_CONNECTION(128) requests run at once per socket. One more is answeredqueue_overflowwith its id and not dispatched. The socket stays open and keeps reading, so the requests already running still answer, and a reply to a server-initiated request still reaches the handler awaiting it — blocking reads at the cap would deadlock that handler. - One live request per id: a request whose id names a request still
running on the socket is answered
invalid_requestwithdata.reason: 'duplicate_request_id'and not dispatched. Two requests under one id would share acancelentry and a reply id, and the first to finish would clear the other's; the running request keeps both. The id is free again once its request has been answered.
The duplicate check runs first, so a duplicate is refused as one whatever the
load. Neither refusal reaches the dispatcher, so neither charges the action
rate limiters — the per-socket ceiling is what bounds a flood of them, and
each costs a parse and an error frame. Both spines refuse the same ids (a
string and a number with the same digits are different ids; 7 and 7.0
are the same one), with the same codes, messages, and data. The one
divergence is a request for an unknown method: Rust looks the method up inside
the dispatch, so the request holds its id until its method_not_found reply
is written, while TypeScript answers it before the id is taken — a client that
reuses the id in the very next pipelined frame can be refused as a duplicate on
Rust and dispatched on TypeScript.
A server never waits on a client to send to it: a broadcast is synchronous and called from anywhere, and a WebSocket's requests keep being answered whether or not their client reads the answers. So a client that stops reading — or reads slower than it is sent to, including by its own requests — makes the server hold what it has not taken. Both long-lived transports bound that, per connection, with one rule:
- The charge: each frame waiting to be written is charged the UTF-8 length
of its text plus
OUTBOUND_FRAME_OVERHEAD_BYTES(64). The fixed part keeps empty and tiny frames from being free. - The rule: a frame is accepted when the charges waiting plus its own are within the connection's budget — and always when nothing is waiting, so a single message larger than the whole budget is still deliverable. Nothing counts frames.
WebSocket. The budget is outbound_queue_bytes, default
DEFAULT_WS_OUTBOUND_QUEUE_BYTES (4 MiB) — a value of its own, not derived from
the inbound message cap (set it on BackendWebsocketTransport, or on
WsEndpointSpec for the transport create_app_server creates; zero or a
non-positive value throws at assembly). Every frame fuz_app sends a
connection goes through one path — responses, error frames, notifications,
broadcasts, send_to_account, server-initiated requests.
- Overflow closes the connection: a frame that does not fit is not
delivered, and the socket closes with
WS_CLOSE_OUTBOUND_OVERFLOW(4005, reasonoutbound queue overflow). The connection leaves the transport in the same synchronous step — later sends miss it and the counts and the per-account cap no longer see it — its in-flight handlers are cancelled, and no frame follows the close frame, the response of the request whose handler overflowed the connection included. - Not a revocation: the credential is still good and nothing took the
connection's place, so
FrontendWebsocketClientreconnects under its ordinary backoff. The close is the client's notice that it missed messages; a consumer re-reads whatever state it tracks on reconnect. - Write deadlines: bytes a socket has not taken
WS_WRITE_STALL_TIMEOUT(10 s) after they were handed to it end the connection with no close frame — the client sees an abnormal close (1006). A close frame getsWS_CLOSE_WRITE_TIMEOUT(1 s), after which a socket still holding unwritten bytes is cut the same way where the runtime can cut one, so a revoked or overflowed connection whose client stopped reading does not keep its buffered frames for the runtime's own close timeout. - One log line, no audit row: an overflow or a stall logs one warning naming the connection, its account, and what was waiting. A client can provoke either at will, so an audit write per event would amplify it.
What the WebSocket bound leaves:
- The raw socket. An
on_socket_openhook receives the socket itself (SocketOpenContext.ws). A frame a consumer sends on it directly goes around the budget and the deadlines;notifyon the same context goes through them. - A socket the runtime does not report on. The pump reads the runtime's
socket buffer (
bufferedAmounton Node'swsand Deno,getBufferedAmount()on Bun). An adapter whose socket exposes neither reads as never backed up: its frames are budgeted while they wait for the pump, and no stall is detected. - A closed socket is not counted. A connection the server closed or
dropped leaves the transport at once, so the per-account connection cap does
not count it while its socket is still closing. What the runtime holds for
such a socket is bounded instead: at a close the transport hands the socket
only the waiting frames its buffer has room for and discards the rest, so —
where the runtime reports its buffer — the buffer holds less than
WS_WRITE_HIGH_WATER_BYTES(64 KiB) plus one frame, and the close frame. On Node'swsand Bun a socket still holding any of it at the close deadline (1 s) is cut. Deno's standardWebSocketcannot be cut: a closed socket whose client is not reading stays in the runtime, holding that much, until the runtime or the client ends it, and the cap does not limit how many such sockets one account leaves behind. - One frame of any size. A frame is accepted when nothing is waiting, whatever its size, so "one frame" above is as large as the largest message the server sends.
SSE. create_sse_response holds what a stream has written and its client
has not read to SSE_OUTBOUND_QUEUE_BYTES (1 MiB, fixed), with the same charge
and the same rule. A frame that does not fit ends the stream: what was unread
is dropped, the body ends with no final frame, and its registration is removed.
A client that is reading reconnects and misses the rows in between — the audit
log itself is the record. One warning names the account; no audit row is
written, since each one would itself be fanned to every stream.
The Rust spine applies the same budgets, charge, rule, close code, reason, and deadlines, pinned on both by the outbound-overflow cross-backend suite. What each implementation decides for itself:
- What "waiting" measures. On the TS spine a frame sent to an idle connection goes straight to the socket; the rest of a burst sent in one synchronous run waits, as does anything sent while the runtime's own socket buffer is backed up. The Rust spine queues every frame for its connection's writer.
- Frames accepted before an overflow. The Rust spine discards them all. The TS spine discards what was still waiting; frames already handed to the socket may reach the client ahead of the close frame.
- Frames waiting at any other close. At a revocation, a cap eviction, the
shutdown, or
close_connection, the TS spine hands its waiting frames to the socket ahead of the close frame while the socket's buffer is belowWS_WRITE_HIGH_WATER_BYTES, and discards the rest. A reply sent to an otherwise idle connection just before a self-revoking close still arrives; a reply queued behind more than that mark of unwritten bytes is not delivered. The Rust spine discards every waiting frame. - Cutting a socket. The cut needs the runtime to drop a socket without the
closing handshake: Node's
wsand Bun do. Deno's standardWebSocketcannot, so there a stalled connection is closed withWS_CLOSE_OUTBOUND_OVERFLOWinstead — removed from the transport and its handlers cancelled all the same — and a stalled or closed socket whose client is not reading stays in the runtime until the runtime or the client ends it. - A stalled SSE stream. The budget bounds memory, not the connection: a stream whose client stopped reading stays open on its socket, holding nothing more, until the client goes away or the server shuts down.
A WebSocket and an SSE stream are authorized once, at open, and from then on
only a revocation's close ends them: per-message WebSocket dispatch re-reads
role grants, never the session or token, and a one-way stream re-reads
nothing. Closes reach connections by identity, so they reach only connections
that are registered. That leaves the moment of opening: the credential is
read by the auth middleware, and the connection is registered later — for a
WebSocket, after the 101, the upgrade, and the adapter's open event. A
revocation landing in between (session revoke, logout, password change, token
revoke, account delete) would close nothing, and the connection would then
open on the dead credential and keep it. On a WebSocket it could also evict a
live socket of the same account past the connection cap.
Both transports close that window the same way, the twin of the Rust spine's admission:
- Register pending. The connection is registered first, un-admitted
(
BackendWebsocketTransport.register_pending,SubscriberRegistry.subscribe_pending). A pending registration is closeable by every revocation from that instant, and is otherwise inert: no broadcast, targeted notification, or server-initiated request is delivered to it, nothing is queued for it, no count includes it, and the caps neither count it nor evict for it. - Re-read. The credential is read again
(
revalidate_resolved_auth,auth/resolved_auth.ts): the session row or API-token row still exists, is unexpired, and belongs to the same account, and the account is not soft-deleted. A daemon token has no row and no revocation event, so only its account is re-read. The re-read uses the resolve path's own queries and liveness predicate, writes nothing, and does not touch a token'slast_used_at— re-checking a credential is not using it. The audit stream also re-reads the acting actor's role grants, since a role revocation closes streams too. - Admit.
admitrefuses when the pending registration is gone — a revocation closed it — and otherwise applies the cap and opens the connection.
A revocation whose close ran after step 1 found the registration; one that committed before step 2 is seen by the re-read. Every revocation closes only after it commits, so every revocation is one or the other.
What a refused connection sees.
- WebSocket. The handshake has already answered
101, so a refusal is a close frame:WS_CLOSE_SESSION_REVOKED(4001, reasonSession revoked) for a dead credential or a registration closed while pending — the same frame a socket revoked a moment later gets, so the two are indistinguishable — andWS_CLOSE_INTERNAL_ERROR(1011, reasoninternal error) when the re-read itself failed. Admission fails closed: a credential that could not be re-checked is not a valid one. Nothing else is written to a refused socket, it evicts nothing, and itson_socket_open/on_socket_closehooks never run. - Audit SSE stream. A dead credential answers
401 authentication_requiredand a lost role403 insufficient_permissions, the statuses the gates would have given a moment later; a failed re-read answers500. A registration closed while pending answers200with a body that is the connect comment and nothing else — to anEventSource, a stream closed the instant it opened — and the client's reconnect is answered by the gates.
Frames sent before admission. A WebSocket client can send as soon as the
handshake completes. Those frames are queued and dispatched in arrival order
once the connection is admitted (and its on_socket_open hook has completed);
they are dropped unread on a refusal. Nothing is dispatched on a credential
that has not been re-read. The queue is bounded twice, since a runtime adapter
applies no backpressure and pending sockets are not capped per account: at
most MAX_PRE_ADMISSION_FRAMES frames, and at most
PRE_ADMISSION_QUEUE_BYTES_FACTOR times the message-size cap in bytes. Past
either, the socket closes with WS_CLOSE_POLICY_VIOLATION (1008). A client
whose reopen flush can exceed either bound — a durable queue raised above the
frame cap, or queued requests totalling more than the byte budget — meets that
close.
What a client must not assume. A server-initiated message sent between the handshake and admission is not delivered, just as one sent before the handshake is not. A client that needs to know its connection is live sends a request and waits for the answer.
A closed socket dispatches nothing. A runtime adapter can keep delivering
inbound frames after the server closes a socket, until the close handshake
completes — on Node, for as long as the client withholds its close frame, up
to the ws close timeout. So a server-side close (revocation, cap eviction,
heartbeat timeout, oversized message, refused admission) removes the
connection and aborts its handlers' signal at close time, and frames arriving
afterward are dropped rather than dispatched: a revoked or evicted socket
cannot keep running handlers by ignoring the close. A connection removed from
the transport without a close is ended on its next frame
(WS_CLOSE_INTERNAL_ERROR) rather than left open and unanswered.
What the guarantee rests on. The two halves above leave one schedule uncovered by themselves: a revocation whose close ran before the registration and which commits after the re-read. The close missed the connection and the re-read saw the credential alive. That schedule needs a close fired before its own commit, and no revocation fires one — see Closing on Revocation. A close runs only after its revocation has committed, so a close that missed the registration belongs to a revocation the re-read sees.
What remains.
-
The re-read is one read at open, not a per-message check. From admission on, a connection ends only when a revocation closes it.
-
A close reaches only the connections registered in the process that ran the revocation. The closer is in-memory (see Single-Process Architecture): a second process serving the same database keeps its connections on a credential this one revoked.
-
A commit whose outcome the server never learns closes nothing. The queued closes are dropped whenever the transaction call throws, and a
COMMITthat landed but whose acknowledgement was lost throws like one that failed. The request answers with an error; repeating an admin revoke-all closes the connections (account_session_revoke_allfirst needs a new login, its caller's session having gone with the commit), while repeating a single session or token revoke, or an account delete, finds nothing left to revoke and closes nothing — and the admin revoke-alls refuse an account that is no longer active, so a deleted account's connections stay open until they disconnect. The expired-session sweep is the exception: it closes for the sessions its delete returned whatever became of its commit, since they are expired either way, and the next pass deletes any row the commit left. -
A role revocation closes an audit stream through the audit listener alone, and the listener hears of a row only once its fail-open audit write lands. If that write fails, the role is revoked, the request succeeds, and a stream already open on that role stays open until it is closed some other way. The Rust spine writes that row inside the revocation's transaction, so there the two succeed or fail together.
-
A role that ends any other way closes nothing: a grant reaching its
expires_at, or arole_grantoractorrow deleted through the db-admin browser, whosedb_admin_row_deleteevent declares no revocation scope. The stream stays open until it reconnects; the role gate refuses it then. -
Expiry is not an event, so nothing closes a connection at the moment its credential expires; the re-read refuses an expired credential at open. What happens afterwards depends on the credential:
- A session's connections are closed by the auth cleanup. Each pass
deletes the expired session rows and closes the connections they opened,
after the delete commits (
cleanup_expired_sessions). With the cleanup scheduled —create_app_server'sauth_cleanupoption, orstart_auth_cleanup— a connection outlives its session's expiry by at most one cleanup interval (DEFAULT_AUTH_CLEANUP_INTERVAL_MS, ten minutes) plus the pass itself. The option is off by default: a server that schedules no cleanup closes nothing on expiry, and the connection runs until it disconnects or a revocation closes it. A pass that fails — the database out of reach — is retried on the next interval. - An expired API token's connection is not closed either way. The
sweep does not touch token rows, so the connection stays open until it
disconnects or a revocation closes it —
account_token_revokestill finds the row to close for.
The Rust spine's sweep makes the same closes, and its servers all schedule it.
- A session's connections are closed by the auth cleanup. Each pass
deletes the expired session rows and closes the connections they opened,
after the delete commits (
-
Rotating the daemon token closes nothing: the token has no row and no revocation event, so a connection opened on an earlier token keeps running.
-
A revocation made by code that neither queues a close nor emits an audit event closes nothing — a consumer's own handler deleting a session row, or a direct database write. A consumer handler that ends a credential closes its connections with
queue_connection_close.
A WebSocket and an SSE stream are authorized once, at open, so revoking the credential behind one does nothing to it unless something closes it. Every revocation handler in fuz_app does, and each close runs after the revocation's transaction commits — never before it, and never for a revocation that rolled back. What ends a credential without a handler — expiry, a raw row delete — is under Connection Admission, "What remains".
Two closers, one order.
- The handler. Every handler that ends a credential queues a close on the
request's post-commit queue:
queue_connection_close(ctx, deps.connection_closer, target). The queue runs once the handler's transaction has committed and is discarded when the handler throws. The close does not depend on the audit write, which is pool-routed and fail-open: a failed audit INSERT leaves the revocation in place and the connections closed. Nor does it wait for that write — the queue runs before the request's in-flight pool writes are awaited, so a slow audit INSERT does not hold a revoked connection open. - The audit listeners.
create_ws_auth_guardandcreate_sse_auth_guardrepeat the close when the audit row is announced. The emitter announces a success row from the same post-commit queue, waiting for the write — so a listener's close is post-commit too, and a rolled-back revocation is never announced. A failure row is announced as soon as it is written; the listeners ignore it. Closing is idempotent, so the overlap with the handler's close is a repeat.
What closes.
| Revocation | Closes |
|---|---|
account_session_revoke |
the revoked session's connections |
account_session_revoke_all, admin_session_revoke_all |
the account's connections |
account_token_revoke |
the revoked token's connections |
admin_token_revoke_all |
the account's connections |
account_delete, account_purge |
the account's connections |
POST /logout, POST /password |
the account's connections |
POST /login, when the session cap evicts |
each evicted session's connections |
account_token_create, when the token cap evicts |
each evicted token's connections |
role_grant_revoke |
the account's audit streams gated on that role (listener only) |
A cap eviction writes no audit row, so the handler's close is the only one it
has. A failed revocation (revoked: false — the id was wrong, or another
account's) closes nothing: it carries a caller-supplied id.
Expired sessions. One close is not a revocation. The auth cleanup's
session sweep deletes expired session rows, and a deleted row can no longer be
closed for by account_session_revoke, so the sweep closes the connections of
each session it deletes — by session id, through the same
deps.connection_closer, after its delete commits. It writes no audit row
(nobody performed an expiry), so no listener repeats the close. It runs only
where the cleanup is scheduled (auth_cleanup on create_app_server, off by
default); the bound it gives is under
Connection Admission, "What remains".
One closer for every transport. deps.connection_closer is a
RealtimeCloser: each close fans out to every transport added to it. A
transport, socket, or stream that throws while closing does not stop the
others — every match is attempted, then the first error is thrown and logged.
create_app_backend creates it, and create_app_server adds each WebSocket
transport it mounts (ws_endpoints) and its audit stream registry
(audit_log_sse), so the standard assembly needs no wiring. Assembling by
hand:
register_ws_endpoint/register_action_wsandcreate_audit_log_ssetakeconnection_closeras a required option and add their transport to it.nullis the explicit opt-out.- The action and route factories whose handlers revoke
(
create_account_actions,create_admin_actions,create_standard_rpc_actions,create_account_route_specs) readconnection_closeroff theirdeps, where it is required. A backend with no live-connection surface leaves its closer empty; hand-built deps for the action factories can passnoop_connection_closer. - A consumer's own
SubscriberRegistryjoins withdeps.connection_closer.add(create_sse_connection_closer(registry)).
A socket that revokes itself is answered first. When a WebSocket request
revokes the credential its own socket runs on, the response is sent before the
post-commit queue runs, so the caller reads its reply and then the
WS_CLOSE_SESSION_REVOKED close. The exception is a socket holding
WS_WRITE_HIGH_WATER_BYTES or more unwritten at that instant — a client that
stopped reading, or one just sent a large frame the runtime is still writing: a
reply waiting behind it is discarded with the close, and the caller's request
rejects on the WS_CLOSE_SESSION_REVOKED close as it does on the Rust spine
(Outbound Bounds).
The Rust spine closes after commit at every site as well (SocketRevoker,
queue_socket_close, RealtimeRevoker). It differs on the self-revoking
socket: its dispatch runs the post-commit queue before the socket loop writes
the response, so the caller gets the close without the reply. The revocation
took effect either way; the cross-backend WebSocket suite records the
difference as capabilities.ws_self_revocation_reply.
In-memory sliding window. Applied to login, bootstrap, password change, and signup — the password-bearing surfaces. Bearer auth carries no rate limiter (see below).
- Login IP rate limiter (5 attempts / 15 min) — Per resolved client IP, shared by login + password change (the same instance, mirroring the Rust spine's
login_ip_rate_limiter) - Signup IP rate limiter (5 attempts / 15 min) — Per resolved client IP, signup only
- Bootstrap IP rate limiter (5 attempts / 15 min) — Per resolved client IP, bootstrap only
- Login account rate limiter (10 attempts / 30 min) — Per
account.idwhen the account exists, per normalized submitted identifier otherwise. Peraccount.idon password change. - Signup account rate limiter (10 attempts / 30 min) — Per submitted username (lowercased), signup only
Success clears the account bucket only. A successful login, password change, or signup resets that principal's account-grain counter — forgiving a user's own typos, which is safe because the key is the account being attacked, so clearing it requires that account's credential. Bootstrap is the exception in the other direction: it has no account-grain bucket (it predates any account), so a successful bootstrap clears nothing.
The per-IP buckets are never cleared. They are the distributed-spray backstop: if a success refunded one, an attacker holding any one credential could interleave their own logins with guesses against arbitrary victim usernames and spray indefinitely from a single address — and under open signup the cheapest version needs no credential at all, since creating a throwaway account would zero the budget. Nothing records on an IP bucket on success on any path, so with no reset each is a pure monotone-within-window failure counter that only time clears.
One IP bucket per surface, not one shared across all four. Because they are
monotone, a shared instance would mean a failure on any surface spends the budget that
bounds guessing on every other one — a fumbled bootstrap token could leave the
operator's login budget nearly exhausted on a deployment where their new account
is the only one that exists, and an open-signup bot could deny login to every
user behind its egress. Login and password change still share (password change
is password-bearing on the same account grain, and the Rust spine shares the
same instance across both); signup and bootstrap each get their own. Consumers
wanting one budget across these surfaces pass the same RateLimiter to
login_ip_rate_limiter / signup_ip_rate_limiter / bootstrap_ip_rate_limiter.
Turning a limiter off is loud. An explicit null disables one limiter, and
rate_limiters: 'disabled_for_testing' every one not passed explicitly — for
test apps and test binaries that drive many round-trips from one address
(./usage.md §Rate limiters). Each explicit null (under 'enforced') and the
mode are startup-logged surface warnings, and the
mode also prints a stderr banner a silenced logger can't hide. Nothing refuses
the mode at runtime: the Rust spine keeps its twin out of production by
construction (only its testing_* crates build
RateLimiterMode::DisabledForTesting, and its release audit forbids linking
them), while the TS side has only the warning and the banner.
The 5-attempt cap was deliberately not widened when the buckets became monotone. Widening buys NAT'd-egress headroom by loosening the one bound that caps credential guessing from a single address; splitting the shared bucket buys the same headroom without touching that bound.
Costs to accept, both bounded by the split above but not eliminated by it:
- On a NAT'd egress the IP budget is unforgiving within its window — a colleague's earlier failed logins still count against you, and successes no longer drain the bucket for everyone. Accidental exhaustion is meaningfully more likely than it was under the refund.
- Sustaining a deliberate lockout got cheaper. Under the refund, an attacker
holding an egress at the cap lost the whole bucket the moment any user logged
in successfully; now losing that race costs them nothing, so ~1 failed request
per
window / max_attemptsholds the address at the cap. The refund was never a defense against this (a full bucket refuses the very success that would clear it) — it was an accident that made the attack fragile. - There is no operator "clear this IP" action. Within a window the only remedies
are waiting it out or restarting the process (in-memory state, see
§Known Limitations).
RateLimiter.resetis public, so a consumer can expose an admin action over it — on IP keys that is an operator escape hatch, not the success-forgiveness the method's TSDoc forbids.
Pinned on both spines by the login_security cross-backend suite (interleaved
successes must still 429 within a bounded request budget) and, in-process, by
rate_limiter.handlers.test.ts (login + signup), password_change.test.ts, and
rate_limiter.bootstrap.db.test.ts.
Rate limiter key normalization: Submitted identifiers are lowercased + trimmed
before lookup. When the account exists, the rate limit is keyed by account.id
rather than the submitted identifier — otherwise an attacker could alternate
between the account's username and email (different strings, same account) to
double the per-account bucket. When the account does not exist, the normalized
identifier itself is the key (keeps case variants alice / Alice / ALICE in
the same bucket).
Enumeration prevention: Both failure paths (account not found, wrong password) record equally on the per-account limiter. The 429 response shape is identical across paths, even though account-found buckets and account-missing buckets differ. A regression test verifies the response is indistinguishable.
Ordering: IP rate limit check runs first — blocked requests pay no DB or
crypto cost. Per-account rate limit check runs after an indexed
account_by_username_or_email lookup so the bucket can be keyed by canonical
account.id. Blocked requests therefore still pay one indexed DB lookup, but
skip the expensive Argon2 verification. This is a deliberate tradeoff to close
the username/email alternation bypass.
Client IP is resolved by trusted proxy middleware (see Trusted Proxy) before rate limiting.
Cross-impl parity: the per-IP login 429 + Retry-After shape is pinned
over the wire on both the TS spine and the Rust spine by the login_security
cross-backend suite — the first request past the per-IP cap on one forwarded IP
returns 429 {error: "rate_limit_exceeded", retry_after} with a
Retry-After: ceil(retry_after) header on both impls (rate_limit_exceeded_response
↔ route_response::rate_limit_exceeded).
An API token is 32 bytes of CSPRNG output (secret_fuz_token_ + 43 base64url
chars) resolved by a blake3 hash lookup. Guessing one is bounded by entropy, not
by throttling: even ignoring every edge control, the residual search space after
subtracting the publicly-visible token id prefix is ~184 bits. A limiter here
changes an unreachable number into a slightly larger unreachable number.
It is not free, though. The check/record must precede the async token lookup to close its own TOCTOU window, so a burst of concurrent requests carrying a valid token records against itself and the last one 429s — an availability bug for exactly the automation bearer auth exists to serve (CI runners, a NAT'd office egress). The limiter also short-circuits ahead of the RPC dispatcher, so a throttled RPC call would answer with a REST-shaped error instead of a JSON-RPC envelope.
Rate limiting belongs on low-entropy credentials — passwords — and on bounding attacker-controlled writes. The Rust spine never had a bearer limiter; this is the converged shape on both.
- Check-then-record race: the login handler uses check-then-record —
check(ip)is sync; async Argon2 work follows (~100ms). Concurrent requests from the same IP may all pass the check before any records. Practical impact: up tomax_attempts + N_concurrentmay pass per window. Single-process architecture limits concurrency. The alternative (record-before-validate) closes the window but throttles concurrent legitimate callers, which is why bearer auth dropped its limiter rather than keeping that shape. - Blocked requests don't extend lockout: A 429 response calls
check()but notrecord(). Continued abuse during lockout doesn't extend the window. - Single-process: See Known Limitations.
The in-memory rate limiter is designed for single-process deployments. In horizontally-scaled setups, each process maintains independent counters — an attacker distributing requests across N instances can attempt N × max_attempts per window.
Mitigations for multi-process deployments:
- Reduce window sizes as compensation (e.g., 3 attempts / 10 min instead of 5 / 15 min) to limit the effective multiplier
- Rate limit at the reverse proxy (nginx
limit_req) for IP-based limiting — this is shared across all backend instances and handles the common case - Use a shared store (Redis, DB table) for application-level rate limiting when proxy-level limiting is insufficient (e.g., per-account limiting)
For v1 single-process deployments, the in-memory limiter is sufficient.
create_app_server applies a body size limit (http/body_limit.ts) before
auth and route handling. Default: 1 MiB (DEFAULT_MAX_BODY_SIZE). Oversized
payloads are rejected with 413 and {error: 'payload_too_large'}
(PayloadTooLargeError schema). Configure via max_body_size on
AppServerOptions; pass null to disable.
A Content-Length the cap cannot read. A Content-Length that is not a
plain run of ASCII digits — abc, -1, +5, 1e9, 0x1F4, an empty value,
a combined 5, 500 — is refused with 400 {error: 'invalid_content_length'}
and Connection: close, before auth and without reading the body. A length
that does not parse cannot be compared to the cap, and Hono's bodyLimit
passes such a request on without counting its body. A digit run of any length
compares as a number, so one far past what a number holds exactly is over the
cap (413). The runtimes' HTTP parsers answer their own bodiless 400 to most of
these before the application runs, so the check is defense in depth where
they do and the refusal where they do not:
- Node refuses every form above, and a repeated
Content-Lengthheader whether or not the values agree. - Deno and Bun refuse them too, except a repeated header with equal
values, which reaches the application as one combined value (
50, 50) — refused here. - The Rust spine (hyper) refuses a value that is not digits and a repeated header with differing values at parse, and accepts a repeated header with equal values as that value. It has no application-level check, and needs none: no unparsed length reaches its limit layer.
Observed on Node 24.14.1, Deno 2.8.3 and Bun 1.4.3.
A request carrying both Transfer-Encoding and a well-formed Content-Length
is measured by the bytes that arrive, since the declared length does not frame
a chunked body. The Node, Deno and Bun parsers refuse such a request with
their own 400. hyper accepts it: it removes the Content-Length, whichever
header came first, frames the body as chunked, and closes the connection after
its response, so the Rust spine's limit layer sees no declared length and
counts the bytes that arrive (hyper 1.11.1 or later — an older hyper left a
Content-Length that preceded Transfer-Encoding on the request and kept the
connection). The status differs by backend; the outcome does not — the
connection closes and a request pipelined behind it is never served, which the
cross-backend suite pins.
Rejection and the connection. A request whose Content-Length exceeds the
cap is rejected on the header, before the body is read, and the 413 carries
Connection: close. The defense-in-depth posture is to close the connection
without reading the body: an unread request body cannot safely share a
keep-alive connection, because the leftover bytes could be parsed as the start
of the next request (request smuggling) — and a server that instead reads past
the body to reuse the connection takes up to the attacker-declared length off
the socket. What the connection does after the 413 is up to the runtime, and
the runtimes differ:
- Node (
@hono/node-server) closes at once, with the body unread. TheConnection: closeheader is what makes it: a 413 without one leaves the adapter holding the connection for a drain window, after which it closes — or, when nothing touched the request body and the rest of it arrives inside that window, reads it off and serves the next request. - Bun closes at once too, with the body unread, honouring the header. Bun 1.4.3 is the tested floor: an older Bun kept the connection after the 413 and served the next request (correctly framed), so run Bun deployments, and the Bun leg of the cross-backend suites, at the floor or later.
- The Rust spine (hyper) closes at once, without waiting for the body.
- Deno closes only once the whole declared body has arrived. The 413 goes
out at once, a request pipelined behind the body is never served, and a
client that withholds the body holds the connection open. Observed on Deno
2.8.3; neither the
Connection: closeheader nor anything the handler does with the request body changes it.
No runtime serves a request behind one refused on its declared length. What differs is who reads the body: on Deno the cap bounds what the application processes, not what the transport reads off the socket — it takes the declared body before it closes. A reverse proxy with its own request-body cap refuses an oversized request before it reaches the runtime.
The limit checks Content-Length first and answers from the header alone,
falling through to Hono's bodyLimit only for a request with no
Content-Length, or with one beside Transfer-Encoding. bodyLimit measures
the bytes that arrive: it reads the body into memory, up to the cap, before the
next middleware runs. Its 413 is sent once more than the cap has been read,
and carries no Connection: close — what the connection does then is the
runtime's default. http/body_limit.test.ts pins each branch.
What the two spines share, and what each decides. Pinned on every backend
by testing/cross_backend/body_size.ts: a declared Content-Length over the
cap answers 413 whatever the request's Origin or credentials, and an
authenticated, allowed-origin request whose chunked body runs past the cap
answers 413. Neither refusal waits for the rest of the body: over a real
socket the suite sends the declared-length request's head alone, and the
chunked body only as far as one byte past the cap, and requires the 413.
Implementation-defined:
- An over-cap body with no declared length, when another refusal also applies. The TS spine has read the body up to the cap before any later middleware runs, so its 413 comes ahead of the origin check, auth and the handler. The Rust spine counts the body as the handler reads it: the origin check answers its 403 on the headers; a handler that never reads the body answers its own status, and hyper closes the connection rather than read the rest; a handler that buffers the body answers 413. Nothing past the cap reaches a handler on either.
- A repeated
Content-Lengthwith equal values. The TS spine answers 400invalid_content_lengthwhere the runtime hands the repeat to the application (Deno, Bun; Node's parser refuses it first). hyper folds the repeat into one value before any layer runs, and the request is accepted.
The cross-backend suite
(testing/cross_backend/body_size_smuggling.ts) pipelines a request behind an
oversized one and requires at most one response from every backend — the
connection closes and the pipelined request is never served — and
server/create_app_server.db.test.ts pins that the reject never reads the
body.
The cap is global, not per-route. The limit is mounted once for the whole
app, so max_body_size (and null) apply to every route — there is no
per-route override. A route that legitimately needs large bodies (e.g. content
uploads) should not be served by raising or disabling the global cap: that
strips the protection from every other route, including the RPC surface,
re-opening an unbounded-body memory-exhaustion surface. Instead, give such a
route a bounded streaming handler that consumes the body incrementally and
enforces its own ceiling (the fact store's put_stream is the in-stack
example), and keep the global cap in place for everything else.
Why streaming, not a larger cap. For a request carrying Content-Length
the middleware rejects on the header and never buffers. For a chunked request
(no Content-Length) it reads and buffers the body up to the cap before
forwarding it — harmless at 1 MiB, but a cap raised to upload sizes would let a
chunked upload buffer that much per request. A streaming handler avoids the
buffer entirely, which is the other reason large uploads belong on their own
streaming route rather than under a wider global cap.
WebSocket messages. The body limit doesn't see WebSocket frames, so
register_action_ws caps each inbound message separately at
DEFAULT_WS_MAX_MESSAGE_BYTES (1 MiB, the same bound as the HTTP body cap;
max_message_bytes on WsEndpointSpec to change it) and closes the socket
with 1009 on a larger one — no per-message error reply, and the Rust spine
likewise ends the connection. The check runs after the
runtime adapter has assembled the message, so it bounds dispatch work, not
what the adapter buffers — where memory matters, also set the adapter's own
frame limit (e.g. the ws library's maxPayload, which defaults to 100 MiB).
What the server sends a socket is bounded separately — see
Outbound Bounds.
Roles are Zod-validated at I/O boundaries via create_role_schema() — not stored
in a DB table. Built-in roles: keeper (system-level) and admin (app-level).
Consumer apps extend with app-defined roles at server init; unknown roles are hard
rejections. A role name is lowercase letters and underscores with no leading or
trailing underscore, at most 64 characters (RoleName, ROLE_NAME_LENGTH_MAX):
a registered role outside that grammar throws at assembly, and a role param
outside it is refused at input validation, before any gate that would record
the caller's text in a failure audit row. The Rust spine holds the same grammar
(is_valid_role_name).
The other caller-supplied strings that reach audit metadata are length-capped
at input validation too, on both spines, counted in UTF-16 code units: an API
token's name (API_TOKEN_NAME_LENGTH_MAX, 255), a cell's kind on
cell_create and as a cell_list filter (CELL_KIND_LENGTH_MAX, 64), and
the scope_kind on role_grant_offer_create and role_grant_assign
(SCOPE_KIND_LENGTH_MAX, 64). A longer value is refused invalid_params
before any row is written. Every .min() / .max() bound on a wire string,
a bare .min(1) aside (it reads the same in either unit), is enforced on
String.length by bounded_string (primitive_schemas.ts), not by Zod's
.max() alone, whose unit depends on the installed Zod; Email is bounded in
UTF-8 bytes by its own refinement.
| Role | Granted how | Scope |
|---|---|---|
keeper |
Daemon token only (filesystem access required) | System-level: role_grants, audit, bootstrap recovery |
admin |
CLI or web (by keeper) | App-level: users, content, config |
| App-defined | Web (by admin) | App-specific (teacher, approved, etc.) |
Role grants vs flags: Every capability comes from a time-bounded, revocable role_grant
with a granted_by field. No role_grant = no capability (safe by default).
Grant authority enforcement: The admin-grant-path gate
(RoleSpec.grant_paths includes 'admin') is checked server-side on every
web grant request — the offer-create flow and the immediate role_grant_assign
alike. Direct API calls respect the same restrictions as the UI.
Keeper's grant_paths is ['bootstrap'] (no 'admin'), so it cannot be
granted via web.
Admin self-replication: The admin role is self-replicating — any admin can
grant admin to another user. The admin-grant-path gate prevents admin from
granting keeper, but does not prevent admin-to-admin grants. For deployments
where admin self-replication is undesirable, implement app-specific role
hierarchy checks in a custom grant guard.
Account-removal target guards: account_delete (soft) and account_purge
(hard) refuse two target classes regardless of caller privilege, each emitting a
fail-loud failure-audit row before any mutation. ERROR_CANNOT_DELETE_KEEPER
(403) blocks any account holding an active keeper role grant — auth resolution
and daemon-token resolution both pivot on the keeper account, so removing it
would brick keeper/daemon auth with no recovery (keeper is non-web-revocable and
purge itself requires keeper auth); keeper removal stays out-of-band.
ERROR_CANNOT_DELETE_LAST_ADMIN (403) blocks the sole remaining active admin —
the admin count and the target's own admin check both filter deleted_at IS NULL,
so a soft-deleted admin can't authenticate, doesn't count toward the tally, and
can itself be removed once another active admin exists. The keeper guard is a
hard lockout stop; the last-admin guard is keeper-recoverable (a keeper can
re-grant admin) and intended only to stop a one-call foot-gun.
IDOR guard: query_revoke_role_grant() requires an actor_id constraint. The
revoke handler resolves the target actor from the URL and returns 404 on mismatch —
a handler cannot revoke a role_grant belonging to a different actor. The same
404-over-403 pattern applies to query_accept_offer, which throws
RoleGrantOfferNotFoundError on both a missing offer id and a wrong-recipient
lookup to avoid disclosing whether an offer id exists.
The 404 mask is scoped to cross-principal disclosure. An offer that exists
on the caller's own account but is targeted to a different actor (a sibling
persona) is refused with 403 role_grant_offer_actor_mismatch, not masked to
404 — every actor in that distinction belongs to the one account already
authenticated, so the 403 leaks nothing across a trust boundary, and masking it
would only obscure a legitimate "not yours to accept, pick the right persona"
signal. The cross-account lookup (an offer on another account) stays 404. Both
arms hold identically on the TS and Rust spines
(role_grant_offer_enumeration.cross.test.ts).
404-over-403 is the general mask, not a per-handler quirk. Any
resource-scoped lookup where existence itself is privileged returns the
same not_found shape for "no such row" and "exists but the caller
can't view it" — never a forbidden that would confirm the id. Beyond
role_grants/offers above, the cell surface applies it uniformly:
cell_get / cell_update / cell_delete / cell_clone all return 404
(ERROR_CELL_NOT_FOUND) when the caller fails the verb's own authorization
predicate — can_view_cell for the reads (cell_get / cell_clone),
can_edit_cell for the writes (cell_update / cell_delete) — identical
to a genuine miss (auth/cell_actions.ts). The uniform property is the
wire shape, not a single shared predicate. A caller therefore cannot
enumerate private cells (or offers, or another actor's grants) by id —
found-and-unauthorized and not-found are wire-indistinguishable.
Cross-impl parity: the 404 masks hold on both spines — the cell 404
(cell_get / cell_update / cell_delete / cell_clone) and the
role-grant / offer 404 return the same wire shape on the TS spine and the
Rust spine via the cell + conformance cross suites. The in-process cell suite
(cell_actions.authz.db.test.ts) additionally pins the mask
byte-identical across an unviewable cell, a view-but-not-edit cell, and a
genuine miss — so no distinguishing field (an id echo, a divergent message)
can creep into one path.
Phase ordering hides route shape from callers who lack the authority to
call. Every dispatch surface (HTTP RPC, the REST bridge, WS) runs 401 →
authorization phase → 403 → 400 → handler: authentication, then actor
resolution, then the credential / scope / role gates, and only then input
validation. A caller any of those refuse never learns the route's input
schema or shape from a parse error (a 400 leaking required fields / types) —
a 400 would confirm the method exists and describe how to call it, to a
channel that should have learned neither. Coarse authority facts before fine
ones, and no shape disclosure before authority is settled: the same argument
that puts the credential gate ahead of the scope gate. Both spines implement
the same order (the Rust spine validates input handler-side, which is what
puts it last there). Pinned by
account_actions.credential_gate.db.test.ts §authority gates precede input
validation and by a cross-backend conformance row. (Module detail:
http/CLAUDE.md §Validation pipeline; auth/CLAUDE.md §Two-phase identity.)
Role-shaped cell grants validate against the registered role set. A
cell_grant can name either an actor ({actor_id}) or a role
({role, scope_id?}); a role-shaped grant admits every actor holding that
role (scope-matched). Because a grant for a role no actor can ever hold would
be silently inert — looking like access was conferred while admitting no
one — cell_grant_create rejects an unregistered role at create time with
invalid_params / cell_grant_unknown_role rather than writing a dead row.
The check runs after the manage-tier authorization (a non-manager still gets
the 404 mask, not a role-registry probe). Actor-shaped grants whose principal
is the cell's own owner are likewise rejected (cell_grant_principal_is_owner)
since owner access is implicit.
Actor-search scope gate: actor_search (the prefix-search picker over
actor.name) is account-grain authenticated, but an unbounded global
search is admin-only. A non-admin caller must pass at least one scope_id;
results are then filtered to actors holding an active role_grant on one of
those scopes (active = revoked_at IS NULL AND (expires_at IS NULL OR expires_at > NOW()), so a revoked or expired membership no longer confers
search visibility). Omitting scope_ids as a non-admin is rejected with
invalid_params / actor_search_scope_required — not an empty result, so the
gate is a hard boundary rather than a silent filter. The admin check is
account-grain (any actor on the caller's account holding a global admin
role_grant). The gate keys on scope_ids presence, not authority over the
scope: a caller who already knows a scope_id can scope a search to it, but
the cap (ACTOR_SEARCH_LIMIT_MAX) plus the per-account rate limit bound the
batched-enumeration surface either way. The sibling actor_lookup (batched
id → label) needs no scope gate — it resolves only ids the caller already
holds, and its wire rows omit account_id / email / timestamps / role
state (control-plane and timing-oracle avoidance).
Duplicate prevention: A partial unique index on
(actor_id, role, COALESCE(scope_id, sentinel)) prevents duplicate active
role_grants per resource scope (global role_grants collapse via the sentinel uuid).
query_create_role_grant() is idempotent (ON CONFLICT DO NOTHING).
Consent as an authorization property: fuz_app treats explicit recipient consent as a security property of web-path role grants, not a UX nicety. Missing consent is an authorization bug, not a missing button.
Two code paths, one invariant:
- Direct grant (
query_create_role_grant) — the low-level write, with no consent step of its own. Two kinds of caller reach it. The process-gated ones are where a reaching-for-consent step would itself be the vulnerability: keeper bootstrap (bootstrap_account.ts), the dev/setup seeding path, and test fixtures — filesystem- or process-gated, not network-reachable. But two network RPC actions also call it directly:self_service_role_set(self-target only, so the grantee and the caller are the same actor — consent is structural rather than a separate step) androle_grant_assign(admin-only, described under Immediate assign below). So "direct grant" names the write, not an off-network boundary; the authorization property comes from each caller's own gate. - Offer flow (
role_grant_offer_create→ recipientrole_grant_offer_accept) — the consentful path. The admin UI drives the samerole_grant_offer_createRPC action as any other grantor; a role_grant row only exists after the recipient atomically accepts. Consumer-app role grants where the recipient should opt in (classroom membership, future workspace invites) use the offer flow. - Immediate assign (
role_grant_assign) — the admin-only direct web path, for a capability unlock where waiting on consent is the wrong model ("you may now post"). It runs the same admin-grant-path gate and the same idempotent write as the offer flow, but skips the offer — no row for the grantee to accept. Its safeguards are admin-only conferral (no holder-propagation — holding a role confers no power to hand it out) and arole_grant_createaudit row, not recipient consent.
Why the split matters for security:
- No unsolicited membership via the offer flow. When a role models
membership the recipient should agree to, exposing it through the offer
flow guarantees a web-path admin cannot draft an account into the group
without the recipient observing and agreeing. Closes a class of
social-engineering attacks where a compromised or hostile admin drafts an
account into a group the account owner has no awareness of ("silent
membership in a private workspace", "drafted teacher of a classroom you've
never heard of").
role_grant_assigndeliberately trades consent for immediacy (the capability-unlock model above), so it is not the path for membership-shaped roles — its boundary is admin-only conferral + audit, an explicit admin action, never a peer or self-service one. - The recipient sees the grant before it takes effect. The
role_grant_offer_receivednotification plus the persistent inbox give the recipient an audit-visible, client-visible record of every pending offer. A surprise role_grant in the wild is a bug. - Keeper-path stays direct by design. Keeper-level operations already require filesystem access (daemon token), so the operator is already privileged — waiting on consent to recover a locked-out instance would be worse, not better.
Admin surface hardening on the offer flow:
- Admin-grant-path gate runs before the offer insert, same as the
previous direct-grant route —
keeper'sgrant_pathsdoes not include'admin', so it cannot be offered via the web. - Self-target rejection (400
role_grant_offer_self_target): the offer query rejectsfrom_actor.account_id == to_account_id. Under the previous direct-grant route an admin granting themselves was a silent idempotent no-op; the offer route surfaces it as an explicit error and emits arole_grant_offer_create outcome=failureaudit event symmetric with the admin-grant-path andauthorizedenial paths, so self-grant probes leave a trail. - Builtin roles are global-only, and every builtin gate reads the global
grant. The dispatcher's role gate, the cell admin bypass, the bare-hash
fact read, cross-account
account_delete, and the offer inbox reads all checkhas_scoped_role(_, 'admin', null)— a scopedadminrow confers nothing. No grant path mints one:role_grant_assignandrole_grant_offer_createrefuse a scopedadmin(400role_grant_builtin_scoped;keeper, scoped or not, stays 403 as never web-grantable), offering a builtin needs a global-admin caller whatever the consumerauthorizecallback says (403role_grant_offer_not_authorized), and accept refuses akeeperoffer, a scopedadminoffer, or a builtin offer whose grantor no longer holds an active globaladmin— re-checked underFOR SHAREat mint time, the refusal audited as a failedrole_grant_offer_acceptand the offer left pending.role_grant_assignalso re-checks the global admin grant in-handler behind the dispatcher gate, so a scopedadmincan't assign itself a global one. Self-service eligibility refuses builtin roles at construction. - Admin retract via RPC, grantor-scoped — admins cancel offers they
issued by calling
role_grant_offer_retractthrough the RPC surface. The grantor IDOR guard (from_actor_id = ctx.actor.id) enforces that one admin cannot retract another admin's in-flight offer, so the original protection holds. Retract emitsrole_grant_offer_retractaudit +role_grant_offer_retractedWS notification to the recipient.
Scope and message on the admin route are intentionally omitted from the
input — admin-path offers are always global (scope_id = null) and
carry no grantor note. Scoped / messaged offers travel through the
consumer RPC surface (role_grant_offer_create), where the consumer's
authorize callback can impose tighter policy than the admin-grant-path gate alone.
See ./identity.md §Direct grant vs offer flow for the data-layer description and audit-event chain.
Consentful grants + revoke-bypass defense: Web-path role grants flow
through role_grant_offer — the recipient must explicitly accept. Multiple
grantors may have coexisting pending offers for the same
(recipient, role, scope). A fourth terminal state, superseded_at,
closes the revoke-bypass path:
- On accept of offer A, all sibling pending offers for the same
(to_account, role, scope)are marked superseded in the same transaction, with arole_grant_offer_supersedeaudit event (reason: 'sibling_accepted',cause_id: A.id). - On revoke of a role_grant, every pending offer for the revoked
(actor's account, role, scope)is marked superseded in the same transaction (reason: 'role_grant_revoked',cause_id: <revoked role_grant id>). - On parent-scope cascade (consumer's polymorphic
scope_idrow deleted viaquery_role_grant_revoke_for_scope), every pending offer at the scope is marked superseded — tuple-matched and orphan, undifferentiated — withreason: 'scope_destroyed',cause_id: <destroyed scope row id>.
Net property: accepting a pending offer means the offer survived every
revoke between its creation and acceptance. An attacker cannot accept a
stale pre-revoke sibling to restore a just-revoked role_grant — any such
sibling was marked superseded when the attacker's companion role_grant was
first accepted or when the admin revoked. A fresh post-revoke grant
requires the grantor to call query_role_grant_offer_create again, which is
audited.
Post-commit WS fan-out: six JSON-RPC notifications (role_grant_offer_received
/ _retracted / _accepted / _declined / _supersede + role_grant_revoke)
ship alongside the above via NotificationSender.send_to_account, scheduled
through emit_after_commit so sends are strictly post-commit and discarded if
the handler's transaction rolls back — a rolled-back transaction cannot leak a
notification for state that never committed (distinct from the eager
pending_effects audit-attempt queue, which intentionally survives
rollback). The two-sided contract and its dispatcher-level enforcement are
documented at ./architecture.md §Fire-and-Forget Pending Effects. Exceptions
inside a send are caught and logged — one failed send cannot corrupt the
already-committed response or starve sibling sends in the same batch. Sends are fire-and-forget
with no delivery receipt: send_to_account returns the socket count but a
non-zero count only means ws.send didn't throw; flows that need durable
delivery must persist the event and hydrate on reconnect. Payload sizes are
bounded at the schema layer — offer.decline_reason at
ROLE_GRANT_OFFER_MESSAGE_LENGTH_MAX (500), role_grant_revoke.reason at
ROLE_GRANT_REVOKED_REASON_LENGTH_MAX (500) — so an untrusted input path
cannot balloon a payload.
Facts are immutable, content-addressed bytes (blake3 hash → bytes), stored in
the optional fact layer and referenced by cells through cell.refs
(auto-extracted blake3: hashes from cell.data). Because facts are
content-addressed, identical bytes from different owners deduplicate to one
fact row — there is no per-fact owner. Authorization therefore lives on the
cell→fact edge (this owner references these bytes from this cell), never on
the anonymous, owner-less hash.
Cell-scoped, per-reference reads. Fact bytes are served through a named referencing cell:
GET /api/cells/:cell_id/facts/:hash
The check is can_view_cell(caller, cell) AND cell.refs includes hash — scoped
to that one reference, never unioned across the fact's other referrers.
A caller who can view a cell that references the bytes reads them through that
cell; a caller who cannot view a given cell cannot read the fact through it,
regardless of who else references the same bytes.
Dedup has zero authorization consequence. Whether two owners' identical
bytes share a fact row is invisible to the read check. If owner A references
bytes from a private cell and owner B references identical bytes from a
public cell, the two are one fact row, but:
- A viewer of B's public cell reads the bytes through B's cell — correct, B published them.
- The same viewer cannot reach A's private reference: naming A's cell fails
can_view_celland returns 404. A's stated privacy intent holds despite the shared storage row.
Content-dedup is a pure storage optimization; it can never widen a fact's effective visibility. This closes the cross-owner leak that a bare-hash, union-of-referrers read model would otherwise create (A's private bytes becoming world-readable the instant B publishes identical bytes).
Served bytes are inert. Fact bytes are caller-supplied content served back
over the app's own origin, so every fact response — on all three serve paths —
carries X-Content-Type-Options: nosniff and
Content-Security-Policy: default-src 'none'; sandbox
(server/serve_fact_route.ts). The CSP denies every fetch and puts the
response in a unique opaque origin, so a stored HTML/SVG payload cannot execute
as same-origin script or reach back into the app. Fact responses also carry
Cache-Control: private, max-age=300 — a deliberate 5-minute window in which a
client may reuse bytes whose authorizing cell grant has since been revoked.
404-mask. An unviewable cell, a missing cell, and a cell that doesn't reference the requested hash all return the same 404 — never 403, never a response that distinguishes "this fact exists elsewhere" from "no such edge." Neither the existence of a fact nor the existence of a cell→fact edge leaks through the public surface. This is the same 404-over-403 mask the cell surface uses (see Authorization, "404-over-403 is the general mask").
The bare-hash endpoint is admin-only.
GET /api/facts/:hash # admin only
An admin's reach already spans every cell, so serving by bare hash grants an admin no escalation — the union concern that makes bare-hash reads a cross-owner leak for non-admins is vacuous for an admin. Non-admin callers are rejected at the auth phase (401 unauthenticated / 403 wrong role) and never reach the handler; the handler additionally re-checks the admin role as fail-closed defense-in-depth. Confidential non-admin reads always go through the cell-scoped route. (An "explicitly-public fact" concept — a producer opting specific bytes into world-readable status — is a possible future refinement; there is no such concept today, so the bare-hash route stays strictly admin-gated.)
External-fact path validation. External (filesystem-backed) facts are
served via an X-Accel-Redirect into nginx's internal facts location
(production) or a disk stream (dev/tests). The stored external_url is
re-validated against the canonical file:<shard>/<rest> shape before it is
trusted to address the filesystem, even though the write path only ever emits
that shape — defense-in-depth against a future row-injection bug handing nginx
an attacker-controlled path.
X-Accel facts location must be internal; (fail-loud at boot). The
X-Accel-Redirect path's confidentiality depends on the nginx facts location
being marked internal; — only the authz'd handler's internal redirect may
reach it. A public facts location would serve any fact's bytes to anyone who
guesses the <shard>/<rest> path, bypassing every cell-visibility check. The
redirect prefix is therefore gated behind a validated XAccelConfig
(server/x_accel.ts): it can only be constructed via create_x_accel_config,
which runs validate_facts_internal_location and throws at boot on a missing or
non-internal; location — so X-Accel serving is impossible to enable without
proving the location is internal. This is a best-effort string check, not a full
nginx parser. It is the only nginx property fuz_app validates — see
§Config Validation.
Account creation is invite-gated by default. When open_signup is enabled
(via app_settings), anyone can create an account without an invite. App
settings are admin-only: both reading (app_settings_get) and changing the
toggle (app_settings_update) require the admin role, and a change is
audit-logged as app_settings_update. The signup handler reads the toggle
fresh from the database on every request, so the value stays consistent
across multiple server processes. Existing per-IP and per-account rate
limiters apply to open signup — no additional rate limiting configuration is
needed.
When invite-gated, admins create invites; signups are matched
against unclaimed invites before account creation proceeds. Signup conflicts
(username or email already taken) return a single generic signup_conflict error
to prevent account enumeration — the response does not reveal which field collided.
Atomic find + claim under row lock: The unclaimed-invite lookup runs
inside the signup transaction via query_invite_find_unclaimed_match_for_update
(SELECT … FOR UPDATE). Concurrent signups matching the same invite
serialize on the row lock; the loser observes a committed claimed_at
on retry, falls through find_for_update's no-row result, and gets
ERROR_NO_MATCHING_INVITE. There is no race window between find and
claim — the win is architectural, not dependent on the
case-insensitive account uniques shadowing a TOCTOU.
Constant-time floor on denial paths: 403 (no_match) and 409
(signup_conflict) responses elapse at least
DEFAULT_SIGNUP_FAIL_FLOOR_MS (250ms) ± DEFAULT_SIGNUP_FAIL_JITTER_MS
(25ms) via the same Promise.all(work, setTimeout) pattern login uses.
Without the floor, an attacker can distinguish no_match (~5ms, bails
before tx) from signup_conflict (~50ms+, Argon2id + tx + rollback) by
response time and use the gap as a username-enumeration oracle. The
Argon2id hash now runs before the tx so legitimate denial paths
unconditionally pay the hash cost (bounded by rate limiters), which
also flattens the timing budget the floor has to cover. 429 (rate
limit) stays fast — same precedent as login.
Under open_signup: true, success (~150ms with Argon2id + tx + session
create) is still measurably faster than signup_conflict
(max(work, 250ms)), so a username-existence oracle survives in
open-signup mode. Considered acceptable because open-signup defeats
invite-gated enumeration anyway — an attacker can sign up under any
free username to confirm a target is not taken, and the conflict
response confirms the rest.
Fallback failure audit on internal errors: Tx-rollback paths that
aren't classified as NoMatchingInviteError or PG unique violation
(Argon2id fault, session-create error, DB outage mid-tx) emit an
outcome: 'failure' audit row with reason: 'internal_error' before
the rethrow propagates. Operators investigating "why did signup 500
for user X at time Y" have the audit row to start from instead of
silence.
Case-insensitive username uniqueness: A LOWER() unique index on
account.username prevents case-variant duplicates (alice vs Alice).
find_by_username uses case-insensitive matching. The original TEXT UNIQUE
column constraint coexists — the LOWER() index is strictly more restrictive.
Three-mode invite matching: find_unclaimed_match uses a single SQL query
with three disjoint modes based on which fields the invite has:
- Email-only invite (email set, username NULL) — matches only if signup provides matching email. Cannot be claimed by username match alone.
- Username-only invite (username set, email NULL) — matches by username.
- Both-field invite (both set) — requires BOTH email and username to match. Opt-in stricter defense for admins who want to pin an invite to a specific person.
Invite creation guards: Creating an invite for a username or email that
already has an account returns 409 with per-field errors
(invite_account_exists_username, invite_account_exists_email). This
prevents dead-on-arrival invites, and both checks are case-insensitive.
Username validation: The Username Zod schema (3-39 chars, starts with
letter, ends with letter/number, middle allows dash/underscore, no @ or .)
is enforced on both signup and invite creation inputs. Email uses z.email()
which requires @ — the two namespaces are disjoint.
No email ownership verification at signup: Signup does not verify that the
user controls the email they provide. An email-only invite for alice@example.com
can be claimed by anyone who knows the address. The invite proves the admin's
intent, not the claimant's identity. Username-only invites have the same
property — they reserve a name, not a person. Both-field invites are strictly
stronger (require knowing both values) but still don't prove ownership.
Email verification is a separate, deferred step — once implemented, accounts
with verified email will require login codes, and the email_verified flag
(already in the schema) will gate sensitive operations.
Future: per-account login method control: app_settings could define the
instance-wide default for whether password login is enabled (e.g.
password_login_enabled: true). Individual accounts would override via a
per-account setting (column on account or separate account_settings table).
This enables progressive hardening — an instance can default to email-only
login once email auth is implemented, while allowing specific accounts to
retain password login during migration. The app_settings value sets the
default for new accounts; existing accounts keep their current setting.
Three layers, because no one of them covers every page:
SameSite=Strict session cookies — the browser won't send the session
cookie on a cross-site request. Cross-site is not cross-origin: a site is
the scheme plus the registrable domain, and the port doesn't count, so a page
on http://localhost:8000 is same-site with a server on
http://localhost:4460 and its requests carry that server's cookie. On most
hosts, a page on a sibling subdomain is same-site too. For a locally-running
server, that means every other local dev server, notebook, or
python -m http.server page.
JSON-only POSTs (http/simple_request.ts) — every POST route that
isn't raw_body requires Content-Type: application/json, bodyless or not
(logout), and a raw_body route refuses a POST with no Content-Type, or
a text/plain, form-encoded, or multipart one — 415
unsupported_content_type, before reading a credential or the body. Those
are the requests a browser sends cross-origin without a preflight, cookie
attached. Any other type makes the browser send an OPTIONS
preflight first, and fuz_app answers none with CORS headers, so the request
is never sent — whatever the origin allowlist says. PUT, PATCH, and
DELETE are always preflighted. A consumer that adds CORS middleware
granting cross-origin requests reopens this. Every fuz_app client sends
application/json. The Rust spine checks the same in each JSON handler.
Origin verification (origin.ts) — an allowlist that rejects requests
from disallowed Origin headers before any handler runs. It is what guards
the WebSocket upgrade, which is never preflighted, so list exact origins: a
port wildcard (http://localhost:*) admits every same-site local page to the
socket, cookie included. It also protects locally-running services from
being called by untrusted websites as the user browses the web. Origin-only:
the Fetch spec mandates Origin on every unsafe method, so the
Referer-fallback arm was inert on modern browsers and was dropped to
converge with the zzz_server Rust port. Non-browser clients (curl, CLI,
server-to-server) without an Origin header pass through — token auth is
the security control for those callers, which don't carry auto-attached
cookies.
So a same-site page, which does get the cookie sent, still can't drive the
server: its POSTs are refused for their content type or never sent, and its
socket upgrade for its origin.
Browser/CLI split: Bearer and daemon tokens are silently discarded when
Origin or Referer headers are present — browsers must use cookie auth. The
middleware calls next() without setting a context, so public actions still
work even with stale credentials. This reduces the attack surface: a stolen API
token cannot be replayed from a browser context. Daemon tokens carry the same
guard for symmetry — they are loopback-only and never legitimately carry an
Origin, so a header-bearing daemon token is dropped (and the request falls
through to credential_type_required downstream rather than a hard fail).
External traffic reaches the API over both cookie and bearer auth. The
reference deployment recipes forward Authorization on the general /api
block, and every deployed consumer builds its site from those recipes. This
section describes that posture and names what it costs, because an earlier
version of this document recommended the opposite ("strip Authorization at
nginx, cookie-only for v1") while no deployment ever did it.
What is actually in force:
- Browser callers — cookie auth. A bearer or daemon-token credential
presented alongside an
Origin/Refererheader is silently discarded by the browser-context guard (see Browser/CLI split above), so a token exfiltrated by XSS cannot be replayed from the page that stole it. - Non-browser callers — bearer auth works through the proxy. This is load
bearing:
fuzf's repo commands, and any server-to-server client, authenticate this way.
What that costs. The browser-context guard closes browser XSS exfiltration.
It does not close replay of a token stolen from CI, an env var, a config
file, or a backup — that attacker uses curl, sends no Origin, and is admitted.
An API token is account-wide and lives until its minted lifetime says
otherwise ({kind:'eternal'} is a legitimate choice a caller must spell), so
a leaked eternal
full-scoped token reaches every action whose spec accepts the bearer channel,
plus any role-gated action whose role the compromised account holds. A narrowed
token is bounded by its scope instead (§Token scoping), which is the second
compensating control below.
Compensating controls, in the order they bite:
- Credential-channel gating on the sensitive specs. An action spec names
which credential types admit; the credential-lifecycle mutations
(
account_session_revoke,account_session_revoke_all,account_token_create,account_token_revoke) are['session']on both spines, so a leaked token cannot mint another token, revoke one, or kill a session. See §Credential-channel gating. Consumers should extend the same treatment to their own high-consequence mutations rather than leaving them at "any authenticated credential". - Per-token scoping. A token minted with a narrowed scope calls only the RPC methods it names and reaches no non-RPC spine surface at all. This is the strongest available control and the right one to reach for when issuing a token to automation. See §Token scoping.
- Revocation, which bounds the window but not the reach. (Bearer auth is deliberately not rate limited — see §Why bearer auth is not rate limited — so revocation and the two controls above are the whole set.)
Not yet available, and the reason this is a stated tradeoff rather than a solved problem: per-token IP binding, and token expiry by default.
If you want the replay surface closed and your clients don't need bearer through the proxy, strip the header at nginx:
proxy_set_header Authorization "";This is a per-deployment decision, not a fuz_app default. Note that a consumer
serving git smart-HTTP or a bearer-authenticated file store needs carve-out
location blocks for those routes, as fuz_forge does.
nginx should serve static files directly — only proxy /api and /health to
the app server. This reduces the attack surface (fewer requests hit the app) and
enables nginx-level caching for immutable SvelteKit assets (/_app).
There is no general nginx-config validator. An earlier validate_nginx_config
treated the Authorization strip as a hard requirement, which made it reject
every deployed consumer's config; it was never wired into a deploy and has been
removed rather than left to drift further from what the recipes build.
The one nginx property that is checked is the facts location, because
confidentiality depends on it: XAccelConfig (server/x_accel.ts) can only be
constructed by passing the config through a check that the facts location is
internal;, so X-Accel fact serving cannot be enabled un-validated. See
§Serving Facts.
Recommended locations:
location /api { proxy_pass ...; }
location = /health { proxy_pass ...; }
location /_app { expires 1y; add_header Cache-Control "public, immutable"; try_files $uri =404; }
location / { try_files $uri $uri/index.html $uri.html =404; }Include server_tokens off to suppress nginx version disclosure and
limit_req for global rate limiting at the proxy layer. Define the zone in the
http context (e.g., /etc/nginx/conf.d/rate_limit.conf):
limit_req_zone $binary_remote_addr zone=global:10m rate=10r/s;Apply in the server block:
limit_req zone=global burst=20 nodelay;This complements fuz_app's per-route in-memory rate limiters and provides shared IP-based limiting even in multi-process deployments.
The app server's static serving middleware remains useful for dev mode (no nginx) and local preview. In production, nginx handles all static requests.
add_header inheritance: nginx's add_header in a child location block
replaces (not extends) inherited headers from the parent server block.
Locations that add their own headers (e.g., /_app with Cache-Control) must
repeat the security headers (HSTS, X-Content-Type-Options, etc.).
try_files / trailingSlash coupling: The fallback chain
$uri $uri/index.html $uri.html matches adapter-static's default
trailingSlash: 'never'. If trailingSlash changes in SvelteKit config, the
nginx pattern must be updated to match.
Recommended security headers in the nginx server block:
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-Frame-Options "DENY" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;Because add_header in a child location block replaces (not extends)
inherited headers from the parent server block, any location that adds its
own headers (e.g., /_app with Cache-Control) must repeat these security
headers.
When running behind nginx on a dedicated server, systemd hardening directives limit what the Deno process can do if compromised:
[Service]
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=read-only
PrivateTmp=true
ReadWritePaths=/var/lib/{app}NoNewPrivileges— Process cannot gain new privileges (no setuid, no capability escalation)ProtectSystem=strict— Entire filesystem read-only except explicitly allowed pathsProtectHome=read-only— Home directories read-only (useReadWritePathsfor app data)PrivateTmp— Isolated/tmp— other services cannot read the app's temp filesReadWritePaths— Allowlist for writable directories (DB sockets, daemon token, logs)
These are defense-in-depth: if an attacker achieves code execution through the Deno process, they are sandboxed to the declared paths. Combined with a dedicated non-root service user, this limits blast radius significantly.
When to adopt: After the deployment is stabilized and server changes are
infrequent. During active development, ProtectSystem=strict requires updating
ReadWritePaths whenever the app writes to a new location — friction that
isn't worth it while things are still changing. Running as root during early
development is acceptable when SSH is key-only with fail2ban.
Dedicated service user: Create a non-root user for the app process. Copy SSH keys from root, verify access, then disable root login. The service user should own the app data directory and have no other privileges.
Client IP is resolved from X-Forwarded-For before auth and rate limiting:
- Rightmost-first XFF walk — strip known trusted proxy entries (CIDR-aware)
- Untrusted connection with an XFF header → header is ignored (spoofed XFF)
normalize_ipstrips IPv4-mapped IPv6 (::ffff:) and lowercases for consistent key comparisons- CIDR prefixes validated at parse time (NaN, negative, over-range rejected)
Cross-impl parity: the login_security cross-backend suite pins XFF
resolution end-to-end on both spines — distinct X-Forwarded-For IPs get
independent rate-limit buckets (a fresh forwarded IP is unaffected by another's
exhausted bucket), proving the limiter keys on the resolved client IP rather than
the loopback TCP peer the request actually arrives on.
For single-proxy setups (nginx colocated with app), use $remote_addr:
proxy_set_header X-Forwarded-For $remote_addr;$proxy_add_x_forwarded_for appends to client-injected XFF headers. The
rightmost-first walk handles this safely, but $remote_addr eliminates
injected data entirely. For multi-proxy chains, $proxy_add_x_forwarded_for
is required and each intermediate proxy must be in trusted_proxies.
All auth mutations attempt a fire-and-forget audit write (never blocks or breaks
auth flows). Delivery is best-effort, not durable: a failed insert is caught
and logged, not retried — the mutation still succeeds and no audit row exists for
that event. Listeners (the audit SSE stream, the socket-revocation listeners) are
notified only after a successful insert — a failure row as soon as it is
written, a success row once the request's transaction has committed — so an audit
outage also suppresses the security reactions chained off it. Enforcement that
must survive an audit failure cannot be built on the listener path alone, which
is why every revocation handler closes connections itself (see
Closing on Revocation).
The audit log's identity columns (actor_id, account_id, target_account_id,
target_actor_id) carry no foreign key — they are plain UUID. An audit log
is an append-only historical record, not a live relational entity: a soft-delete
keeps the account row (so the ids still resolve), and a hard purge leaves the raw
id intact on historical rows for forensic correlation rather than nulling it.
Deleting or purging an account additionally snapshots its identifying values
(username / email, and per-actor name) into the deletion event's metadata,
so the identity behind a now-orphaned id survives even after the row is gone.
Append-only is also enforced at the one HTTP surface that could have broken it:
http/db_routes.ts's NON_DELETABLE_TABLES refuses a row DELETE on audit_log
with 400 table_not_deletable, ahead of the key-shape check. audit_log has a
single-column primary key, so the keeper-gated browser would otherwise delete a
trail row and report success. It is the sharpest of the four excluded tables
because it is both the trail and part of how revocation propagates — the SSE and
WS auth guards close live connections by listening to audit events, so a raw row
delete would be invisible to them. The read side is gated the same way: the
browser exposes only the consumer's declared browsable_tables, minus the
NON_BROWSABLE_TABLES credential floor (account, auth_session,
api_token, bootstrap_lock) — account matters most, since the browser
would otherwise be the one surface returning the password-hash corpus in bulk.
Unlisted tables answer exactly like nonexistent ones. Direct DB access remains
out of scope.
Each event records an outcome (success or failure), so login/bootstrap/password
change failures are tracked without needing separate event types.
Instrumented event types:
login, logout, bootstrap, signup, password_change, session_revoke,
session_revoke_all, token_create, token_revoke, token_revoke_all,
role_grant_create, role_grant_revoke, account_delete, account_purge,
account_undelete, actor_delete, actor_purge, actor_undelete,
invite_create, invite_delete, app_settings_update, db_admin_row_delete
(plus the role_grant_offer_* consent-flow events)
Admin read surface: audit_log_list RPC action (filterable by event type,
outcome, account, or gap-fill cursor), audit_log_role_grant_history RPC
action, admin_session_list RPC action (all active sessions with
usernames), and the optional GET /audit/stream SSE endpoint for
realtime feeds.
- CSRF — covered by
SameSite=Strict, JSON-onlyPOSTs, and Origin verification (see CSRF Protection). No additional tokens needed. - Session fixation — sessions are server-generated via
crypto.getRandomValues, never accepted from client input. - Session binding (IP/user-agent) — not implemented. IP binding breaks mobile
users whose IP changes on network switch. User-agent binding is easily spoofed
and creates false-positive lockouts on browser updates. The real defenses are
HttpOnly+Secure+SameSite(prevents exfiltration) and session limits (bounds blast radius of a stolen cookie). - Password complexity rules — NIST 800-63B guidance: complexity requirements
push users toward predictable patterns. Length-only validation is used
(
PASSWORD_LENGTH_MIN–PASSWORD_LENGTH_MAX). - Timing attacks on token validation — bearer token and session validation use
blake3 hash-then-compare (
===on hex strings). Recovering a 64-char hex hash character-by-character, then reversing blake3, against 32 bytes of token entropy is not a practical attack.timingSafeEqual(fromnode:crypto) is used where it matters: daemon token validation and bootstrap token comparison. Cookie signature verification is constant-time too, but via a different primitive — WebCryptosubtle.verifyHMAC-SHA256 (keyring.ts), whose comparison the spec requires be constant-time. Password-path timing defense is separate:verify_dummy()equalizes Argon2id timing across found/not-found login paths, and an additional 250ms ±25ms floor on 401 responses equalizes every other path difference (DB lookup, rate-limit branches). See Account Enumeration Prevention above.
The in-memory rate limiter, the live-connection registries, and daemon token state are designed for single-process deployments:
- Rate limit counters are not shared across processes. In a horizontally-scaled deployment, an attacker distributing requests across N instances can attempt N × max_attempts per window.
- Live connections are registered in the process that accepted them. A
revocation closes the WebSockets and SSE streams of its own process only
(
deps.connection_closerand the audit listeners are both in-memory), and the per-account connection cap and per-session stream cap count per process. The expired-session sweep closes the same way: a second process keeps the connections it holds on a session this one swept, and its own pass finds the row already gone. - Daemon token rotation is file-based. Multiple processes sharing the file may read stale state between token write and fsync.
For multi-process deployments, rate limiters would need Redis or a shared DB table;
revocation closes would need a cross-process channel (a shared pub/sub, or
LISTEN/NOTIFY); daemon tokens would need a distributed lock or a different
rotation strategy.
In-memory rate limiter state resets on server restart. An attacker can resume
brute-forcing immediately after a restart without waiting for the previous
window to expire. Use nginx-level limit_req as a complementary defense —
nginx rate limit state persists across app restarts and provides IP-based
protection independent of the application.
The last-admin guard (ERROR_CANNOT_DELETE_LAST_ADMIN) reads the active-admin
count and then deletes in separate statements (no SELECT … FOR UPDATE /
serializable wrap), so two concurrent account_delete / account_purge calls
targeting the last two distinct active admins can each observe count = 2 and
both commit, leaving zero active admins. This is accepted: the guard stops the
one-call foot-gun, not concurrent races; the outcome is keeper-recoverable
(re-grant admin), and the keeper account holds admin at bootstrap and is itself
protected by the stricter keeper guard. The keeper guard has no equivalent race
(single, non-web-revocable target). Revisit if a threat model makes concurrent
admin-removal a real concern.
Unique constraint violations in signup use PostgreSQL error code 23505
(unique_violation) rather than string matching on error messages. This is
robust across PostgreSQL versions and locales. The same pattern is used in
db_routes.ts for foreign key violations (23503).