docs(decisions): ADR-0020 - timed Parquet snapshots of the live database

[Fl0p]

What this is

ADR-0020 only, Status: Proposed. The ticket gates implementation behind CTO
review of the decision, because the snapshot storage format is a one-way door.
Nothing in internal/ or docker-compose.yml is touched yet; the mechanism, the
env rows in README.md / docs/index.md and the docs/operations/ restore
procedure land on this same branch once the decision is signed off.

The decision

cotel takes its own snapshots with EXPORT DATABASE ... (FORMAT PARQUET, COMPRESSION ZSTD) on a timer, into a second named volume. This is option 1 from
the ticket, and the recommendation held up under measurement rather than being
taken on faith.

Measured on robmini, against a probe copy of the live 152.6 MB database

65,184 spans / 3,048 daily_usage rows / 17 users / schema_version 10, all
through the production image's own binary (--db-query, which opens the file
access_mode=read_only).

Operation	wall, incl. ~0.4 s container start + open	output
baseline SELECT 1	0.42 / 0.36 / 0.36 s	-
EXPORT DATABASE Parquet + ZSTD	0.69 / 0.58 / 0.51 s	4,399,961 B
EXPORT DATABASE Parquet (snappy)	0.55 s	8,629,574 B
EXPORT DATABASE CSV	0.77 s	71,012,385 B
IMPORT DATABASE into an empty volume	0.96 / 0.74 s	a 36.2 MB file

So the statement costs 0.15-0.3 s of connection time and 4.2 MiB. There is no
"it blocks ingest for a noticeable time" case that would have flipped the choice to
option 3.

Restore was executed, not assumed: spans, daily_usage and users counts,
max(start_time), schema_version and all four indexes come back identical, and
the cotel binary opens the file the CLI wrote.

Two findings that shaped the design

• cotel serves everything through one connection (rw.SetMaxOpenConns(1), and
  ReadOnly() shares that pool), so the duration of the statement, not the size of
  the output, is the decisive number. It also means the snapshot cannot race the WAL
  checkpoint: they are serialised by construction.
• /api/v1/export (option 3) is not a backup of the database at all. Its ZIP
  carries spans.csv and daily_usage.csv only; users, api_tokens, settings
  and schema_version are absent, so an instance restored from it rejects every
  agent's ingest token. EXPORT DATABASE writes all six tables.

Scope question for the reviewer

The ADR proposes shipping restore as cotel --db-import <dir> so a restore needs
only the image already on the host, instead of the hand-version-matched DuckDB CLI
that is step 4 of docs/operations/duckdb-recovery.md. That is an addition to the
ticket's scope. Say the word if you would rather have it as a separate ticket, and
the restore procedure will be written against the CLI instead.

Verification

Docs-only change. docs/decisions/index.md updated. No em dashes, no tracker IDs in
the diff.

---

Renumbered 2026-10-05: opened as 0019-..., but 0019-ci-never-mutates-an-issue.md (PR #127) landed on main first, so this ADR is now 0020 and the branch is rebased onto current main. No cross-reference to it existed outside docs/decisions/index.md.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Fl0p]

Heads up on the number, not the content: 0019 is taken on main as of #126 and #127 - the alert-recovery record moved there from a duplicate 0017, and this branch was cut before that. Renumber this one to 0020 (file, heading and the index row) before merge, or it lands as the third ADR-0019 this morning.

[Fl0p]

ADR number collision - renumber before merge. origin/main already carries docs/decisions/0019-ci-never-mutates-an-issue.md (landed in #126 / #127 while this branch was open). This PR adds docs/decisions/0019-production-database-snapshots.md.

Git will not warn you: the two filenames differ, so the only conflict is in index.md - resolve that by hand and main ends up with two ADR-0019s. Rebase on current origin/main, rename this one to 0020-production-database-snapshots.md, update index.md and any internal links.

Check with ls docs/decisions/ | cut -c1-4 | uniq -d - must print nothing.

-- Daedalus (CTO)