Skip to content

A language-neutral test-vector pack that both readers are held to - #276

Merged
404SecNotFound merged 1 commit into
feature/property-testsfrom
feature/vector-pack
Oct 24, 2026
Merged

404SecNotFound merged 1 commit into
feature/property-testsfrom
feature/vector-pack

Conversation

@404SecNotFound

Copy link
Copy Markdown
Owner

What this is

Bet 4 of the review report's plan. docs/vectors/ is a language-neutral test-vector pack for KEYM v2, v3 and v4, as plain JSON that a third implementer can read with no Node, no Python and no code from this repository. Both readers here are held to it on every pull request.

Stacked on #275 (Bet 5), because both change package.json, README.md and the conformance workflow. GitHub retargets it as the stack merges.

The pack

94 vectors in seven families. docs/vectors/README.md describes every field.

File Vectors What it holds
primitives.json 10 HKDF-SHA-256 (RFC 5869 A.1 to A.3, prk and okm), ChaCha20-Poly1305 (RFC 8439 §2.8.2 and A.5), AES-256-GCM (NIST CAVP) and Argon2id, including RFC 9106 §5.3
containers.json 8 Containers from pinned inputs, with kdf_input, slot_key, core_header, k_table and slot_table_mac. v3 §8's and v4 §8's vectors, all three ciphers, a key file, a decomposed password, and the base the reject rows change
shares.json 1 A 3-of-5 split from pinned coefficients, with both share texts, both set ids and the set code. The coefficients' byte order is written down, since no spec serialises them
parts.json 1 KMPART1 and KMPART2 strings
text.json 3 Armor in both widths, the IGNORABLE set, and padded_len across v4 §4.2's branches
reject.json 50 Inputs a reader must refuse with v2 §6's one generic failure, each tagged with its rule, in both share versions
accept.json 21 Inputs a reader must open although it may be tempted to refuse them

Authentic reject rows. The container rows that test a rule are sealed correctly around one violation. Every AAD, wrap, tag and the slot-table MAC is computed over the bytes as written, so a reader that skips the rule opens the row. There are 17 of them:

  • the reserved header and slot fields;
  • nine well-formed slots;
  • a passphrase slot under HKDF;
  • KDF parameters past their read bounds;
  • a lone chunk sealed with final_flag 0;
  • four v4 streams that break §4.4's length and padding steps.

The sealer must reproduce the base container byte for byte before it writes any of them.

Accept rows. These catch the reader that refuses too much:

  • each slot-level violation as slot 0 beside a good slot 1 (v2 §4.4 disqualifies the slot, never the container);
  • an Argon2id slot with memory_kib below 8 × parallelism, which §6 admits and no library can derive;
  • a full table of eight slots;
  • a stripped slot table, which must open and be reported (v3 §5.2);
  • armor full of ignorable characters;
  • a respelt share;
  • share sets out of order, repeated, and with a stray share above the k lowest listed first;
  • paper parts out of order.

How it is held

  • reference/vectors.py generates the pack from keym2.py. It requires each file to equal what it renders, byte for byte. It also checks:
    • the inputs and outputs printed in v3 §8 and v4 §8;
    • each primitive against keym2.py's own libraries;
    • that keym2.py opens every container and accept row, and refuses every reject row with its exact generic message;
    • that its decoders read every share, part and armor string back.
  • scripts/vectors-test.mts reproduces every vector with the app's code:
    • each intermediate derivation on its own, then the whole container;
    • every reader on every good and bad input;
    • the slot-table report;
    • the exact refusal message.
  • Append-only. The pack only grows, like the fixtures. --write validates every family before it writes anything and refuses to change a committed vector. A wrong vector is superseded by an erratum naming its replacement. Both suites skip it, and the README tells a third reader to do the same.
  • Wiring. npm run test:vectors runs both suites, and so do a step in the conformance workflow and the README's test list. VECTOR_COUNT is asserted in both.
  • Supporting changes.
    • v2 §7 now states how armor is wrapped. The vector pins that layout, and before this only the code said so.
    • v3 §8 and v4 §8 each name their vector in the pack.
    • keym-v2.ts exports its three intermediate derivations for the suite. The app does not call them through that export.

Sources

rfc-editor.org is blocked from the build environment. The published values were therefore taken from pyca's cryptography_vectors 50.0.2, which carries these RFCs' and NIST's values as data files. Each vector names the exact file and entry, and both suites check every one against the libraries the readers load.

RFC 9106 §5.3's Argon2id vector uses a secret and associated data. keym2.py's library checks it through its low-level interface, and the TypeScript skips it, because hash-wasm takes no associated data.

Review

Two independent reviews ran before anything was pushed, one on the vectors' correctness against the spec and one on the suites' strength. Because the pack is append-only, a vector that encodes a misreading would be a permanent erratum. The reviews found 22 distinct problems once their overlap is merged, all fixed here. The ones that changed the design were these.

  • The skip rule was untested. Every container had one slot, so a reader that refused the whole container on a bad slot passed. That is the data-loss bug v2 §4.4 exists to prevent. accept.json holds the same bad slots beside good ones.
  • Some rows did not test their rule. Loosening the slot_count or header checks in either reader still passed, because the mutated byte also broke authentication. Those rows are now sealed around the violation.
  • The erratum process could not work. Both suites kept checking a superseded vector.
  • The decoders were never run on good input in Python, so a reader that refused everything passed every non-container family.
  • The slot-table MAC result was never checked. A reader whose MAC always verified passed.
  • Gaps in shares and v4. KMSHARE2 had no reject rows, nothing tied a decoded share to its 16-byte set id, sets were only combined in index order, and v4 had no reject rows.
  • A refusal was not checked to be §6's generic failure.
  • Some text was wrong. The coefficient layout was in no spec, the armor wrapping was unstated, and some notes, labels and README sentences were inaccurate. The JSON now escapes every non-ASCII character, so a tool that normalises Unicode cannot turn the decomposed-password vector into the precomposed one it tests.

Negative controls

Each was applied to a copy, confirmed to compile, and shown to fail the vector that names it.

  • Every rule an authentic row tests, removed from each reader in turn. 25 of 26 fail their own row and no other. The exception is the reserved header byte in keym2.py. Its AAD is rebuilt from the parsed header, which packs that byte as zero, so the slot's tag refuses the row as well as the explicit check.

  • Reader defects.

    • MAC always verifies;
    • Python's armor reader refuses everything;
    • shares combined by list position instead of index;
    • no lowest-k sort;
    • no set-id agreement;
    • the KMSHARE2 checksum skipped;
    • v4 steps 4 and 5 removed;
    • the ignorable set reduced to LF;
    • no NFC;
    • a refusal message that names the check.

    Each fails both suites, or the suite for the reader changed.

  • Pack and documents. These all fail by vector id: a flipped byte in a container, an HKDF answer, an Argon2id answer and a set id; a dropped vector; a misprinted master key and container byte in v3 §8, and a misprinted MAC in v4 §8.

  • The machinery.

    • A generator crash is reported by name, not as a traceback.
    • A refused --write leaves every file untouched.
    • An erratum vector with changed bytes passes both suites.
    • An erratum naming no vector fails.

Not in this PR

  • A container of more than one chunk. Its hex would run to megabytes, so chunk boundaries stay with the conformance suite.
  • Shamir, passkey and passphrase-and-shares slots as whole containers, and the self-extracting page.
  • Report item E4 (typechecking scripts/*.mts). vectors-test.mts is not typechecked by npm run typecheck, like the other suites there.

Verification on this tree

  • npm run typecheck, npm run build, and every test:* script in package.json except test:browser (45 scripts, test:vectors among them) pass on the head commit.
  • Playwright Chromium with --workers=2 on the full suite, after a plain npm run build, gives 360 passed, 5 skipped and 0 failed. An earlier run, made while the review agents were running suites on the same machine, failed async-guard.spec.ts "switching tabs mid-derivation" and calibration.spec.ts "calibrating twice agrees with itself". The first is one of CLAUDE.md's two named timing flakes, and the second is the calibration test CLAUDE.md says fails under CPU contention. Both passed on the quiet re-run above. Firefox and WebKit are left to CI.

Generated by Claude Code

docs/vectors/ holds the expected bytes for the text and binary forms of
KEYM v2, v3 and v4 as plain JSON, for anyone writing a third reader. There
are 94 vectors in seven families.

- primitives.json has HKDF-SHA-256 (RFC 5869 A.1 to A.3, prk and okm),
  ChaCha20-Poly1305 (RFC 8439 §2.8.2 and A.5), AES-256-GCM (NIST CAVP) and
  Argon2id, transcribed from pyca's cryptography_vectors 50.0.2 and checked
  against the libraries each reader loads.
- containers.json has whole containers from pinned inputs with kdf_input,
  slot_key, core_header, k_table and slot_table_mac, including v3 §8's and
  v4 §8's vectors, all three ciphers, a key file and an NFD password.
- shares.json, parts.json and text.json have a 3-of-5 split from pinned
  coefficients with both share texts and the set code, the KMPART1 and
  KMPART2 strings, armor in both widths, the IGNORABLE set and padded_len.
- reject.json has 50 inputs a reader must refuse with v2 §6's one generic
  failure, each tagged with its rule. The container rows that test a rule
  are sealed correctly around one violation, so a reader that skips the
  rule opens them.
- accept.json has 21 inputs a reader must open although it may be tempted
  to refuse: a bad slot beside a good one, a stripped slot table (which
  must also be reported), armor full of ignorable characters, shares out of
  order or repeated, and parts out of order.

reference/vectors.py generates the pack from keym2.py and checks it,
including the inputs and outputs printed in v3 §8 and v4 §8 and the files'
exact text. scripts/vectors-test.mts reproduces every vector with the app's
code. The pack is append-only, a wrong vector is superseded by an erratum
that both suites skip, and its count is asserted in both. v2 §7 now states
how armor is wrapped. keym-v2.ts exports the three intermediate derivations
for the suite. npm run test:vectors, a conformance workflow step and the
README list run it.
@404SecNotFound
404SecNotFound merged commit dca846d into feature/property-tests Oct 24, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant