From 9cca049bd06902cdaf89f46ec2997e4f9e7a776c Mon Sep 17 00:00:00 2001 From: Dev <9416650+aoinoikaz@users.noreply.github.com> Date: Sun, 1 Nov 2026 08:48:19 -0500 Subject: [PATCH] Release 0.12.0.0: reclaim-nars, and a narinfo refused without its NAR Dates the changelog section, removes its fragments and bumps the version. Major bump: the exports only grow, but cacheApp now refuses a narinfo it used to accept (no stored NAR, no FileHash or FileSize, or a URL not under nar/). --- CHANGELOG.md | 5 +++++ changelog.d/narinfo-requires-its-nar.md | 1 - changelog.d/reclaim-unreferenced-nars.md | 1 - nova-cache.cabal | 2 +- 4 files changed, 6 insertions(+), 3 deletions(-) delete mode 100644 changelog.d/narinfo-requires-its-nar.md delete mode 100644 changelog.d/reclaim-unreferenced-nars.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 6cfe76a..7bff782 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,10 @@ # Changelog +## 0.12.0.0 - 2026-11-01 + +- **`nova-cache-server reclaim-nars` lists, and with `--delete` removes, the NARs no narinfo references.** A client uploads a path's NAR before its narinfo, as upstream's `BinaryCacheStore::addToStoreCommon` does, so a NAR whose narinfo the server refuses, or whose push dies before the narinfo, stayed under `nar/` for good: the server has no route that deletes, and gains none. The subcommand runs on the host against the store directory. A narinfo names its NAR by its `URL` alone, as upstream reads it (`getFile(info->url)`), with the compression suffix part of the name. Three rules guard a run. A NAR is eligible only once it is two weeks old by its modification time (`--min-age-hours` overrides), because a NAR is unreferenced until its push sends the narinfo and nothing bounds how long a push takes; git's `gc.pruneExpire` gives unreferenced objects the same two weeks against the same race. A narinfo that cannot be read, parsed, or resolved to `nar/` blocks the whole run instead of being skipped, since the NAR it names would otherwise look unreferenced. And a store holding NARs and no narinfo at all blocks it too, since one that lost its `narinfo/` directory, which the server recreates empty when it starts, looks the same as one whose every NAR is unreferenced. A run is not atomic with the server, so it belongs on a cache with no push in progress: a narinfo that arrives during a run for a NAR past the minimum age, as upstream's `nix copy` can send after skipping a NAR already present, loses that NAR. Upstream ships no garbage collection for binary caches, so there is no behavior to match beyond the URL. The library gains `NovaCache.Reclaim` (the pure selection and the store scan) and, in `NovaCache.Store`, `listNarFiles`, `listNarInfoFiles`, `removeNar` and `fileStoreAt`, a constructor that creates no directories; `time` joins the dependencies. +- **A narinfo is refused unless the NAR its `URL` names is stored and matches its `FileSize` and `FileHash`.** `PUT /.narinfo` validated the narinfo's own fields and signed it without looking at the NAR, so a NAR that arrived short or wrong sat behind a signed narinfo: clients rejected the NAR on its hash, and nova-nix's push never repaired it, because a push skips every path whose narinfo exists. Warp before 3.4.17 made the short case reachable: it ended a chunked request body at the connection's end of input as if the terminating chunk had arrived (yesodweb/wai#1129), so a `PUT /nar/` cut off mid-chunk got 200 and stored a prefix. The narinfo is now refused with a 400 naming the problem unless its `URL` resolves, exactly as a GET for it is routed, to a file under `nar/` of `FileSize` bytes that hashes to `FileHash`, so an upload that went wrong fails at the narinfo. nova-nix's next push sends both again. Upstream's `nix copy` skips a NAR whose file already exists, so a wrong file under that name goes on refusing the narinfo until `reclaim-nars` removes it. A narinfo without `FileSize` or `FileHash` is refused as well, since the signature does not cover them and nothing else records which file the narinfo describes, and so is a `URL` that is not under `nar/`, such as an absolute one, which upstream can read but its `BinaryCacheStore` never writes. Upstream's `BinaryCacheStore::addToStoreCommon` and nova-nix's push both send the two fields and a relative `nar/` URL, and both upload the NAR before its narinfo, so a complete upload from either is accepted. `cacheApp` refuses requests it used to accept, which is why this is a major version. The server reads the stored NAR through once more when the narinfo arrives, in constant memory, and answers 500 if that read fails. The library gains the check's pure pieces in `NovaCache.Server` (`narUrlFileName`, `narClaim`, `judgeStoredNar`, `renderNarRefusal`). + ## 0.11.1.1 - 2026-08-26 - **A symlink target with a separator serialises to its POSIX spelling on every platform.** Windows stores a symlink's reparse point with backslashes, so a target written `bin/tool` reads back `bin\tool`, and `getSymbolicLinkTarget` returned that verbatim into the NAR. NAR targets are host-independent byte strings in the POSIX spelling, so the archive of the same logical tree diverged between Windows and POSIX, and an unpack-then-recheck round-trip failed its on-disk hash comparison on Windows for any symlink whose target had more than one component. Both producers now normalise the separator to `/` when reading a target on Windows, where a backslash is only ever a separator and encodes as the single byte `0x5C` that no multi-byte UTF-8 sequence contains, so the byte-level replacement is exact. POSIX is untouched, where a backslash is a legitimate filename byte. diff --git a/changelog.d/narinfo-requires-its-nar.md b/changelog.d/narinfo-requires-its-nar.md deleted file mode 100644 index a337b02..0000000 --- a/changelog.d/narinfo-requires-its-nar.md +++ /dev/null @@ -1 +0,0 @@ -- **A narinfo is refused unless the NAR its `URL` names is stored and matches its `FileSize` and `FileHash`.** `PUT /.narinfo` validated the narinfo's own fields and signed it without looking at the NAR, so a NAR that arrived short or wrong sat behind a signed narinfo: clients rejected the NAR on its hash, and nova-nix's push never repaired it, because a push skips every path whose narinfo exists. Warp makes the short case reachable: it ends a chunked request body at the connection's end of input as if the terminating chunk had arrived (yesodweb/wai#1129), so a `PUT /nar/` cut off mid-chunk got 200 and stored a prefix. The narinfo is now refused with a 400 naming the problem unless its `URL` resolves, exactly as a GET for it is routed, to a file under `nar/` of `FileSize` bytes that hashes to `FileHash`, so an upload that went wrong fails at the narinfo and the next push sends both again. A narinfo without `FileSize` or `FileHash` is refused as well, since the signature does not cover them and nothing else records which file the narinfo describes. Upstream's `BinaryCacheStore::addToStoreCommon` and nova-nix's push both always send the two fields and both upload the NAR before its narinfo, so a correct client is never refused. The server reads the stored NAR through once more when the narinfo arrives, in constant memory, and answers 500 if that read fails. The library gains the check's pure pieces in `NovaCache.Server` (`narUrlFileName`, `narClaim`, `judgeStoredNar`, `renderNarRefusal`). diff --git a/changelog.d/reclaim-unreferenced-nars.md b/changelog.d/reclaim-unreferenced-nars.md deleted file mode 100644 index b3291c8..0000000 --- a/changelog.d/reclaim-unreferenced-nars.md +++ /dev/null @@ -1 +0,0 @@ -- **`nova-cache-server reclaim-nars` lists, and with `--delete` removes, the NARs no narinfo references.** A client uploads a path's NAR before its narinfo, as upstream's `BinaryCacheStore::addToStoreCommon` does, so a NAR whose narinfo the server refuses, or whose push dies before the narinfo, stayed under `nar/` for good: the server has no route that deletes, and gains none. The subcommand runs on the host against the store directory. A narinfo names its NAR by its `URL` alone, as upstream reads it (`getFile(info->url)`), with the compression suffix part of the name. Three rules guard a run. A NAR is eligible only once it is two weeks old by its modification time (`--min-age-hours` overrides), because a NAR is unreferenced until its push sends the narinfo and nothing bounds how long a push takes; git's `gc.pruneExpire` gives unreferenced objects the same two weeks against the same race. A narinfo that cannot be read, parsed, or resolved to `nar/` blocks the whole run instead of being skipped, since the NAR it names would otherwise look unreferenced. And a store holding NARs and no narinfo at all blocks it too, since one that lost its `narinfo/` directory, which the server recreates empty when it starts, looks the same as one whose every NAR is unreferenced. A run is not atomic with the server, so it belongs on a cache with no push in progress: a narinfo that arrives during a run for a NAR past the minimum age, as upstream's `nix copy` can send after skipping a NAR already present, loses that NAR. Upstream ships no garbage collection for binary caches, so there is no behavior to match beyond the URL. The library gains `NovaCache.Reclaim` (the pure selection and the store scan) and, in `NovaCache.Store`, `listNarFiles`, `listNarInfoFiles`, `removeNar` and `fileStoreAt`, a constructor that creates no directories; `time` joins the dependencies. diff --git a/nova-cache.cabal b/nova-cache.cabal index d4c7de2..370fd16 100644 --- a/nova-cache.cabal +++ b/nova-cache.cabal @@ -4,7 +4,7 @@ cabal-version: 3.4 -- 3.0 an in-package component name shadowed its external namesake -- (the hazard the zstandard library's name dodges, documented there). name: nova-cache -version: 0.11.1.1 +version: 0.12.0.0 synopsis: Nix binary cache protocol: NAR, narinfo, signing and a WAI server description: The Nix binary cache protocol in Haskell: nix-base32, NAR serialization