Repository navigation
fix: share reproducible Ferrocene repositories - #52
Merged
Merged
Conversation
4og
requested review from
AlexanderLanin,
MaximilianSoerenPollak,
dcalavrezo-qorix and
nradakovic
as code owners
November 13, 2026 14:40
4og
force-pushed
the
fix/ferrocene-reproducible-repositories
branch
from
November 16, 2026 11:20
699bb79 to
d3d086a
Compare
Declare checked Ferrocene repositories reproducible after successful generation, with compatibility guards for legacy Bazel and unchecksummed custom archives. Add focused repository-context regression checks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Include the Ferrocene repository-context regression target in the existing example smoke-test job. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
4og
force-pushed
the
fix/ferrocene-reproducible-repositories
branch
from
November 16, 2026 13:31
d3d086a to
e17d833
Compare
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block the change enabling reproducible Ferrocene repository caching.
No open issues 🎉
What changed in this PR
This PR enables Bazel to share reproducible Ferrocene repositories through repo_contents_cache, reducing duplicate compiler extractions across output bases.
Changes:
- Declare repositories reproducible after successful extraction and file generation, only when the primary checksum and metadata API are available.
- Add regression checks for reproducibility, compatibility, and failure handling, and run them in CI.
- Refresh the two affected extension digests in the lockfile.
| File | Description |
|---|---|
MODULE.bazel.lock |
Refreshes Ferrocene extension digests. |
extensions/ferrocene_toolchain_ext.bzl |
Adds the guarded reproducibility declaration. |
extensions/ferrocene_toolchain_ext_test.py |
Tests metadata timing, checksum guards, compatibility, and failures. |
extensions/BUILD.bazel |
Registers the extension regression test target. |
.github/workflows/examples.yml |
Runs extension tests alongside example tests. |
Review effort: Balanced
dcalavrezo-qorix
approved these changes
Nov 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Ferrocene repository rule downloads and extracts the compiler archive, optional coverage tools and optional Miri sysroot, but does not declare the resulting repository reproducible. Bazel therefore creates separate extracted copies in each output base instead of sharing them through
repo_contents_cache, even when the repository inputs and checksummed archives are identical. The download cache alone does not prevent this duplication.In two existing Baselibs worktrees, each QNX x86_64 copy occupied about 916 MB and each Linux x86_64 copy about 925 MB. Both pairs had 222 matching payload paths but no shared cross-worktree file identities. This is the same class of issue addressed by eclipse-score/bazel_cpp_toolchains#183.
Fix
Return
ctx.repo_metadata(reproducible = True)only after all archive extractions and generated file writes succeed.The declaration is guarded by both API availability and a nonempty primary
sha256. Older Bazel versions and unchecksummed custom primary archives retain their existing behavior without a reproducibility declaration. Existing checksum requirements for optional coverage and Miri archives remain unchanged.Add focused repository-context regression checks and refresh the two affected extension digests in the lockfile. Archive packaging and cross-architecture host-tool duplication are unchanged.
Verification
Verified commit:
4597cfa66f31f573b106f93c8fb258c217d0f746.Local checks passed with Bazel 8.6.0:
buildifier -mode=check -lint=warn extensions/BUILD.bazel extensions/ferrocene_toolchain_ext.bzl bazel mod deps --lockfile_mode=error bazel test --lockfile_mode=error --config=smoke //examples/basic/... //extensions:ferrocene_toolchain_ext_test bazel coverage --lockfile_mode=error --config=smoke //examples/basic:basic_testThe five regression checks cover declaration after generation, optional checked archives, an empty primary checksum, a context without the metadata API, missing optional checksums, and extraction/generation failures. Removing the declaration in memory caused the expected regression failures. Example tests passed, and coverage data contained executed lines from
examples/basic/src/lib.rs. Compatibility with an older API was mock-tested, not tested with an older Bazel executable.For direct cache-sharing verification, both existing Baselibs worktrees used Bazel 8.8.2 with shared repository, repository-contents and disk caches. Each temporarily pinned this exact local commit through a local Git override. After lock refresh, A fetched only the QNX and Linux x86_64 Ferrocene repositories first, then B fetched the same variants with
--lockfile_mode=error. Profiles were captured for both phases.The resolved directories and representative
cargoandrustcdevice/inode identities matched between A and B. Both existing comparison helpers returnedSHAREDand exit 0. B left all warmed payload identities and allocations unchanged, added no cache keys or Ferrocene trees, and added zero extracted Ferrocene payload bytes. Combined unique-tree allocation fell by 1,841,152,000 bytes, excluding small output-entry and marker overhead.Profiles distinguish extraction from cache lookup: A executed two Ferrocene repository implementations and six archive extractions. B executed zero Ferrocene implementations, download/extract calls or extraction events; its repository-fetch spans were cache lookup/materialization only.
A's small QNX integration build also passed:
Build success and action-cache hits were not used as proof of repository sharing. No caches were cleared, no fetch was forced and no SDK/output directory was manually deleted. After recording the evidence, both Baselibs module/lockfiles were restored byte-for-byte to their pre-test state, preserving the existing C++ override, branches, unrelated edits and cache configuration. The restored 0.10.0 pin does not retain the fix for future rematerialization until this change is adopted.