Skip to content

Land #143 (Zakura native block source) with cargo-vet coverage - #148

Merged
zmanian merged 8 commits into
mainfrom
claude/project-thread-uqvw9x
Nov 17, 2026
Merged

zmanian merged 8 commits into
mainfrom
claude/project-thread-uqvw9x

Conversation

@zmanian

@zmanian zmanian commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Before: #143 (erenyegit's optional Zakura native block source for sovright-p2p-ingress) failed cargo vet on 60 new crate versions and conflicted with main after #144.

After: this branch is main plus #143's head (8cc066d) merged as is, with the conflict resolved and every one of the 60 versions covered, so cargo vet --locked passes. Merging this with a merge commit also marks #143 merged.

How: the Zakura crates are covered by publisher trust rather than exemptions. 34 of them are published by Zakura's own release workflows through crates.io trusted publishing (github:zakura-core/common, github:zakura-core/zakura) and 3 by Sean Bowe (ebfull), whom librustzcash already trusts. Zakura ships a new major every few days, so trust holds across bumps where per-version exemptions would churn, and zakura-equihash (critical, see supply-chain/README.md) stays off the exemption list.

Changes

  • Merge of feat(p2p-ingress): optional Zakura native block source #143 onto main. One conflict in crates/sovright-p2p-ingress/src/main.rs: fix(ingress): skip block getdata for blocks already received on another connection #144 added received_blocks to spawn_peer; kept feat(p2p-ingress): optional Zakura native block source #143's restructured loop and passed received_blocks.clone().
  • supply-chain/:
    • cargo vet regenerate imports: covers itertools, lru-slab, rand_pcg, web-time.
    • 37 Zakura trusted entries (above).
    • 9 trusted entries for publishers the imported sets already trust: djc (chrono, quinn-proto), dtolnay (ref-cast, ref-cast-impl), epage (toml_datetime, toml_edit, toml_parser, winnow), nuttycom (incrementalmerkletree). This also retires 4 older exemptions.
    • safe-to-deploy exemptions for the 10 small crates left: maybe-rayon, metrics, portable-atomic, proc-macro-crate, rapidhash, sha2-asm, uint, and crossbeam-deque, crossbeam-epoch, crunchy raised from safe-to-run.
    • Exemptions: 337 to 340.
  • Note: zakura-chain turns on sha2's asm feature, so with zakura-native on, the relay's sha2 uses the asm backend too (feature unification). Default builds are unaffected.

Merge order with #135: after this lands, #135 has three textual conflicts (ci.yml job list, ingress dev-dependencies, nu7_transport_tests.rs now using #143's LoopbackRelay helper, which needs #135's new RelayBridge fields). #135's default RawWire::Legacy keeps #143's forwarded_at dedup working; on the opt-in Dual/Bound research wires the native source would refetch announced blocks.

Still open from #143's own list: no run against stock zakurad.

Checklist

  • cargo fmt --all passes
  • cargo clippy --workspace --all-targets -- -D warnings passes (default, and with zakura-native)
  • cargo test for sovright-p2p-ingress (feature on and off) and sovright-zakura-native
  • New functionality includes tests (from feat(p2p-ingress): optional Zakura native block source #143)
  • Commit messages are descriptive

Supersedes #143.


Generated by Claude Code

erenyegit and others added 8 commits November 12, 2026 03:58
Adds an off-by-default `zakura-native` feature to sovright-p2p-ingress.
When built with the feature and enabled at run time, the ingress takes
block advertisements from Zakura native peers, fetches each body by
hash, and hands it to the existing RelayBridge::forward_block, so native
and legacy blocks share one set of checks and one dedup state.

- New crate sovright-zakura-native (zakura-network =15.0.0,
  zakura-chain =14.0.0) with a bounded advert queue, bounded concurrent
  fetches, in-flight coalescing and a per-fetch timeout.
- Legacy peers are unchanged; a native failure does not stop them.
- No change to the relay wire protocol or the sidecar.
- Tests use fake Zakura peers on loopback only; stock zakurad has not
  been tried.
- New CI job builds the feature on Rust 1.98.1 and runs its tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Native block events put the consensus hash in `hash`. Legacy block events,
and the crate README, use `hash` for the relay object hash (the join key
between p2p_block_received and p2p_relay_block_forwarded) and
`consensus_hash` for the chain's hash. Compute `hash` with the helpers the
legacy path already uses (hash::raw_hash_from_header and
inventory_hash_to_display). The failed-forward warning keeps logging the
chain hash, now under `consensus_hash`.

A targeted test pins both fields of the native events to the expressions
the legacy event tests assert, and to the consensus hash the NU7 fixture
records. Native refetch behaviour is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in the passive inclusion observation stream (#142). Two conflicts:

- main.rs: the legacy peer loop sits inside the outcome block on this
  branch; spawn_peer now also takes the observer, as on main.
- README.md: both branches added a section at the same place; both stay,
  main's observation section first.

config.rs, relay_bridge.rs and crawler.rs merged without conflict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #143, findings 2, 3, 4 and 5.

- One block in progress per announcing peer. The fetcher checks this
  before it waits for anything, so a busy peer's further announcements
  are skipped at once and do not hold back other peers' announcements.
- A fetch permit is held only while the request runs. Each body holds a
  body slot instead, from the start of its fetch until the forwarder is
  done with it: 2 x max_concurrent_fetches slots. The fetched channel has
  room for every slot, so a finished fetch never waits holding anything.
  Dropping a body's delivery, however its fetch or forward ends, gives
  the slot, the hash and the peer back.
- A re-serialized body over 2,000,000 bytes is dropped as a failed fetch.
  The check runs after Zakura decoded the block, so it bounds the bodies
  the source holds, not process memory.
- Every AdvertiseBlock gets the not-seen error, whatever its source; only
  native-sourced ones are queued. In zakura-network 15.0.0 the gossip
  sink builds every advertisement with a native source, so the other case
  is not expected.
- The bootstrap_peers docs no longer call them the only peers dialed.

Native crate unit tests cover a busy source, a slow consumer, an
oversized body, a non-native advertisement and a dropped delivery.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #143, finding 1.

- A hash stays in progress until the forwarder drops its delivery, so a
  repeated announcement during the forward is coalesced.
- After a forward, the block is remembered under its announced hash only
  if the bridge now holds its own forward record for the header
  (RelayBridge::forwarded_at, a read of the existing dedup ring). Only
  ForwardClaim::commit writes that record, for a validated body whose
  send finished. A refused body, a failed or cancelled send and the
  emergency bypass, including its repeat reported as Deduplicated, leave
  nothing, so they never hide a later good copy. The forward mode is not
  used.
- At most 64 entries, each for 30 s from the bridge's record time. A
  repeat does not refresh an entry, and an entry counts only while the
  bridge still holds the same record, so it never outlasts the bridge's
  window or capacity. These are starting values matching the bridge
  defaults, not production-tuned values.
- The record is the bridge's local state. It says nothing about relay
  peers receiving the block or a node accepting it.

Tests drive the forwarder against the block body tests' bridge: bad then
good body, failed send, cancelled send, bypass and its repeat, a legacy
record time, a shorter bridge window and a bridge record pushed out by
capacity. One loopback test checks that a second peer's announcement of
a forwarded block is not fetched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #143, findings 6, 7, 8 and 9.

- Config: the source is mainnet only, and every bootstrap peer must
  parse the way zakura-network 15.0.0 reads it. Zakura skips an entry it
  cannot read with only a warning, so a bad list used to start quietly
  with no peers. The node id parser comes from zakura-quic, already in
  the lock through zakura-network and now a direct =0.5.0 dependency.
- The native start runs in its own task with a 30 s limit; the legacy
  peers start without waiting for it.
- A start that fails or times out, or a fetcher or forwarder that ends or
  panics while the source runs, ends main with an error. Nothing
  restarts them. The legacy peers ending first is their own outcome, and
  the native stop that follows is not reported as a failure.
- zakura_native::start takes the native network. The daemon passes
  mainnet; the tests keep testnet for the NU7 testnet fixture.
- README: the failure rules, and that an unusable identity directory is
  a Zakura warning and a new key, not a start error.

Tests: a forwarder or fetcher that dies is reported; a hanging start
times out while the legacy side runs; a failed start ends the run with
its error; the legacy side ending first is not a native failure; a start
on a taken UDP address fails; malformed bootstrap peers and testnet are
refused in config; Zakura's default bootstrap peers pass the check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#143 added 60 crate versions with no safe-to-deploy coverage. This
covers all of them without growing the exemption list much:

- Refresh imports (cargo vet regenerate imports): 4 versions.
- Trust Zakura's publishers: 34 Zakura crates are published by
  Zakura's release workflows through crates.io trusted publishing
  (github:zakura-core/common, github:zakura-core/zakura), and 3 by
  Sean Bowe (ebfull), whom librustzcash already trusts. Zakura ships a
  new major every few days, so trust holds across bumps where
  per-version exemptions would not. zakura-equihash is a critical crate
  and is now covered by trust, not an exemption.
- Trust publishers the imported audit sets already trust (djc,
  dtolnay, epage, nuttycom): 9 versions; this also retires 4 older
  exemptions.
- safe-to-deploy exemptions for the 10 small crates left (crossbeam
  deque and epoch move up from safe-to-run, crunchy too). sha2-asm is
  pulled in because zakura-chain enables sha2's asm feature; with
  zakura-native on, that unifies onto the relay's sha2.

Exemptions: 337 -> 340.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zmanian
zmanian marked this pull request as ready for review November 15, 2026 18:59
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T19:01:54.834309Z 597689d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@zmanian
zmanian merged commit 9bf8459 into main Nov 17, 2026
51 checks passed
zmanian pushed a commit that referenced this pull request Nov 17, 2026
…-body branch

Four conflicts, each kept from both sides:
- ci.yml: the bound-path diagnostic job and #148's zakura-native job are both kept.
- sovright-p2p-ingress dev-dependencies: both sets kept.
- nu7_transport_tests.rs: uses #148's LoopbackRelay helper; its RelayBridge
  now sets this branch's `raw_wire: RawWire::Legacy`.
- relay-sidecar main.rs: #145's per-block reconstruction log and metric go
  into this branch's compact_fast_path_outcome, so bound tasks log it too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants