Skip to content

perf(dataplane): reduce IPv6 resident memory footprint in multi-shard mode - #261

Merged
vxfemboy merged 7 commits into
mainfrom
perf/ipv6-memory-footprint
Nov 18, 2026
Merged

vxfemboy merged 7 commits into
mainfrom
perf/ipv6-memory-footprint

Conversation

@vxfemboy

Copy link
Copy Markdown
Member

Summary

Resolves #255.

Addresses the IPv6 multi-shard resident memory footprint disparity where resident set size (RSS) peaked at 335.8 MB during IPv6 testing compared to 171.4 MB during IPv4 testing under shards=auto (8 worker threads).

Root Cause

  1. FecReassembler in Transport pooled up to 256 in-flight objects and repair buffers across 3 flow classes per shard, holding up to ~393 MB across 8 worker shards once exercised.
  2. In FlowTable::evict_expired, an active flow at the head of self.order broke the eviction loop immediately, permanently starving and retaining expired background IPv6 discovery/multicast flows queued behind it.
  3. tun_drained and tick_tun_drained in DataPlane dynamically pushed Vec<u8> buffers on bursts >32 packets without trimming back down during idle sweeps.

Key Changes

  1. FecReassembler Pool Reduction: Sized FecReassembler capacity in crates/yip-transport/src/lib.rs from 256 to 64 objects per class, slashing max pool retention across 8 shards from ~393 MB to ~98 MB.
  2. FlowTable Sizing & Starvation Fix:
    • Sized FlowTable in Classifier::new to 1,024 entries with 10s TTL.
    • Refactored FlowTable::evict_expired in crates/yip-transport/src/flow.rs with order_ms tracking to rotate active entries to the back of the queue, allowing expired flows queued behind them to be evicted.
    • Bypassed heuristic flow table tracking in crates/yip-transport/src/classify.rs for non-TCP/UDP control protocols (ICMPv4, ICMPv6, multicast), preventing background discovery traffic from churning the flow table.
  3. DataPlane Scratch Hygiene:
    • In DataPlane::tick (bin/yipd/src/dataplane.rs), truncated and shrunk tun_drained and tick_tun_drained back to 32 elements when grown after reorder bursts.
    • Reclaimed oversized scratch buffers exceeding 65,792 bytes during idle tick sweeps.

Verification

  • cargo fmt --all -- --check: Passed cleanly.
  • cargo clippy --workspace --all-targets -- -D warnings: Passed cleanly with zero warnings.
  • cargo test --workspace: All unit, integration, and netns tests passed.
  • cargo deny check: Passed.
  • cargo shear: Passed.
  • cargo mutants -p yip-transport --file crates/yip-transport/src/flow.rs --file crates/yip-transport/src/classify.rs: 77 mutants tested (75 caught, 2 unviable, 0 missed).
  • Zero unsafe code and zero emojis strictly preserved.

@vxfemboy
vxfemboy temporarily deployed to benchmark-approval November 18, 2026 18:34 — with GitHub Actions Inactive
@vxfemboy
vxfemboy temporarily deployed to benchmark-approval November 18, 2026 18:34 — with GitHub Actions Inactive
@vxfemboy
vxfemboy temporarily deployed to benchmark-approval November 18, 2026 18:34 — with GitHub Actions Inactive
@vxfemboy
vxfemboy temporarily deployed to benchmark-approval November 18, 2026 18:34 — with GitHub Actions Inactive
@vxfemboy
vxfemboy temporarily deployed to benchmark-approval November 18, 2026 19:07 — with GitHub Actions Inactive
@vxfemboy
vxfemboy temporarily deployed to benchmark-approval November 18, 2026 19:07 — with GitHub Actions Inactive
@vxfemboy
vxfemboy temporarily deployed to benchmark-approval November 18, 2026 19:15 — with GitHub Actions Inactive
@vxfemboy
vxfemboy temporarily deployed to benchmark-approval November 18, 2026 19:46 — with GitHub Actions Inactive
@vxfemboy
vxfemboy merged commit 1a6035e into main Nov 18, 2026
14 of 15 checks passed
@vxfemboy
vxfemboy deleted the perf/ipv6-memory-footprint branch November 18, 2026 20:18

This branch was previously deployed

1 inactive deployment
benchmark-approval — ab3cce84 Deployed Nov 18, 2026 by vxfemboy via iperf-netem-bench #207
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(dataplane): reduce IPv6 resident memory footprint in multi-shard mode

1 participant