Skip to content

Wake pending TCP operations and finish the session on reset - #109

Open
szonov wants to merge 1 commit into
narrowlink:mainfrom
szonov:fix/wake-on-tcp-reset
Open

szonov wants to merge 1 commit into
narrowlink:mainfrom
szonov:fix/wake-on-tcp-reset

Conversation

@szonov

@szonov szonov commented Oct 18, 2026

Copy link
Copy Markdown

Receiving RST currently changes the TCP state to Closed and continues the session loop. It does not wake a pending reader or writer, and the session task may keep waiting for another packet or timer. A pending shutdown also waits for the task to finish.

After marking the connection closed, release the state lock, wake the pending reader and writer, and exit the loop. The existing task epilogue then sends the destroy notification and completes the shutdown waker. Preserve the current stream behavior: already accepted data is delivered before EOF, and writes to a closed connection fail.

Also close the registration races on a multi-threaded runtime:

  • After storing the read waker, recheck Closed. If RST ran before registration, schedule the reader again to observe EOF; if RST runs later, it sees the stored waker.
  • Before registering shutdown, recheck Shutdown::Ready under the same mutex used by the task epilogue. Never overwrite a completed shutdown with Pending, and refresh the waker when the waiting task changes.

This addresses incoming RST, independently of #104 (peer FIN/EOF) and #105 (shutdown waiting for the last data ACK).

Tests

  • reset_wakes_pending_read_write_and_shutdown: exercises both RST and RST|ACK. Leaves data in flight, closes the peer's receive window to block a write, registers separate read/write/shutdown wakers, then sends the reset. Checks all three wakers, completed session task, destroy notification, EOF, rejected writes, and completed shutdown. The paused clock does not advance: a timer or another packet cannot mask a missing wakeup.
  • reset_preserves_queued_data_before_eof: queues incoming data before RST and reads it in small chunks after the session task finishes, checking that queued and temporary-buffer bytes arrive exactly once before EOF.
  • reset_during_read_registration_does_not_lose_wakeup: on a two-worker runtime, completes a real incoming RST between the read-state snapshot and waker registration. Checks that the read is ready or explicitly rescheduled, then observes EOF.
  • reset_during_shutdown_registration_does_not_lose_wakeup: completes RST in the corresponding shutdown-registration window and checks that shutdown immediately returns ready without overwriting its completed state.
  • reset_wakes_the_latest_shutdown_waiter: changes the shutdown waker and checks that RST wakes the current waiter rather than the stale one.
  • handshake_wait_times_out_without_peer_ack: with Tokio time paused, leaves the handshake incomplete and checks that the fixture's wait expires after one virtual second instead of spinning forever.
  • handshake_wait_is_woken_by_peer_ack: checks that a pending handshake wait is woken by the peer ACK and completes without advancing virtual time.

The first two tests fail on unpatched upstream. The two registration-race tests fail on the initial RST patch without the registration fixes; independently removing each new fix also makes its corresponding test fail. Both multi-threaded tests pass in 20 consecutive runs after the fixes.

The registration tests use thread-local, test-only hooks immediately after the state snapshots. A real TCP worker processes the reset and finishes while the polling thread is paused there; states and notifications are not mocked. The hooks are excluded from non-test builds. The handshake helper uses the existing state-transition notification, registering its waker under the state lock, rather than spinning with yield_now. Its timeout therefore works with paused Tokio time too. Waiting for the initial SYN-ACK is also bounded. Repeating the review's missing-ACK fault injection now produces the expected handshake timeout failure without hanging.

Validated on upstream 44bb34fe50bd9c04290b4b4f9bbeece1626a95d4 on macOS arm64 with Rust 1.99.0:

  • cargo fmt -- --check
  • cargo test --offline --locked --all-features: 43 unit tests and 23 documentation tests pass.
  • cargo clippy --offline --locked --all-targets -- -D warnings

The same commit and Cargo.lock were also validated on Linux ARM64 (Debian 12, Rust 1.98.1): all 43 unit tests and 23 documentation tests pass, as do formatting, Clippy with all targets/features and -D warnings, and the examples/tests build. The registration-race and handshake test groups were also run separately and pass. Checks ran offline in a container with networking disabled.

No changes to dependencies or public APIs. No live TUN/network test was performed.

Native Linux AMD64 validation also passes on Debian 13 with Rust 1.98.1, using the same commit and Cargo.lock: 43 unit tests, 23 documentation tests, formatting, Clippy for all targets/features with -D warnings, and the examples/tests build. The registration-race and handshake groups also pass separately on AMD64.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant