Skip to content

http: emit 'drain' after flushing coalesced chunks - #66782

Open
junyup0503 wants to merge 4 commits into
nodejs:mainfrom
junyup0503:http-drain-multibyte
Open

junyup0503 wants to merge 4 commits into
nodejs:mainfrom
junyup0503:http-drain-multibyte

Conversation

@junyup0503

Copy link
Copy Markdown

Since #64987, chunked writes made while the response is auto-corked are coalesced, and write() returns false based on the byte length of the buffered chunks. When uncork() flushes them, net.Socket counts string chunks in UTF-16 code units, so with multi-byte text the socket can stay below its own highWaterMark and never emits 'drain'. If the flushed writes complete asynchronously, the check at the end of uncork() also runs while bytes are still pending, so the response stalls with writableNeedDrain === true and writableLength === 0.

This re-checks the drain condition in the write callback of the flushed data, unless the message or its socket has been destroyed. It does not change the accounting itself (e.g. encoding the coalesced strings to Buffers), to avoid adding cost to the write path that #64987 optimized.

The new test uses a Duplex that counts strings like net.Socket and completes writes asynchronously, so it does not depend on OS socket buffer sizes. It hangs on main without the change and passes with it. It also checks that 'drain' is not emitted after the response is destroyed from a write callback. parallel/test-http*, parallel/test-stream*, parallel/test-net*, sequential/test-http* and make lint-js pass locally.

Fixes: #66781
Refs: #64987

I used an AI coding agent to help investigate and write this change. I reproduced the issue and verified the fix and the tests myself.

Since chunked writes are coalesced while the message is corked,
write() decides backpressure from the byte length of the buffered
chunks and sets kNeedDrain. When uncork() flushes them, net.Socket
accounts string chunks in UTF-16 code units, so with multi-byte text it
can stay below its own highWaterMark. It then never needs to drain and
never emits 'drain'. If the flushed writes complete asynchronously, the
check at the end of uncork() also runs while bytes are still pending,
so the response stalls with writableNeedDrain === true and
writableLength === 0.

Re-check the drain condition once the flushed data has been written,
unless the message or its socket has been destroyed in the meantime.

Fixes: nodejs#66781
Refs: nodejs#64987
Assisted-by: AI coding agent
Signed-off-by: junyup0503 <97156898+junyup0503@users.noreply.github.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/http
  • @nodejs/net

@nodejs-github-bot nodejs-github-bot added http Issues and PRs related to the http subsystem. needs-ci PRs that need a full CI run. labels Oct 16, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Welcome to Node.js, and thank you for your first contribution!

Before review, please take a moment to read:

Please make sure every commit is signed off. For a first pull request, GitHub Actions require collaborator approval and Jenkins CI must be started by a collaborator or triager, so an initial wait is normal.

@par333k

par333k commented Oct 16, 2026 •

Copy link
Copy Markdown
const currentSocket = this[kSocket]

The socket reference is captured when uncork() runs, so it is null if the message had no socket yet and the write was buffered in outputData. Re-reading this[kSocket] in the callback checks the socket the data was actually flushed to. In practice _flush() usually clears kNeedDrain before this callback runs, so this is a defensive guard rather than a fix for an observed crash.

Comment thread lib/internal/http/outgoing.js Outdated
Comment on lines +382 to +384
// Re-check once the flushed data has been written.
if (!err && this.writableNeedDrain && !socket.destroyed &&
this.writableLength === 0) {

@par333k par333k Oct 16, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Re-check once the flushed data has been written.
if (!err && this.writableNeedDrain && !socket.destroyed &&
this.writableLength === 0) {
const currentSocket = this[kSocket];
if (!err && this.writableNeedDrain && currentSocket &&
!currentSocket.destroyed && this.writableLength === 0) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I think without this there is likely a reachable crash here, good catch @par333k

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion! @par333k
Applied in 96a0377. The regression test and five related HTTP tests all pass.

@codecov

codecov Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.69%. Comparing base (1c90519) to head (3f263a8).
⚠️ Report is 258 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #66782      +/-   ##
==========================================
+ Coverage   90.49%   90.69%   +0.19%     
==========================================
  Files         797      797              
  Lines      277598   278906    +1308     
  Branches    53336    53631     +295     
==========================================
+ Hits       251220   252950    +1730     
+ Misses      16790    16311     -479     
- Partials     9588     9645      +57     
Files with missing lines Coverage Δ
lib/internal/http/outgoing.js 97.99% <100.00%> (+0.01%) ⬆️

... and 128 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pimterry pimterry left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment here needs resolving, but otherwise I think this is good to go and fixes a real regression. Thanks for the fix @junyup0503!

Comment thread lib/internal/http/outgoing.js Outdated
Comment on lines +382 to +384
// Re-check once the flushed data has been written.
if (!err && this.writableNeedDrain && !socket.destroyed &&
this.writableLength === 0) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I think without this there is likely a reachable crash here, good catch @par333k

Read the current socket after the coalesced write callbacks run, since
uncork() may initially buffer data before a socket is assigned. Check
that the current socket exists and is not destroyed before emitting
drain.

Refs: nodejs#66782 (comment)
Assisted-by: AI coding agent
Signed-off-by: junyup0503 <97156898+junyup0503@users.noreply.github.com>
@pimterry pimterry added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 30, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Oct 30, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@junyup0503

Copy link
Copy Markdown
Author

I applied @par333k’s suggestion locally but missed the co-author attribution. Could we include them as a co-author in the final commit when landing this PR?

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@inoway46 inoway46 added the resume-ci Add this label to resume the latest eligible Jenkins CI run on a PR with an approving review. label Nov 1, 2026
@github-actions github-actions Bot added resume-ci-failed Resuming CI with the resume-ci label failed and requires manual intervention. and removed resume-ci Add this label to resume the latest eligible Jenkins CI run on a PR with an approving review. labels Nov 1, 2026
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Failed to resume CI

✖ Failed to load PR CI job 78767: Unexpected token '<', ..."

Full Auto Start CI output
�[36m⠋�[39m Validating Jenkins credentials
�[36m⠋�[39m Validating Jenkins credentials
✔  Jenkins credentials valid
�[36m⠙�[39m Looking for CI runs for pull request 66782
�[36m⠙�[39m Looking for CI runs for pull request 66782
�[36m⠙�[39m Getting PR from nodejs/node/pull/66782
�[36m⠙�[39m Getting reviews from nodejs/node/pull/66782
�[36m⠙�[39m Getting comments from nodejs/node/pull/66782
✔  Found PR CI job 78767
�[36m⠹�[39m Querying data for job/node-test-pull-request/78767/
�[36m⠹�[39m Querying data for job/node-test-pull-request/78767/
�[36m⠹�[39m Querying API for job/node-test-pull-request/78767/

[DEBUG] [SyntaxError: Unexpected token '<', ..."    
  
  <!DOCTYPE "... is not valid JSON]
✖  Failed to load PR CI job 78767: Unexpected token '<', ..."    
  
  <!DOCTYPE "... is not valid JSON

View workflow run

Comment thread lib/internal/http/outgoing.js Outdated
Comment on lines +384 to +385
if (!err && this.writableNeedDrain && currentSocket &&
!currentSocket.destroyed && this.writableLength === 0) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!err && this.writableNeedDrain && currentSocket &&
!currentSocket.destroyed && this.writableLength === 0) {
if (!err && this.writableNeedDrain && !this.destroyed && this.writableLength === 0) {

Maybe?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in 3f263a8, thanks. CI needs a fresh run — the auto-resume failed with an infra error.

Rely on the message's own destroyed state instead of the socket's when
re-checking the drain condition after the coalesced chunks are flushed.
writableNeedDrain already excludes destroyed and finished messages.

Refs: nodejs#66782 (comment)
Co-authored-by: Robert Nagy <ronagy@icloud.com>
Co-authored-by: Park Jin Woong <35551417+par333k@users.noreply.github.com>
Assisted-by: AI coding agent
Signed-off-by: junyup0503 <97156898+junyup0503@users.noreply.github.com>
@junyup0503
junyup0503 requested a review from ronag November 3, 2026 05:49
@pimterry pimterry added the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Nov 3, 2026
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. Only starts once the PR has an approving review. label Nov 3, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@sxa

sxa commented Nov 3, 2026

Copy link
Copy Markdown
Member

The AIX test here is an issue while will not occur on a re-run, so once the existing PR run is complete a re-run should allow it to pass

@aduh95

aduh95 commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Related Windows failure:


---
duration_ms: 195.014
exitcode: 1
severity: fail
stack: |-
  node:assert:173
    throw err;
    ^

  AssertionError [ERR_ASSERTION]: function should not have been called at c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http-outgoing-chunked-drain-multibyte.js:63
      at ServerResponse.mustNotCall (c:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:570:12)
      at ServerResponse.emit (node:events:514:20)
      at socketOnDrain (node:internal/http/server:925:9)
      at Duplex.emit (node:events:514:20)
      at afterWrite (node:internal/streams/writable:744:12)
      at onwrite (node:internal/streams/writable:727:7)
      at WritableState.onwrite (node:internal/streams/writable:351:26)
      at Immediate.flush [as _onImmediate] (c:\workspace\node-test-binary-windows-js-suites\node\test\parallel\test-http-outgoing-chunked-drain-multibyte.js:28:22)
      at process.processImmediate (node:internal/timers:567:21) {
    generatedMessage: false,
    code: 'ERR_ASSERTION',
    actual: undefined,
    expected: undefined,
    operator: 'fail',
    diff: 'simple'
  }

  Node.js v28.0.0-pre
...

The default stream highWaterMark is 16 KiB on Windows and 64 KiB
elsewhere. With the smaller value the fake socket exceeds its own
highWaterMark in UTF-16 code units and emits 'drain' by itself before
the write callback destroys the response, which trips mustNotCall().
Pin the highWaterMark so the payload sizes stay on the intended side of
it on every platform, and assert that the socket never drains on its
own in the destroy scenario.

Refs: nodejs#66782 (comment)
Assisted-by: AI coding agent
Signed-off-by: junyup0503 <97156898+junyup0503@users.noreply.github.com>
@junyup0503

Copy link
Copy Markdown
Author

Thanks. That was a test assumption: the default highWaterMark is 16 KiB on Windows, so the fake socket emitted 'drain' on its own before the write callback destroyed the response. Fixed in 0ecaa63 by pinning the socket's highWaterMark; reproduced and verified locally with the 16 KiB default. A fresh CI run is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

http Issues and PRs related to the http subsystem. needs-ci PRs that need a full CI run. resume-ci-failed Resuming CI with the resume-ci label failed and requires manual intervention.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

http: 'drain' is never emitted for chunked responses with multi-byte strings (v24.22.0 regression)

8 participants