Repository navigation
Release messageRouter persistent reference in ProducerConfig destructor (per-producer GC leak) - #511
Open
amit-motorq wants to merge 1 commit into
Conversation
ProducerConfig stores the user's messageRouter JS callback in a Napi::FunctionReference created with Napi::Persistent. Persistent handles are GC roots, so the callback — and everything its closure captures — can never be collected. Producer deliberately extends ProducerConfig's lifetime (shared_ptr held by the Producer wrapper), but neither Producer::Close nor ~Producer nor ~ProducerConfig ever calls Reset() on the reference. The result: any application that closes producers and creates new ones (e.g. periodic recycling of producers for config/credential rotation or partition-count refresh) accumulates unreachable-but-pinned callback graphs forever. Each pinned graph also keeps the JS Producer/Client ObjectWrap alive, so their finalizers never run and the underlying C++ producer/client allocations are never freed either — the leak shows up as steady RSS growth while the JS heap stays flat. Reset the reference in ~ProducerConfig. The destructor runs only after the last shared_ptr is released, i.e. after the Producer wrapper is gone and its C producer freed, so no routing callback can be invoked afterwards. Reset() is a no-op for a never-assigned reference (no messageRouter configured). Verified with heap snapshots taken before and after producer create/close cycles: closed producers' router closures were retained via (GC roots) -> (Global handles) -> messageRouter -> [context] -> this and grew linearly with churn; with this change they are collected and the retained set stays at the live producer count. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a producer is created with a custom
messageRouter,ProducerConfigstores the user's JS callback in aNapi::FunctionReferencecreated withNapi::Persistent(src/ProducerConfig.cc):Persistent handles are GC roots: the referenced JS function — and everything its closure captures — can never be garbage-collected for as long as the handle exists.
Producerdeliberately extends theProducerConfiglifetime (comment insrc/Producer.h: "Extend the lifetime of the producer config since it's env and router function could be used when sending messages"), but nothing ever releases the reference: notProducer::Close(), not~Producer(), and not~ProducerConfig()— which is empty:ProducerConfig::~ProducerConfig() {}Consequence: any application that closes producers and creates new ones (periodic producer recycling for config/credential rotation, partition-count refresh, connection recovery, etc.) accumulates unreachable-but-pinned callback graphs forever, one per closed producer. Each pinned graph also keeps the JS
Producer/ClientObjectWrap alive, so their finalizers never run — and because N-API addons free their underlying C++ objects from those finalizers, the C++ producer/client allocations are never freed either. The leak therefore presents as steady RSS growth per closed producer while the JS heap stays flat, making it easy to misdiagnose as a native-side leak.Evidence
Heap snapshots taken from a long-running Node service that recycles producers on a 30-minute TTL (via
client.createProducer→producer.close()cycles):(GC roots) → (Global handles) → messageRouter → [context] → captured scope → this → producer/client graph— i.e., a native persistent reference that survivedclose().Minimal reproduction sketch (needs a broker; run node with
--expose-gc):Fix
Call
messageRouter.Reset()in~ProducerConfig():Safety notes:
shared_ptr<ProducerConfig>is released — i.e., after theProducerwrapper has been collected and its C producer freed — so no routing callback can be invoked afterwards (the extended-lifetime contract inProducer.his preserved for the entire live lifetime of the producer).Reset()is a no-op for a never-assigned reference (producer created without a custommessageRouter).Testing
messageRoutersemantics during the producer's lifetime are untouched.--expose-gc(not enabled in the currentnpm testscript); the WeakRef-based reproduction above verifies the fix when run manually under--expose-gc.tests/producer.test.js— custom routing cases) cover that routing still works.