Repository navigation
cache: replace read/dirty with a concurrent hash-trie - #6
Merged
Merged
Conversation
twmb
force-pushed
the
trie-cache
branch
4 times, most recently
from
September 27, 2026 05:41
b4b66ce to
ef85207
Compare
twmb
commented
Sep 27, 2026
The sync.Map-style read map and locked dirty map are replaced with a concurrent hash-trie modeled on Go's internal HashTrieMap. Loads are lock-free, inserts and deletes lock only the trie node they change, and Delete unlinks the entry immediately rather than waiting for a promotion. Get runs the miss function in the calling goroutine unless it has a stale value to return while the load runs. If an inline miss panics or calls runtime.Goexit, the panic reaches the caller and Gets waiting on that load run their own load. Gets that collapsed onto a load all return its result, including an error that is not cached. Expiry uses the monotonic clock, so a wall clock step no longer moves TTLs. Expire does nothing to a loading or already expired value, and never moves an expiry later. Expire replaces the loading with an expired copy rather than writing its expiry, so a concurrent Set or Swap can no longer make it lost. A stale window starts when the value expires, and TryGet, Get, and Clean agree on it. CompareAndSwap and CompareAndDelete only match unexpired, unerrored values. MaxIdleAge skips an extension that would move the expiry by less than 1s (or age/64), so hot keys do not write their expiry on every hit. The autoclean goroutine holds its cache through a weak pointer, so a cache that is never stopped is still garbage collected and its goroutine exits. StopAutoClean waits for the goroutine to exit. NewItem and NewSet now honor AutoCleanInterval, which they previously ignored. A loading holds its error and stale in a side struct that only errored or stale-carrying loadings allocate, so a Cache[int, int] value takes 48 bytes. Lookups do not make the key escape, Clean does not allocate, Get with no expiry does not read the clock, and Set.Get no longer allocates a wrapper closure per call. The tests cover every statement of the trie and cache. They include the stdlib sync.Map tests, a property test against a mutex guarded map, structural checks of the trie, hook-driven tests for specific races, and a linearizability checker. The checker runs every operation, Get with its singleflight rules, and expiry under a frozen test clock, including with forced hash collisions. CI runs them with the race detector on amd64 and arm64, and without it on 386. This requires Go 1.25: the package uses a generic type alias (1.24) and the tests use sync.WaitGroup.Go (1.25).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This replaces the sync.Map-style read map and locked dirty map with a
concurrent hash-trie modeled on Go's internal HashTrieMap, which also backs
sync.Map since Go 1.24. Loads are lock-free, inserts and deletes lock only the
trie node they change, and Delete unlinks the entry immediately.
Against main (Apple M2, Go 1.27, interleaved runs):
The heap per key is higher: an uncompressed 16-way trie has about 0.36
interior nodes per key. Leaf buckets would bring it to about 108B, but they
need a version-checked lock-free read path, which is not worth ~30B per key.
Behavior changes:
value to return. A panic in an inline miss reaches the caller, and Gets
waiting on that load run their own load.
that is not cached (MaxErrorAge(0)). On main, each waiter ran its own load.
whether Get or Set wrote it; main only did this for values written by Set
or Swap. The window starts when the value expires, and Get, TryGet, and
Clean agree on it.
longer be lost to a concurrent Set or Swap (it replaces the loading
instead of writing the old one's expiry; main had the same race).
age/64), so an entry can expire up to 1s early.
stopped is still garbage collected, and the goroutine exits.
StopAutoClean now waits for the goroutine to exit.
tests use sync.WaitGroup.Go (1.25). Go 1.24 and 1.25 are both past end of
support.
The public API is otherwise unchanged. The tests cover 100% of the
statements in cache.go and trie.go, stable across repeated runs. A
linearizability checker runs every operation, Get with its singleflight
rules, and expiry under a frozen test clock, including with forced hash
collisions; it found the Expire race above.
Every remaining allocation, for a Cache[int, int]:
key's life.
read values without locks, so a value is never updated in place. sync.Pool
reuse is unsafe because a reader may still hold the old pointer.
carry 16B for the error.
goroutine's closure.
concurrent Set or Swap cannot make the Expire lost.
including hits: the stale refresh passes it to a goroutine, and Go's escape
analysis is per function. Avoiding it needs a new API.
Changelog
cache.Cacheis now backed by a concurrent hash-trie. Loads are lock-free,and Get, Set, and Delete are faster and allocate less.
Getruns the miss function inline when it has no stale value to return;a panic in the miss function now reaches the caller.
Get as Stale while their stale window is open, as it already did for values
written by Set.
MaxIdleAgeextensions under 1s are skipped; the autoclean goroutine nolonger keeps an unstopped cache alive;
NewItemandNewSethonorAutoCleanInterval.