Repository navigation
fix(ctrltest): judge garbage collection from a live read and precondition every write - #7
Merged
Merged
Conversation
…tion every write The simulated garbage collector acted on a List taken at the start of the sweep: it deleted dependents by name with no preconditions, so a dependent recreated under the same name after the List (new UID), or the same dependent adopted by a live owner since (same UID), was deleted anyway. Orphan strips and finalizer removal were strategic merge patches of the listed copy with no resourceVersion. Each candidate now goes through collect, which follows upstream's attemptToDeleteItem: read the dependent's metadata live, classify each owner from a fresh lookup (solid, dangling, or waiting on foreground deletion), strip the dangling and waiting references if a solid owner remains, and otherwise delete under the live read's UID and resourceVersion. A conflict is followed by one re-read, and the delete is retried under the UID alone only if the object is the same incarnation with unchanged owner references, as upstream's deleteObject does, so an object under constant unrelated writes is still collected. An owner the sim cannot resolve ends the attempt with nothing done rather than counting as solid. Orphan strips and finalizer removal are JSON merge patches under a resourceVersion lock from a fresh read (strategic merge is refused for custom resources); finalizer removal checks the owner's UID and retries on conflict, like upstream's removeFinalizer. The orphan finalizer now comes off only after every strip has landed, so the comment saying orphaning always finishes in one sweep is removed, as is the one claiming strips update the listed objects in place, which they no longer do. The new tests drive a hand-stepped sim against envtest from a stale List, with hooks between the sim's read and its write. Signed-off-by: Brent Graveland <graveland@supabase.io>
go mod tidy adds checksums for the otel, stdr and gonum modules in the build graph, which the Lint job's tidy check requires. Signed-off-by: Brent Graveland <graveland@supabase.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Diff: 62% test, 36% code, 2% generated (776 lines changed)
Problem
ctrltest's simulated garbage collector deletes dependents from a List it took at the start of the sweep. It deletes by name, with no preconditions. If an owner and its children are recreated under the same names before the delete lands, the sim deletes the new children. If a listed child is adopted by a live owner in the meantime (same UID), the sim deletes that child too. In multigres-operator'sTestThrash/child_CR_thrash, under load, this deleted a recreated Shard'spool-0Pod and PVCs, leaving two PRIMARY Pods and Terminating PVCs. The real GC cannot do this, so consumers end up chasing failures that exist only in the harness.Fix
Candidates go through
collect, modelled on upstreamattemptToDeleteItem:A conflict triggers a re-read. The delete is retried under the UID alone only for the same incarnation with unchanged owners (upstream
deleteObject), so objects under constant writes are still collected. An unresolvable owner stops the attempt with nothing changed. Orphan strips and finalizer removal read fresh and use JSON merge patches under a resourceVersion lock. Finalizer removal also checks the owner's UID and retries on conflict. The orphan finalizer is removed only after every strip has landed.Tests
gc_stale_test.gosteps a sim against envtest from a stale List, with hooks between the sim's read and its write. The cases are a recreated child, adoption before and after the read, a recreated owner, a concurrent orphan strip, write churn, and a genuine orphan, which is still collected. Of 15 guard mutations, 14 fail the tests. The survivor drops the UID from the first delete and is equivalent, since a recreated object always has a new resourceVersion.Risks
blockOwnerDeletionand propagation policy are still not modelled (documented).go.sum.main's Lint job fails its tidy check without it.