Repository navigation
fix(durable): journal WithEventData payloads for recovery - #230
Conversation
ad5c962 to
1047e24
Compare
1047e24 to
05d4a81
Compare
05d4a81 to
6157108
Compare
b953e8c to
a9d0723
Compare
a9d0723 to
57218cc
Compare
57218cc to
cb86fd7
Compare
ACR on this PR (one reviewer per dimension)Round 2 combined
Main changes the reviews drove:
Known and left open: listed under "Please check these" in the description. All seven dimensions are confirmed. |
29b6cc0 to
53ce7e5
Compare
53ce7e5 to
90b2432
Compare
90b2432 to
33be29d
Compare
33be29d to
3679dcd
Compare
3679dcd to
a67e2a1
Compare
a67e2a1 to
c773966
Compare
1836226 to
fd16d32
Compare
WithEventData payloads are recorded and supplied again by Recover and StateAt, through the additive state.FireEventData. The live Fire now uses the decoded payload too, so the live run and a replay can't differ. An error payload is rebuilt from its message. A payload that can't be recorded, or is over WithMaxEventDataBytes, refuses the Fire before the kernel runs. FileStore refuses a Record too large for it to read back. Relates to #214. Signed-off-by: Ray Karnes <ray.karnes@stablekernel.com>
fd16d32 to
27c5701
Compare
Matches the change on main: sentinel text in private constants, and the "durable: " prefix rather than "crucible/durable: ". Signed-off-by: Ray Karnes <ray.karnes@stablekernel.com>
Signed-off-by: Ray Karnes <ray.karnes@stablekernel.com>
What this change does
WithEventDatapayloads now survive recovery, and a live run and a replay see the same payload, apart from the known cases listed below. It sits onmainas the bottom of stack #239.Handle.Firerecords the payload in the newRecord.EventData, andRecoverandStateAtsupply it on replay.state.FireEventDatais a new additive function in the frozenstatemodule, so a host can read the payload fromFireOptions.Firenow uses the decoded payload too.Assignsees the same value live and on replay: a struct arrives as a map and an integer as afloat64.errors.New, live and on replay. The service and actor paths rebuild errors the same way on replay.errors.Isno longer matches it.WithMaxEventDataBytes(8 MiB by default), refuses theFirewith the newErrEventDataEncodingbefore the kernel runs. The Handle stays usable.durable:error text from fix(durable): add write fencing and reject conflicting steps #229.FileStore.Appendrefuses a Record over its 16 MiB line limit with the newErrRecordTooLarge, instead of writing a lineLoadcan't read. An oversized line drops itsSnapshotfirst, which recovery never reads, so large checkpoints keep working.Please check these:
in.Event.(int)now fails atFire. That assertion already failed on recovery. Six of seven reviewers backed this, over keeping the live value and documenting the difference.durableneedsstate.FireEventData, which isn't instate1.0.0.statereleases v1.1.0 first, anddurable'srequiremoves there before adurabletag.FileStoreline, from a raised cap or a largeEvent,EntriesorEffects, getsErrRecordTooLargeafter the kernel ran, which leaves the Handle out of step with the journal. fix(durable): lock the Handle and refuse steps after a failed Append #233 makes that Handle refuse further steps.Storethat encodes fields one by one must keep the two newRecordfields, or payloads are lost.Recoveror checkpoint, and a map with collidingMarshalTextkeys encodes differently between runs, though the journal still matches live. An event with no JSON form, such as+Infas afloat64event, journals an emptyEvent, so replay skips that step and its payload. That predates this PR. An error kept in context becomes a map once a checkpoint carries it, and in anerror-typed fieldRecoverthen fails. A pointer chain millions deep overflows the stack injson.Marshal, which Go can't recover.Why
Payloads passed with
WithEventDatawere lost on recovery, so a transition'sAssignsaw different data on replay than it did live. Journaling alone isn't enough, because live saw the Go value and replay its JSON form.Relates to: #214
Evidence
Run locally on Go 1.27.3 at
43edd5e:mage checkgovulncheckacross the suitego test ./...indurableTestRecover_FireWithEventData_MatchesLivego test -race ./...indurablego test ./...instateGOWORK=off go build ./...indurableCI is green at the same commit: 112 of 112 checks pass.
Checklist
git commit -s) per the DCOtype: subject)mage checkpasses locally (lint + race tests + govulncheck)durable/event_data_test.go,state/fire_event_data_test.go