Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions docs/src/content/docs/durable/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ Two stdlib-only reference implementations ship in-tree:
- **`FileStore`** (`NewFileStore`): on-disk, a directory of per-instance
subdirectories, each an append-only journal, an atomic checkpoint, an
idempotency ledger, a dispatched-effect log and, once fenced, a `fence` file.
`Append` refuses a Record over its 16 MiB line limit with `ErrRecordTooLarge`.
Each append flushes to stable storage; each checkpoint uses
write-temp-plus-rename for crash-safe atomicity.
Use it for durability across restarts without a database.
Expand Down Expand Up @@ -126,8 +127,13 @@ in by implementing `HistoryStore` (the in-tree `MemStore` does so under

## A note on serialized payloads

Events, service done-data, actor done-data, and actor messages are recorded as
their JSON form. A parent reducer that type-asserts a non-JSON Go type from
`AssignCtx.Event` observes the JSON-decoded shape on a replayed `onDone`. A
Events, service done-data, actor done-data, and actor messages are recorded as their
JSON form. A parent reducer that type-asserts a non-JSON Go type from `AssignCtx.Event`
observes the JSON-decoded shape on a replayed `onDone`. A `WithEventData` payload is
JSON-decoded live as well as on replay, so a reducer gets the same value both ways: a
struct is a map and an integer a `float64`. An error stays an error with the same
message, with any invalid UTF-8 replaced. `Handle.Fire` refuses a payload it can't
record, or one over `WithMaxEventDataBytes` (8 MiB by default), with
`ErrEventDataEncoding`, and nothing applies. A
typed-codec option to carry the concrete Go value across the journal boundary is
reserved for a later, additive change.
28 changes: 25 additions & 3 deletions durable/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,30 @@ and this module adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.
fence the old owner: a stale write before then is accepted, and the new owner's
write at that step gets `ErrStepConflict`.

- **`WithMaxEventDataBytes`** caps a `WithEventData` payload. `ErrEventDataEncoding` reports
a refused payload, and `ErrRecordTooLarge` a Record too large for a `FileStore` line.
`Record.EventData` and `Record.EventDataError` hold the payload.

### Changed

- Requires Go 1.27.3.
- **A `Store` implementation has new work to do.** It must enforce fence tokens and
return `ErrStepConflict`, or `WithOwnerFence` protects nothing. Existing
implementations still compile, so nothing flags this.
- **A `WithEventData` payload reaches `Assign` decoded, live as well as on replay.** A
struct arrives as a map and an integer as a `float64`, which loses precision above
2^53, so an assertion to the original
type now fails at `Fire`, not at `Recover`. An error arrives as an error with the same
message, any invalid UTF-8 replaced, and `errors.Is` no longer matches it.
- **`Handle.Fire` can refuse a payload.** One with no JSON form that decodes again, or
over `WithMaxEventDataBytes` (8 MiB by default), returns the new `ErrEventDataEncoding`
before the kernel runs. Fires of that size used to pass, because the payload wasn't
journaled.
- **`FileStore.Append` refuses a Record over its 16 MiB line limit** with the new
`ErrRecordTooLarge`. It used to write a line `Load` couldn't read back unless a
checkpoint compacted it away. An oversized line now drops its `Snapshot` first, which
recovery never reads, so large checkpoints keep working.
- **A `Store` implementation has new work to do.** It must enforce fence tokens, return
`ErrStepConflict` and keep `Record.EventData` and `Record.EventDataError`, or
`WithOwnerFence` protects nothing and payloads are lost. Existing implementations
still compile, so nothing flags this.
- **`MemStore.Append` without `WithIdempotencyKey` refuses a Record JSON can't encode,**
as `FileStore` already did.

Expand All @@ -41,6 +59,10 @@ and this module adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.

### Fixed

- **`WithEventData` survives recovery.** `Handle.Fire` records the payload in the new
`Record.EventData` and `Record.EventDataError`, and `Recover` and `StateAt` supply it
on replay. This needs `state.FireEventData`, so `durable` requires `state` v1.1.0 or
later, and its `require` must move there before a `durable` tag.
- **Recording-clock buffer race.** Closing a step's `Record` (draining the
recording clock) and the clock-read index reads used to mirror timer deadlines
now take the recording clock's mutex, so they are safe against a `Now()` the
Expand Down
12 changes: 9 additions & 3 deletions durable/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ Two stdlib-only reference implementations ship in-tree:
- **`FileStore`** (`NewFileStore`): on-disk, a directory of per-instance
subdirectories, each an append-only journal, an atomic checkpoint, an
idempotency ledger, a dispatched-effect log and, once fenced, a `fence` file.
`Append` refuses a Record over its 16 MiB line limit with `ErrRecordTooLarge`.
Each append flushes to stable storage; each checkpoint uses
write-temp-plus-rename for crash-safe atomicity.
Use it for durability across restarts without a database.
Expand Down Expand Up @@ -137,9 +138,14 @@ otherwise.

## Caveat: serialized payloads

Events, service done-data, actor done-data, and actor messages are recorded as
their JSON form. A parent reducer that type-asserts a non-JSON Go type from
`AssignCtx.Event` observes the JSON-decoded shape on the replayed `onDone`. A
Events, service done-data, actor done-data, and actor messages are recorded as their
JSON form. A parent reducer that type-asserts a non-JSON Go type from `AssignCtx.Event`
observes the JSON-decoded shape on a replayed `onDone`. A `WithEventData` payload is
JSON-decoded live as well as on replay, so a reducer gets the same value both ways: a
struct is a map and an integer a `float64`. An error stays an error with the same
message, with any invalid UTF-8 replaced. `Handle.Fire` refuses a payload it can't
record, or one over `WithMaxEventDataBytes` (8 MiB by default), with
`ErrEventDataEncoding`, and nothing applies. A
typed-codec option to carry the concrete Go value across the journal boundary is
reserved for a later, additive change.

Expand Down
9 changes: 5 additions & 4 deletions durable/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,11 @@
//
// # Caveats
//
// Payloads are serialized at the journal boundary. Event, service done-data,
// actor done-data, and actor messages are recorded as their JSON form; a parent
// reducer that type-asserts a non-JSON Go type from AssignCtx.Event observes the
// JSON-decoded shape on the replayed onDone. A typed-codec option to carry the
// Payloads are serialized at the journal boundary. Event, service done-data, actor
// done-data, and actor messages are recorded as their JSON form; a parent reducer that
// type-asserts a non-JSON Go type from AssignCtx.Event observes the JSON-decoded shape
// on a replayed onDone. A WithEventData payload is JSON-decoded live as well as on
// replay, so a reducer gets the same value both ways. A typed-codec option to carry the
// concrete Go value across the boundary is reserved for a later, additive change.
//
// The kernel stays pure and dependency-free. The durable module is host-side:
Expand Down
14 changes: 14 additions & 0 deletions durable/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ const (
errorInstanceExists = "instance already exists"
errorEffectDispatch = "effect dispatch failed"
errorStepOutOfRange = "time-travel step out of range"
errorEventDataEncoding = "event data can't be recorded"
errorRecordTooLarge = "record too large for the journal"
)

// Sentinel errors a Store reports. Callers match them with errors.Is.
Expand All @@ -38,6 +40,18 @@ var (
// a newer owner has taken the instance over. Nothing is written.
ErrFenced = errors.New(errorPrefix + errorFenced)

// ErrEventDataEncoding wraps a Handle.Fire refused before the kernel ran, because
// its state.WithEventData payload has no JSON form that decodes again, panics while
// encoding (such as an error that's a typed nil), or exceeds WithMaxEventDataBytes.
// Nothing applied, so the Handle stays usable.
ErrEventDataEncoding = errors.New(errorPrefix + errorEventDataEncoding)

// ErrRecordTooLarge is reported by FileStore.Append for a Record whose journal line
// exceeds the store's line limit, which a later Load could not read back, even
// without its Snapshot. No journal line is written, but a Handle's kernel has already
// applied the step.
ErrRecordTooLarge = errors.New(errorPrefix + errorRecordTooLarge)

// ErrCheckpointNotAdvancing is reported by Store.Checkpoint when throughStep
// does not advance beyond the instance's current checkpoint.
ErrCheckpointNotAdvancing = errors.New(errorPrefix + errorCheckpointNotAdvancing)
Expand Down
Loading
Loading