Repository navigation
Fix caller lifecycle and back off lease-loss recovery - #73
Conversation
Give overlapping synchronous callers separate reusable workers so one pass cannot discard another turn’s staged intents. Refresh each runtime role’s process registration before claiming work after stale cleanup or retention.
|
Docs7 for cardmagic/solid-objects-js
Commit |
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
@greptileai Please review the latest head, 1d4d254. The reset race now has a failing-before/passing-after regression and stopped workers are excluded from the idle pool. |
|
@greptileai Please review the latest head, 59dd0d3. The caught rejection now has an explicit Error type guard; all lifecycle regressions and type/style checks pass. |
A lost activation counts as an attempted turn, so the demo worker could exit before its reclaimed message committed on a delayed CI runner. Drain available turns and assert completion before checking state. Additional attempts can repeat external effects while the durable state still commits exactly once. Reproduce the failure with a real SQLite lease expiry before the change. The new regression and a Node 24.4.0 subprocess stall now pass, along with 638 tests and the type, style, and documentation checks.
|
@greptileai Please review the latest head 5979a9e. The floor CI failure was reproduced with a 300 ms synchronous stall: the demo exited after a lost turn before the message committed. The demo now drains retries and checks completion; a permanent SQLite regression failed before the fix and passes afterward. Normal and injected-stall Node 24.4.0 demos, 638 tests, and type/style/documentation checks pass locally. |
The PostgreSQL and MySQL commit-fencing tests could lose their 100 ms lease before reaching the commit barrier on a delayed runner. A 200 ms handler stall reproduces the 15-second timeout in both adapters. Keep that stall and advance a controlled database clock only after the commit has acquired its locks. The competing claim must still wait and must not recover the committed message. No runtime defaults change. Validate 69 PostgreSQL checks, 56 MySQL checks with 7 conditional skips, 638 default-suite checks with 32 conditional skips, types, and format.
|
@greptileai Please review the latest head 78338eb. The remaining MySQL CI failure was reproduced in both server adapters: a 200 ms handler delay expired the old 100 ms lease before the commit barrier. The tests now advance a controlled database clock after the commit acquires its locks, keeping that delay and the original fencing assertions. PostgreSQL 69/69, MySQL 56 passed with 7 conditional skips, default suite 638 passed with 32 conditional skips, and quality checks pass. |
Align the package version, exported runtime version, and dated changelog so the caller lifecycle fixes can be tagged after merging.
|
@greptileai Please review the latest release-preparation commit. It changes only package.json, src/version.ts, and the dated changelog to prepare JavaScript 0.17.8. All 638 tests, type/style/documentation checks, build, release notes generation, and the packed-package smoke test pass. Ruby remains regression tests only, with no release bump. |
Independently reported lease loss could retry indefinitely while resolved SQLite promises prevented timers and input from running. Yield between busy worker and drain passes, preserving immediate instrumentation aborts. Recover attempt one immediately, then persist the existing retry delay for total attempts minus one. Preserve fences, mailbox order, and handler failure limits. Add child-process starvation regressions, durable recovery coverage, and the matching Ruby telemetry contract and documentation. Baseline: 20 new regression failures and two passing controls. All pass with the fix, alongside full SQLite, PostgreSQL, and MySQL suites.
cardmagic
changed the title
|
@greptileai Please review the current head 89d4f96, including durable interrupted-message backoff, event-loop fairness, shutdown behavior, and preservation of caller lifecycle and fencing guarantees. |
Address review feedback by keeping regression fixture choices explicit. Preserve the adapter and worker mappings while removing nested conditionals. Validation: pnpm run format:check, pnpm run check, and pnpm test (660 passed, 32 conditional skips).
|
@greptileai Please review current head 96f2137, including the interrupted-message recovery backoff and event-loop fairness changes from 89d4f96. The nested database-selection finding is fixed, including the matching new fixtures. Local formatting, type/documentation checks, and all 660 SQLite tests pass. |
Handle fenced failures and rejections like lost completions, preserving recovery and activation.lost callbacks. Flatten shutdown checks in all worker loops as requested in review. Before the fix, two new recovery tests raised LostActivation and two callback shutdown cases exceeded their external time limits. All pass now. Type, formatting and full SQLite/PostgreSQL/MySQL suites pass.
|
@greptileai Please review current head ec3ddc3. Both nested-branch findings are fixed. This head also fixes the loss-event gap found in the Ruby review: a failure/rejection fence loss now reports activation.lost and recovers normally. Four new regressions first failed and now pass, including callback-driven shutdown at the third loss. Full SQLite/PostgreSQL/MySQL suites pass. |
1 participant
Independently reported failures exposed shared synchronous-caller ownership, missing process registrations after cleanup, and a worker that could retry an expired turn indefinitely without servicing timers or input. This change isolates overlapping callers, restores active process registrations, and yields between busy SQL worker and drain passes.
Interrupted-message recovery is immediate after claim attempt 1. After attempts 2, 3, 4, ... it persists the existing
retryDelayMillisecondscallback's result for arguments 1, 2, 3, ... in ready-message availability. Defaults grow from 1 to 2 to 4 seconds up to 60 seconds. Total claim attempts include earlier handler failures. Mailbox order stays intact and other actors remain eligible; lease loss alone never becomes a dead letter. Handler-failuremaxAttempts, setting names/defaults, and schema are unchanged. An application must still impose any hard retry/time limit.Lease loss is also reported when persisting a handler failure or rejection loses its fence. Those attempts remain recoverable, and callback-driven shutdown sees them. Four additional regressions first failed with an escaping
LostActivationor an external child-process timeout; all now pass.New tests failed on the previous head: the lost-turn child needed an external kill with zero timer ticks; continuous actor/effect/reminder/broadcast loops and drain helpers also suppressed timers; recovery immediately retried instead of persisting backoff. All now pass. Instrumentation abort at the third lost attempt passed before and after. Earlier activation-lease and caller-lifecycle regressions remain green on SQLite, PostgreSQL, and MySQL.
Validation:
pnpm run format:check,pnpm run check,pnpm test(664 pass, 32 conditional skips), PostgreSQL full suite (685 pass, 11 skips), MySQL full suite with--fileParallelism=false(674 pass, 22 skips), Node 24.4 focused regressions (36 pass), Cloudflare (60 pass), and browser (16 pass). Shared telemetry fixtures match Ruby. MySQL initially hit three 5-second timeouts while other suites shared the host; the serial rerun passed without changing test deadlines.The library-only reproduction now honors its 3-second abort timer: 3 lost attempts, 88 timer ticks, exit at 3.03 seconds; previously 25 lost attempts and zero ticks until an external kill at 10.5 seconds. Paired 1,000-short-turn medians: 211 ms before, 226 ms after (+7.3%); maximum timer gap fell from 215 ms to 32 ms. One synchronous handler still cannot be preempted.
Release coverage, package artifact, crash-recovery, browser, and dependency-audit checks pass. The earlier at-least-once smoke test also passes.
Prepares JavaScript 0.17.8; coordinate with the Ruby recovery-backoff change before tagging. No tag or publication is part of this PR.
Companion parity PR: cardmagic/solid-objects-ruby#90