Repository navigation
feat: queue a rearm when extending a timer whose handler is running - #256
Merged
Merged
Conversation
Benchmark gate✅ gate passed
Gated (fails the build): |
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #254. With this change,
publish(..., timer_conflict="extend")also works on a timer whose handler is already running. In #254 that case returnedNoneand the heartbeat was lost. That broke the operator-timeout case, where a ping that lands while the expiry runs can't arm the next timer, and it meant a handler couldn't rearm its own timer either.Contract
When the extend hits a leased row, it queues a rearm in three new nullable columns:
rearm_at,rearm_payloadandrearm_headers. It returns the row id and leaves the running instance alone: the lease, the claimed deadline and the payload are unchanged. A rearm only moves later, using the same rule as a pending extend, and an extend whose deadline isn't later than the running deadline queues nothing.How each terminal write handles a queued rearm:
deliveries_countare reset.deliveries_countis kept, somax_deliveriesstill stops a handler that fails on every run.cancel_timernow locks the row first. It deletes an unleased row, or drops the rearm queued on a leased one, and still never deletes a leased row. Locking first closes a race where a run could finish and apply its rearm between the cancel's check and its write.Cost
The common path is unchanged. A completion is still one
DELETE ... WHERE rearm_at IS NULL, and only when it matches nothing does a follow-upUPDATEapply the rearm. That fallback runs only for rows with a queued rearm or a lost lease.just bench-checkpasses against the existing baseline. The same fallback pattern covers the batched delete and the DLQ path.cancel_timernow takes two statements instead of one.An extend and a completion are always ordered. The completion's
DELETEwaits on the row lock the extend holds, then re-checksrearm_atand falls through to applying it. An integration test exercises that order.Schema migration
This change needs a migration: three nullable columns plus
outbox_rearm_ck(rearm_at IS NULL OR acquired_token IS NOT NULL).docs/operations/alembic.mdgains an "Upgrading to 0.19" section. It adds the columns without rewriting the table and adds the CHECK asNOT VALIDbefore validating it. I ran that Alembic snippet against a table in the 0.18 shape, andvalidate_schema()passed afterwards. Run the migration before deploying, because every completion readsrearm_at. Older versions ignore the columns, andvalidate_schema()doesn't flag extra columns. A missing rearm column error now links to the upgrade section. The release number 0.19 is my assumption.Tests
deliveries_count;Falsewhen there is none.just test: 817 passed with the coverage gate on.just bench-check: OK.What private-users still owns
A rearm doesn't undo what the running handler did. The expiry handler still has to re-check last activity in the same transaction as the OFFLINE transition. With this PR, a ping that lands during the expiry is no longer lost: the timer fires again for the newer deadline.