Repository navigation
feat(automation): tell the foreman about Mail left unread after its notice - #166
chriscoveries wants to merge 3 commits into
Conversation
…otice A Mail notice is sent once, and its delivery proves nothing: a notice typed into a busy terminal is accepted and can still be lost, and then nobody hears about that Mail again. With the new group delivery setting `health_digest_interval_seconds` above zero, the unread tick also sends the foreman one `system.notify(kind="health_digest")` that lists, one line per Actor, Mail still unread `mail_overdue_after_seconds` (default 7200) after its first notice: how many items, how long the oldest has waited, and how many notices went out. A problem the last digest listed is not sent again unless `health_digest_repeat_seconds` is set; a new one goes out at most once per interval. The digest copies no message bodies. The default, 0, sends no digest.
|
I am deferring the broader health-digest feature. The overdue-Mail observation is useful, but it introduces escalation/repeat policy into delivery before we have established that a new foreman-facing message stream is the right user workflow. There is also a concrete integration gap: this diff modifies the old Please narrow this first to a passive, Group-scoped overdue-Mail summary using the current projection, with no polling-triggered runtime work. If a delivered escalation is still needed, define its recipient, repeat/clear rules and interaction with #165 separately, then validate the actual daemon path, including digest-only settings. Avoid a generic health framework as part of this Mail change. |
Main now finds due Mail notices with an incremental projection (inbox/notices.rs) and keeps the ordered reducer only as a test-time cross-check. The health digest moves into the projection: the notice state remembers the last digest (its time and problem keys) and the time of each Mail notice, and the projection lists Mail still unread mail_overdue_after_seconds after its first notice and sends the digest to the foreman under the same rules. The ordered reducer keeps the digest, so the cross-check covers it.
The unread pass returned before it looked at the digest when Mail and reply notices were both off, so a group that wants only the digest never got one. The early return now also requires the digest to be off. The notice paths are unchanged: a delay of 0 still turns its notice off.
A Mail notice is sent once, and its delivery proves nothing: a notice typed
into a busy terminal is
acceptedand can still be lost. After that nobodyhears about the Mail again, neither the recipient nor the foreman who
coordinates the group. If the notice is missed, that Mail and any that
follows it can wait unnoticed until the Actor happens to check its Inbox.
The new group delivery setting
health_digest_interval_secondsturns onone message to the foreman that lists what in the group needs a look, one
line per problem. This PR defines one problem: an Actor's Mail still unread
mail_overdue_after_seconds(default 7200) after its first Mail notice thatwas not withdrawn. The line names the Actor, how many items wait, how long
the oldest has waited and how many notices went out, for example
peer: 1 Mail item(s) unread, the oldest for 2m, after 1 Mail notice(s).The digest is a
system.notify(kind="health_digest")addressed to theforeman;
context.problem_keysnames each problem's episode (Actor andoldest waiting Mail). It goes out when a problem appears that the last
digest did not list, at most once per interval, and, with
health_digest_repeat_secondsabove zero, again after that long whileproblems remain. It copies no message bodies and advances no Mail cursor.
The default, 0, sends no digest, so nothing changes unless a group sets it.
In the case above, with a digest interval set, the foreman would have heard
two hours (the default overdue time) after the notice, instead of nobody
hearing until the Actor read its Inbox.
One message with keyed lines, rather than an escalation per notice, keeps
the foreman's terminal quiet when several Actors fall behind at once, and
lets other problem kinds add a line later instead of a new message type.
The companion change that optionally repeats the notice to the recipient
(
feat/mail-notice-repeat-v3on our fork) is independent of this one; bothtouch
unread_notices, so whichever lands second needs a small rebase.How we built it. Base
3d3d10cb. The unread tick already walks eachActor's waiting Mail and the Mail notices that named it. For an Actor whose
oldest waiting Mail was first named by a notice
mail_overdue_after_secondsago, it now records a problem line; after the Actor loop it sends the
digest to the group's unique enabled foreman, if the tick may deliver to
that foreman. The last digest in the ledger, with its problem keys, decides
whether a new one is due, so there is no new state file (
automation.rs).group_settings_updateaccepts the three keys underdelivery(
ops/settings.rs), and the IPC standard documents them and the digest.4 files, +287/−2, 123 of the added lines the test.
How we tested it.
real ledger: Mail to a peer; its notice; the
claimedandacceptedfactsdelivery writes when it types a notice into a terminal. Right after the
notice no digest; once the Mail is overdue, one digest to the foreman
naming the peer, the age of the Mail and the notice count, without the
Mail's text; nothing more before the repeat interval; one more after it;
after the peer reads its Inbox, none. On upstream main it fails, because
no digest ever comes: red reference run with only this test on main
3d3d10cb, https://github.com/chriscoveries/cccc/actions/runs/39669878635core and daemon automation, Mail notice, settings and digest tests (87 tests).
-D warnings, nextest, Windows smoke):https://github.com/chriscoveries/cccc/actions/runs/39669874854
Tested on: claw, Debian 13, Skylake 8vCPU AVX2/AVX-512, 15G RAM.