Skip to content

fix(inspect): T1 outcome signals and checkable first-episode config (v0.1.3) - #41

Merged
joshua-temple merged 8 commits into
mainfrom
fix/inspect-t1-outcomes
Nov 6, 2026
Merged

joshua-temple merged 8 commits into
mainfrom
fix/inspect-t1-outcomes

Conversation

@joshua-temple

@joshua-temple joshua-temple commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

TLDR

  • T1 packs carry code-computed end signals; the judge prompt maps them to outcomes
  • The snapshot logged right after the root prompt counts as loaded, so those corrections are checkable
  • Larger pack (last reply kept whole); Gate B estimate rebased on measured bytes per token

Why

  • Real runs left most outcomes "could not tell"
  • Corrections in a session's first episode were forced unknown by snapshot timing

Evidence

Check Test
Snapshot after root prompt is loaded a snapshot logged right after the root turn, before any reply, is the config loaded for that prompt
End signals from code end signals: computed by code from the last reply and later git and episode links
Asks-person check asksPerson: a question, lettered choices, or a questions list at the end; a plain report is not
Last reply kept whole trimming shortens middle replies first; the last reply keeps its full opening and closing
Config lines per turn config: four flagged turns with six best lines each all reach the judge
Honest Gate B estimate: the per-call input estimate brackets measured calls, and stays honest at the larger cap
Snapshot fallback for facts episodes end signals: computed by code from the last reply and later git and episode links
Trim order over the cap, instruction lines go before the first message or a flagged message is cut
Real T1, final code, 3 runs could not tell 5, 1, 6 of 12 (v0.1.2: 8); 12 of 12 judged, 0 invalid, exit 0, check-share OK
Version chore(inspect): version 0.1.3; all tests pass

How to Verify

  1. cd inspect && npm test && npm run lint && npm run test:visual
  2. node scripts/dry-run.mjs --require-node20
  3. Real T1 run: Gate A history, Gate B yes; read the Outcomes line in the appendix
  4. shrine-inspect check-share
cd inspect && npm test

…s loaded

- The harness logs loaded instruction files just after the prompt
- Only a first snapshot logged after a reply stays may-have-changed
- Code-computed end signals: open question, subagent started, revert, fixup, later episode
- Outcome rules in the judge prompt use them; unknown only when none fits
- Trimming keeps the last reply whole with its closing; middle replies cut first
- Pack cap and Gate B estimate at 2.5 bytes per token, checked against measured calls
… excerpt cap

- reverted, fixup_later, continued_later come from the linker flags, not the nearest links only
- fixup_later and continued_later are weak links; only a revert makes reworked
- Dropped work with no open question counts as abandoned
- A questions list must be a heading or label line
- Config excerpt up to 2,400 characters so four flagged turns keep their best lines
…ssage

- Flagged middle messages and the first and last turns are cut only after instruction lines
- continued_later counts a later episode on the same files within a day, as a hint
- A redirect or taste choice no earlier text could cover is labelled the person's
- Matches how the episode cutter reads a question the agent moved past
- The abandoned rule names started_agent as support for work still running
…ed_agent

- Facts episodes carry no linker flags, so reverted and fixup_later read the stratum outcome first
- started_agent could not tell a finished subagent from a running one, so it is gone
- continued_later keeps the next episode of the same session: that is where an open question is answered
@joshua-temple joshua-temple changed the title fix(inspect): T1 outcome signals and checkable first-episode config fix(inspect): T1 outcome signals and checkable first-episode config (v0.1.3) Nov 6, 2026
@joshua-temple
joshua-temple merged commit 8974d26 into main Nov 6, 2026
8 checks passed
@joshua-temple
joshua-temple deleted the fix/inspect-t1-outcomes branch November 6, 2026 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant