Skip to content
Merged
9 changes: 5 additions & 4 deletions docs/src/content/docs/guide/inspect.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import { Aside, Steps } from '@astrojs/starlight/components';
*SHRINE Inspect is a local, read-only command-line tool. It measures your AI coding setup and session history against SHRINE and reports what to change first. It changes nothing itself.*

<Aside type="caution" title="Pre-release">
This is a trial release. Get the newest `inspect-v*` release from the [releases page](https://github.com/stablekernel/SHRINE/releases). The steps below use `inspect-v0.1.2`. Trial testers also follow [TRIAL.md](https://github.com/stablekernel/SHRINE/blob/main/inspect/TRIAL.md).
This is a trial release. Get the newest `inspect-v*` release from the [releases page](https://github.com/stablekernel/SHRINE/releases). The steps below use `inspect-v0.1.3`. Trial testers also follow [TRIAL.md](https://github.com/stablekernel/SHRINE/blob/main/inspect/TRIAL.md).
</Aside>

- The first release reads one harness: Claude Code; other harnesses come later
Expand All @@ -35,6 +35,7 @@ This is a trial release. Get the newest `inspect-v*` release from the [releases
At T1 only, 12 scrubbed episode excerpts go to your model provider through your own coding agent login, one call each.

- Each excerpt holds your messages, the start of each reply, the end of the last reply, and how the episode ended
- Code adds end signals: an open question for you in the last reply, a later revert or fixup, a later episode on the same files within a day
- Code searches all your loaded instruction files and sends only the lines that match your corrections
- When no line matches, that search counts as evidence; "could not tell" is kept for files that may have changed since or could not be read in full

Expand All @@ -53,9 +54,9 @@ It checks that it changed nothing it read. If anything changed, the report shows
2. Download the three release files into a new folder

```sh
mkdir -p ~/shrine-inspect-0.1.2 && cd ~/shrine-inspect-0.1.2
base=https://github.com/stablekernel/SHRINE/releases/download/inspect-v0.1.2
curl -fLO "$base/shrine-inspect-0.1.2.tgz"
mkdir -p ~/shrine-inspect-0.1.3 && cd ~/shrine-inspect-0.1.3
base=https://github.com/stablekernel/SHRINE/releases/download/inspect-v0.1.3
curl -fLO "$base/shrine-inspect-0.1.3.tgz"
curl -fLO "$base/install.sh"
curl -fLO "$base/SHA256SUMS"
```
Expand Down
17 changes: 10 additions & 7 deletions inspect/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,11 +79,13 @@ Status: pre-release trial (background runs, a thin skill, a checksummed release)
- The consent, with the approved estimate, is recorded in `facts.json`; the judge command is refused until it is there; every run asks again
- Sample: 12 episodes with a correction candidate, seeded (seed in the report footer), recency thirds, at most 4 per project unless one project holds them all, 3x weight for sessions with 2 or more corrections
- Fewer than 12 eligible: all are judged, labelled thin evidence; under 10 sessions: T1 is not offered
- Evidence pack per episode: your messages (up to 1,200 characters, 12 messages), the opening 400 characters of each reply, the closing 300 characters of the last reply, tool names without arguments, how the episode ended (commit, pull request, next task or idle, session end), and the instruction lines that match the flagged messages
- Instruction lines: code searches every instruction file loaded for that session for words from each flagged message and sends only matching lines, under their heading (at most 1,600 characters)
- Every string is scrubbed for secrets, then masked (names, paths, hosts) before a judge or a file sees it; about 1,500 tokens per pack at most
- Over the cap: replies cut first, then middle messages that were not flagged (the first and last always stay), then long messages, then instruction lines, then flagged middle messages
- Gate B estimate: pack size (bytes / 4) plus a per-call overhead and output measured on Claude Code 2.1.318, then 10% retry reserve and 30% margin on the top
- Evidence pack per episode: your messages (up to 1,200 characters, 12 messages), the opening 400 characters of each reply, the closing 400 characters of the last reply, tool names without arguments, how the episode ended (commit, pull request, next task or idle, session end), end signals computed by code, and the instruction lines that match the flagged messages
- Instruction lines: code searches every instruction file loaded for that session for words from each flagged message and sends only matching lines, under their heading (at most 2,400 characters)
- End signals: the last reply asks you something (a question, lettered choices, a questions list) and no tool call follows, a later revert or fixup commit, a later episode on the same files within a day
- The judge uses these and the last reply to pick resolved, abandoned (left open), or reworked; "could not tell" is for an episode none of these fits
- Every string is scrubbed for secrets, then masked (names, paths, hosts) before a judge or a file sees it; about 4,000 tokens per pack at most (the cap, at 2.5 bytes per token)
- Over the cap: middle replies cut first, then middle messages that were not flagged (the first and last always stay, the last reply whole with its closing), then instruction lines, then flagged middle messages, then the first and last messages
- Gate B estimate: pack size (bytes / 2.5) plus the prompt, a per-call overhead and output measured on Claude Code 2.1.318, then 10% retry reserve and 30% margin on the top
- Minutes: 10 seconds per call, judging only (not the scan or time at gates); real time varies by machine, login, and network
- The judging time limit (the estimate plus 50%) is the only fixed time figure
- Session text is data for the judge, never instructions: it sits inside a fence named by a random nonce, and the judge prompt says so
Expand Down Expand Up @@ -146,8 +148,9 @@ T1 trial (recorded, not fixed):
- At 12 episodes the top of the Gate B range is tens of thousands of tokens at any pack size: the fixed cost per call (prompt, overhead, output) times 12, plus reserve and margin (`test/judge/workorder.test.mjs`)
- A correction you later reversed still gives a before and after from your own words
- The harness version shown at Gate B comes from your newest session log, not from the installed command
- A session whose loaded-instructions snapshot is logged only after an episode starts is read as "may have changed", so its corrections stay "could not tell"
- Instruction lines are matched by shared words (common words left out), not meaning: a rule worded with other words is not found, and its correction reads as yours
- The loaded-instructions snapshot logged right after an episode's first message counts as loaded for it; a first snapshot logged only after a reply is read as "may have changed", so its corrections stay "could not tell"
- End signals are word and pattern checks: a reply that asks something without a question mark, lettered choices, or a questions heading is not seen as asking
- Instruction lines are matched by shared words (common words and simple plural or tense endings left out), not meaning: a rule worded with other words is not found, and its correction reads as yours. Sending whole files would fix this but sends far more of your config, so it is not done

Leak check, `check-share` and the dry run (`test/release/leak-known-limits.test.mjs`):

Expand Down
18 changes: 9 additions & 9 deletions inspect/TRIAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ A local health check of your Claude Code setup and session history, measured aga
node --version
```

2. Download the 3 release files into a new folder. With the GitHub CLI, use `gh release download inspect-v0.1.2 -R stablekernel/SHRINE` in place of the `curl` lines.
2. Download the 3 release files into a new folder. With the GitHub CLI, use `gh release download inspect-v0.1.3 -R stablekernel/SHRINE` in place of the `curl` lines.

```sh
mkdir -p ~/shrine-inspect-0.1.2 && cd ~/shrine-inspect-0.1.2
base=https://github.com/stablekernel/SHRINE/releases/download/inspect-v0.1.2
curl -fLO "$base/shrine-inspect-0.1.2.tgz"
mkdir -p ~/shrine-inspect-0.1.3 && cd ~/shrine-inspect-0.1.3
base=https://github.com/stablekernel/SHRINE/releases/download/inspect-v0.1.3
curl -fLO "$base/shrine-inspect-0.1.3.tgz"
curl -fLO "$base/install.sh"
curl -fLO "$base/SHA256SUMS"
```
Expand All @@ -33,7 +33,7 @@ A local health check of your Claude Code setup and session history, measured aga

```
install.sh: OK
shrine-inspect-0.1.2.tgz: OK
shrine-inspect-0.1.3.tgz: OK
```

4. Install. Run it from the folder that holds the files.
Expand All @@ -43,9 +43,9 @@ A local health check of your Claude Code setup and session history, measured aga
```

- Usage: `sh install.sh [--skill] [path/to/tgz]`.
- With no path, it uses `shrine-inspect-0.1.2.tgz` next to `install.sh`, not in your current folder.
- With no path, it uses `shrine-inspect-0.1.3.tgz` next to `install.sh`, not in your current folder.
- It checks Node, then checks the tarball against the SHA-256 written in `install.sh`, then extracts it. A mismatch installs nothing.
- The CLI goes to `~/.local/share/shrine-inspect/0.1.2/`, linked as `~/.local/bin/shrine-inspect`.
- The CLI goes to `~/.local/share/shrine-inspect/0.1.3/`, linked as `~/.local/bin/shrine-inspect`.
- `--skill` copies the Claude Code skill to `~/.claude/skills/shrine-inspect/`. Leave it out to use the CLI only.
- To install somewhere else, set the install folder and the link folder first:

Expand All @@ -68,7 +68,7 @@ A local health check of your Claude Code setup and session history, measured aga
```

```
shrine-inspect 0.1.2
shrine-inspect 0.1.3
```

- All commands: `shrine-inspect --help`.
Expand Down Expand Up @@ -187,7 +187,7 @@ shrine-inspect check-share

- T0: nothing leaves your machine. No network, no model calls.
- T1: 12 short episode excerpts go to your own Claude Code login, one call each, billed to your account like normal use. Secrets are removed and names, paths, and hosts are masked first.
- Each excerpt holds your messages, the start of each reply, the end of the last reply, how the episode ended, and only the instruction lines that match your corrections. Code searches all your loaded instruction files for those lines; the rest stay on your machine.
- Each excerpt holds your messages, the start of each reply, the end of the last reply, how the episode ended (with end signals code computed, such as an open question for you), and only the instruction lines that match your corrections. Code searches all your loaded instruction files for those lines; the rest stay on your machine.
- Nothing goes to SHRINE, Stable Kernel, or any other service. No telemetry.

## Read the report
Expand Down
4 changes: 2 additions & 2 deletions inspect/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion inspect/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "shrine-inspect",
"version": "0.1.2",
"version": "0.1.3",
"description": "Health check of an AI coding setup against the SHRINE docs. Local, read-only, no network.",
"license": "Apache-2.0",
"author": "Stable Kernel",
Expand Down
2 changes: 1 addition & 1 deletion inspect/skill/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "shrine-inspect",
"version": "0.1.2",
"version": "0.1.3",
"description": "Thin skill that starts and watches the shrine-inspect CLI. Holds no logic; install the CLI from the same release.",
"license": "Apache-2.0",
"author": { "name": "Stable Kernel" },
Expand Down
8 changes: 4 additions & 4 deletions inspect/skill/skills/shrine-inspect/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ allowed-tools: Bash(node --version), Bash(shrine-inspect version:*), Bash(shrine

# SHRINE Inspect

Skill version: 0.1.2
Skill version: 0.1.3

This skill only starts the `shrine-inspect` CLI and watches it. The CLI does every read, every gate, and every judge call.

Expand All @@ -31,23 +31,23 @@ This skill only starts the `shrine-inspect` CLI and watches it. The CLI does eve
2. Check the CLI. Run:

```bash
shrine-inspect version --skill-version 0.1.2
shrine-inspect version --skill-version 0.1.3
```

If the command is not found, or it stops with `E_VERSION`, stop. Tell the user to install the CLI and this skill from the same release (see TRIAL.md in the release).

3. Start a run in the background. Use `--tier T1` only when the user asked for judging ("Pulse"); else use `--tier T0`. Session id: `${CLAUDE_SESSION_ID}`. If that id is empty or still shows as a placeholder, leave out `--session-id` and its value. Never leave `--session-id` with nothing after it.

```bash
shrine-inspect start --tier T0 --session-id ${CLAUDE_SESSION_ID} --skill-version 0.1.2 --json
shrine-inspect start --tier T0 --session-id ${CLAUDE_SESSION_ID} --skill-version 0.1.3 --json
```

Keep the `run_id` from the output.

4. Watch the run. Run this about every 15 seconds while the state is `running`:

```bash
shrine-inspect status <run_id> --skill-version 0.1.2 --json
shrine-inspect status <run_id> --skill-version 0.1.3 --json
```

- `running`: give the user one short line with the phase. Check again.
Expand Down
11 changes: 9 additions & 2 deletions inspect/src/adapters/claude-code/transcript.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ import { readSession } from './events.mjs';

// Events of one episode: from its root human turn (matched by uuid, else by start
// time) through its end time. Subagent events are left out. The live config is
// the newest instruction snapshot at or before the episode start, else the first
// one in the session (loaded at start, before the first prompt was logged).
// the snapshot logged right after the root turn (loaded for that prompt), else
// the newest one at or before the episode start, else the first one in the
// session, marked late.
export function sliceEpisode(timeline, ep) {
const main = timeline.filter((e) => !e.sub);
let from = main.findIndex((e) => e.kind === 'human' && e.uuid && e.uuid === ep.root_turn_id);
Expand All @@ -21,6 +22,12 @@ export function sliceEpisode(timeline, ep) {
events.push(e);
}
const snaps = main.filter((e) => e.kind === 'instructions');
// The harness logs the files it loads for a prompt just after that prompt. A
// snapshot logged after the root turn, with nothing else between them, is the
// config loaded for the root prompt, so it counts as loaded at the start.
let loaded = null;
for (let i = from + 1; i < main.length && main[i].kind === 'instructions'; i++) loaded = main[i];
if (loaded) return { events, snapshot: loaded.files, snapshot_late: false };
const before = snaps.filter((e) => e.t && e.t <= ep.start).at(-1);
if (before) return { events, snapshot: before.files, snapshot_late: false };
// A snapshot logged after the episode started may hold files loaded later on, so
Expand Down
Loading
Loading