diff --git a/README.md b/README.md index c8a60d6..f332094 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,12 @@ agents through SQL and a CLI. Nothing leaves the machine. - **A status line in Claude Code:** `qx $3.42 · ctx 182k · cache 96%`, from the harness's own numbers. - **Nudges** (toasts, never added to the model's context): the prompt cache expired after idle time and the next call re-wrote it; the context passed 200k. -- **`/clear-tool-outputs [keep N]`:** compacts by replacing old tool outputs with short stubs instead of a summary. +- **`/clear-tools [outputs] [keep N]`:** compacts by replacing old tool calls (their inputs) and old tool outputs + with short stubs instead of a summary, in one pass, so the prompt cache is written once. `outputs` clears only + the outputs; `keep N` leaves the last N whole. Also as `/compact tools [outputs] [keep N]`. The next call + writes the whole new context to the cache again; the toast gives that cost and after how many calls the + smaller context pays it back, so a clear pays most right after the cache expired. A run with nothing to clear + changes nothing. Small inputs (300 chars or less) and ToolSearch results stay whole. - **`/quantext view`:** opens a live viewer on `http://127.0.0.1:4747` with every session on the machine: - the context of each call, exactly as the model received it (system prompt, tool schemas, messages), with the tokens and cost of each block, and what was read from the cache or written to it; diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index cdaa684..dda216f 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -30,7 +30,7 @@ agents (SQL, CLI, later MCP). Nothing leaves the machine. | Part | Runs in | Can do | Cannot do | |---|---|---|---| -| **Mod** | Claude Code's mod sandbox (no DOM, no Node; I/O only through `$`) | Read each call's `usage`, status line, toasts, commands, `/clear-tool-outputs`, `/clear-tool-calls`, `/clear-tools`; `$.http.fetch` to the engine; start the engine with `$.process.run` | Open a server, use npm packages or Node built-ins, keep a child process alive (`$.process.spawn` dies with the session) | +| **Mod** | Claude Code's mod sandbox (no DOM, no Node; I/O only through `$`) | Read each call's `usage`, status line, toasts, commands, `/clear-tools [outputs] [keep N]`; `$.http.fetch` to the engine; start the engine with `$.process.run` | Open a server, use npm packages or Node built-ins, keep a child process alive (`$.process.spawn` dies with the session) | | **Engine** | A separate Node ≥ 22.18 or Bun process, started detached | Watch and parse transcripts, write the store, serve HTTP + SSE and the viewer's files | — | | **Viewer** | The browser | Read the engine's API | — | @@ -222,7 +222,7 @@ Each step ends with tests, and later steps never reach below the layer they depe | 6 | **Query layer + CLI**: `qx ingest`, `qx sessions`, `qx call`, `qx sql` with views | 1 | An agent answers "top tool outputs by lifetime cost" with `qx sql` | | 7 | **API + engine process**: HTTP JSON + SSE, Host check, single instance, idle exit | 1 | SSE fires on a new call; a second start reuses the running engine | | 8 | **Live context viewer**: reuse the eduagent visualiser components with a read-only Claude Code adapter; the teaching features above | 1 | A running session updates in the browser call by call; meets the UX bar | -| 9 | **Mod**: status line, nudges, `/clear-tool-outputs` from in-process `usage`; `/quantext view` starts the engine and opens the viewer | 1 | Works on a fresh Claude Code install; harness features work with no Node or Bun | +| 9 | **Mod**: status line, nudges, `/clear-tools` from in-process `usage`; `/quantext view` starts the engine and opens the viewer | 1 | Works on a fresh Claude Code install; harness features work with no Node or Bun | | 10 | **Agent surface**: schema docs, a skill, MCP later | **2** | The reviewer questions are answered and checked in the viewer | Code from the earlier prototype (the Python parser and incremental store, pricing, the mod widget) is ported inside diff --git a/docs/known-issues.md b/docs/known-issues.md index 6e9dbca..c9582f4 100644 --- a/docs/known-issues.md +++ b/docs/known-issues.md @@ -31,8 +31,17 @@ workaround), **low** (cosmetic, rare, or only a cost). | medium | mod | No install straight from GitHub: the marketplace entry points at `./dist/plugin`, which is built, not committed (ADR 27). | A release job commits `dist/plugin` to a `plugin` branch, and the marketplace entry names that ref. | | medium | mod | CI does not run the mod's tests: `pnpm test:mod` needs the `claude` CLI and skips loudly without it. | Install a pinned Claude Code in CI and set `QUANTEXT_REQUIRE_CLAUDE=1`. | | medium | mod | A fake engine that listens on a stale `engine.json` port can answer `/health` with the right version and pid, and the mod opens it. Accepted inside the single-user trust boundary: only a local process of the same user can do it. | A secret in `engine.json` (mode 600) that `/health` echoes, checked by the mod. | -| medium | mod | `clearToolOutputs` and `clearToolCalls` drop the `handle` of a message they change (`clear-tool-outputs.ts`, `clear-tool-calls.ts`), so the engine rebuilds it from role, text and tool blocks: images or extra text blocks in that message can be lost or joined. | Leave a message whole when it holds more than tool blocks and one text block. | -| medium | mod | After `claude --resume` of a session cleared by `/clear-tool-outputs`, `/clear-tool-calls` or `/clear-tools`, Claude Code 2.1.322 sends the original tool inputs and outputs again (seen in the resumed request: the full `Write` content, a `[Tool use interrupted]` row, a thinking block twice). In the same session the stubs hold. The likely cause is that resume joins the assistant rows of one API response by message id, and the kept rows still carry the original id. | Report it to Claude Code with a repro; until then, run the clear command again after a resume (not yet tested). | +| low | mod | `/clear-tools` drops the `handle` of a message it changes, so Claude Code rebuilds it. On a live clear (session 244) the rebuilt messages kept all their thinking, text and image blocks (same counts before and after). Not tested: an image inside a tool result that is kept, in a message that changed. | Add that case to a live check (or a harness test) and see that the image stays. | +| medium | mod | `/clear-tools` clears even when it saves little: a clear on a large context re-writes the whole cache for a small per-call saving (the toast now gives the cost and payback, review H1). | Skip with "saves little" when the saving is under ~10% of the context unless the user adds `force`; have the expiry nudge say "the cache expired: /clear-tools now costs no extra write". | +| low | mod | The `/clear-tools` reply ("Clearing …", ~21 tokens) and Claude Code's "Compacted …" stdout (~22 tokens) enter the model's context as ``. | A shorter (or empty) reply; say in the docs that this text enters. | +| low | mod | The `/clear-tools` toast counts an image tool result as 0 chars (about 1.5k tokens each), and the tokens it gives are chars / 4. | Add a fixed cost per image result whose `text` is empty. | +| low | mod | `keep N` counts calls and outputs on their own, and small inputs count toward N: an interrupted call with no result shifts the count, so a call can stay while its output gets a stub. | Count both from one list, the ids of the last N tool uses; say "each parallel call counts" in the help text. | +| low | mod | Tests miss: parallel calls with the `keep` boundary inside one message, a check that every `tool_use` id keeps its `tool_result`, an `updateLive` failure after a clear; fixtures need `as never` casts. | Add those cases, and a typed fixture helper in `test/kit.ts`. | +| low | mod | Bare `/compact tools` is a clear with `keep 0`, so a user who means "focus the summary on tools" gets a clear and no summary. | Say so in the README, or ask for `/compact tools keep N` only. | +| medium | adapter | Late in a long context, per-block token estimates for tool-call blocks are 3 to 4 times too high (session 244, call 880: a 129-char Bash input counted as 499 tokens), so the viewer's tool-calls figure is too high. | Cap a block's estimate at about 1 token per 1.5 chars; find where the call's residual goes to the last blocks. | +| low | pricing | On session 244 the engine priced a cache read at $0.22/M and a 1-hour write at $8.80/M (40x); list prices give 20x, so a payback the viewer derives may be off. | Check those prices against the provider's list for the model. | +| medium | mod | After `claude --resume` of a session cleared by `/clear-tools` (either form), Claude Code 2.1.322 sends the original tool inputs and outputs again (seen in the resumed request: the full `Write` content, a `[Tool use interrupted]` row, a thinking block twice). In the same session the stubs hold. The likely cause is that resume joins the assistant rows of one API response by message id, and the kept rows still carry the original id. | Report it to Claude Code with a repro; until then, run `/clear-tools` again after a resume (not yet tested). | +| low | mod | `/compact tools` with words the mod does not read (e.g. `/compact tools keep -1`) is not a clear: it goes to Claude Code as summary instructions, and costs a summary. `/clear-tools` checks its words first, so only the typed `/compact` form has this. | Answer `/compact tools ` with a skip that names the usage; the cost is that instructions starting with "tools" can no longer go to the summary. | | low | mod | Permission prompts for `/quantext view` are untested; an interactive session may ask before each `$.process.run` (`node --version`, `uname`, `sh`, `open`). | Try it on a fresh install; if it asks several times, find the runtime with one `sh -c 'command -v node bun'` and cache it in `$.store`. | | low | mod | `xdg-open` can block on some Linux desktops; after 5 s the mod says "open it in a browser" although it opened. | Run it detached, as the engine is started. | | low | mod | Two `/quantext view` at once both find no engine and both run `sh`. Safe: the engine's writer lock lets one serve and the other exit. | An in-memory "starting" flag, so the second waits for the first. | diff --git a/packages/mod/src/clear-tool-calls.ts b/packages/mod/src/clear-tool-calls.ts index 7b291fa..9e43e88 100644 --- a/packages/mod/src/clear-tool-calls.ts +++ b/packages/mod/src/clear-tool-calls.ts @@ -1,28 +1,25 @@ /** - * `/clear-tool-calls [keep N]`: replace the input of old tool calls with a short stub, so the model's own Write - * contents, Bash heredocs and Agent briefs leave the context. Each call keeps its name and id, so it still pairs - * with its result. Deterministic: no model call. It runs as `/compact tool-calls keep N`. + * The calls half of `/clear-tools`: replace the input of old tool calls with a short stub, so the model's own + * Write contents, Bash heredocs and Agent briefs leave the context. Each call keeps its name and id, so it still + * pairs with its result. Deterministic: no model call (`clear-tools.ts`). */ import type { SessionMessage, ToolUseSummary } from "claude-code"; -import { tokens } from "./format.ts"; +import { approx } from "./format.ts"; /** Starts every stub, so a second run sees which inputs are cleared already. */ export const CLEARED_INPUT = "[Old tool input cleared"; -/** Inputs up to this size (~200 tokens) stay: a stub would save little. */ -const SMALL_INPUT_CHARS = 800; +/** Inputs up to this size (~75 tokens) stay: a stub would save little. */ +const SMALL_INPUT_CHARS = 300; /** A field up to this size stays in the stub input (a path, a name, a short description). */ -const SMALL_FIELD_CHARS = 200; +const SMALL_FIELD_CHARS = 80; +/** A stub must save at least this many chars, or the input stays (one of many short fields grows). */ +const MIN_SAVED_CHARS = 100; const HEAD_CHARS = 60; const sizeOf = (v: unknown): number => JSON.stringify(v)?.length ?? 0; const str = (v: unknown): string => (typeof v === "string" ? v : ""); const lineCount = (s: string): number => (s === "" ? 0 : s.split("\n").length); -/** Tokens at ~4 characters each: `850`, `4.2k`, `12k`. */ -const approx = (chars: number): string => { - const n = Math.round(chars / 4); - return n < 1000 ? `${n}` : n < 10_000 ? `${(n / 1000).toFixed(1)}k` : tokens(n); -}; const head = (s: string): string => { const first = s.split("\n")[0] ?? ""; return first.slice(0, HEAD_CHARS); @@ -72,7 +69,7 @@ const isStub = (u: ToolUseSummary): boolean => Object.values(u.input).some((v) => typeof v === "string" && v.startsWith(CLEARED_INPUT)); /** - * Stubs the input of every tool call but the last `keep`; small inputs stay. A changed message loses its engine + * Stubs the input of every tool call but the last `keep`; small inputs, and those a stub would not shrink, stay. A changed message loses its engine * handle and is rebuilt from role, text and tool blocks. Calls stubbed already do not count toward `keep`. */ export function clearToolCalls(messages: readonly SessionMessage[], keep = 0) { @@ -95,6 +92,7 @@ export function clearToolCalls(messages: readonly SessionMessage[], keep = 0) { const size = sizeOf(u.input); if (size <= SMALL_INPUT_CHARS) return u; const input = stubInput(u.tool, u.input); + if (size - sizeOf(input) < MIN_SAVED_CHARS) return u; cleared++; chars += size - sizeOf(input); return { ...u, input }; diff --git a/packages/mod/src/clear-tool-outputs.ts b/packages/mod/src/clear-tool-outputs.ts index 4d52f81..25c93f4 100644 --- a/packages/mod/src/clear-tool-outputs.ts +++ b/packages/mod/src/clear-tool-outputs.ts @@ -1,17 +1,24 @@ /** - * `/clear-tool-outputs [keep N]`: replace old tool outputs with the stub Claude Code's own microcompact writes, - * and keep everything else, in place of a summary. It runs as `/compact tool-outputs keep N` (`clear-tools.ts`). + * The outputs half of `/clear-tools`: replace old tool outputs with the stub Claude Code's own microcompact + * writes, and keep everything else, in place of a summary (`clear-tools.ts`). */ import type { SessionMessage } from "claude-code"; /** Claude Code's own stub for a cleared tool output. */ export const CLEARED = "[Old tool result content cleared]"; +/** Tools whose results load deferred tool schemas (`tool_reference`): a stub would unload them. */ +const KEEP_WHOLE = new Set(["ToolSearch"]); + /** * Stubs every tool result but the last `keep`. A message left whole keeps its engine handle; one changed loses - * it, so the engine rebuilds it from its role, text and tool blocks. Stubs already there do not count. + * it, so the engine rebuilds it from its role, text and tool blocks. Stubs already there, and ToolSearch + * results (kept whole), do not count. */ export function clearToolOutputs(messages: readonly SessionMessage[], keep = 0) { + const keepWhole = new Set( + messages.flatMap((m) => m.toolUses.filter((u) => KEEP_WHOLE.has(u.tool)).map((u) => u.tool_use_id)), + ); let left = keep; let cleared = 0; let chars = 0; @@ -23,7 +30,7 @@ export function clearToolOutputs(messages: readonly SessionMessage[], keep = 0) const results = [...before] .reverse() .map((r) => { - if (r.text === CLEARED) return r; + if (r.text === CLEARED || keepWhole.has(r.tool_use_id)) return r; if (left > 0) { left--; return r; diff --git a/packages/mod/src/clear-tools.ts b/packages/mod/src/clear-tools.ts index e1649c0..352f975 100644 --- a/packages/mod/src/clear-tools.ts +++ b/packages/mod/src/clear-tools.ts @@ -1,47 +1,86 @@ /** - * The three clear commands: `/clear-tool-outputs`, `/clear-tool-calls` and `/clear-tools` (both in one - * compaction, so the prompt cache is written once, not twice). Each runs as `/compact keep N`, which the - * mod's `session.compact` hook answers with no model call. + * `/clear-tools [outputs] [keep N]`: stub old tool calls (inputs) and old tool outputs in one compaction, so the + * prompt cache is written once; `outputs` stubs only the outputs. It runs as `/compact tools [outputs] keep N`, + * which the mod's `session.compact` hook answers with no model call. */ import type { SessionMessage } from "claude-code"; +import type { Live } from "../types/index.js"; import { clearToolCalls } from "./clear-tool-calls.ts"; import { clearToolOutputs } from "./clear-tool-outputs.ts"; -import { tokens } from "./format.ts"; +import { approx, grouped, tokens, usd } from "./format.ts"; +import { CACHE_READ, CACHE_WRITE_1H, CACHE_WRITE_5M, cacheWriteUsd } from "./prices.ts"; -export type ClearKind = "tool-outputs" | "tool-calls" | "tools"; +export const USAGE = "Usage: /clear-tools [outputs] [keep N]"; -/** Each command: its name, the `/compact` word it queues, and the words for what it clears. */ -export const CLEAR_COMMANDS = [ - { name: "clear-tool-outputs", kind: "tool-outputs", what: "tool outputs" }, - { name: "clear-tool-calls", kind: "tool-calls", what: "tool calls" }, - { name: "clear-tools", kind: "tools", what: "tool calls and outputs" }, -] as const satisfies readonly { name: string; kind: ClearKind; what: string }[]; +/** What to clear: only outputs, or calls and outputs; `keep` is how many of the last ones stay whole. */ +export type ClearArgs = { outputsOnly: boolean; keep: number }; -/** Reads `keep N`, `N` or nothing (keep 0); undefined when the text is something else. */ -export function parseKeep(args: string): number | undefined { - const m = /^\s*(?:keep\s+)?(\d+)?\s*$/i.exec(args); - return m ? Number(m[1] ?? 0) : undefined; +/** Reads `[outputs] [keep N]` (`keep` may be left out before N); a reason when the words are something else. */ +export function parseClearArgs(args: string): ClearArgs | { error: string } { + const words = args.trim().toLowerCase().split(/\s+/).filter(Boolean); + const outputsOnly = words[0] === "outputs"; + const rest = outputsOnly ? words.slice(1) : words; + const hasKeep = rest[0] === "keep"; + const n = hasKeep ? rest[1] : rest[0]; + const extra = rest[hasKeep ? 2 : 1]; + if (n !== undefined && !/^\d+$/.test(n)) + return { error: hasKeep ? `keep needs a whole number, not "${n}"` : `unknown word "${n}"` }; + if (hasKeep && n === undefined) return { error: "keep needs a whole number" }; + if (extra !== undefined) return { error: `unknown word "${extra}"` }; + return { outputsOnly, keep: Number(n ?? 0) }; } -/** The kind and N of `/compact [keep N]`; undefined for any other compaction. */ -export function clearRequest(instructions: string | undefined): { kind: ClearKind; keep: number } | undefined { - const asked = /^\s*(tool-outputs|tool-calls|tools)\b(.*)$/is.exec(instructions ?? ""); - const keep = asked ? parseKeep(asked[2] ?? "") : undefined; - return asked && keep !== undefined ? { kind: asked[1]?.toLowerCase() as ClearKind, keep } : undefined; +/** The `/compact` arguments that run `a`. */ +export const compactArgs = (a: ClearArgs): string => `tools ${a.outputsOnly ? "outputs " : ""}keep ${a.keep}`; + +/** What `a` clears, in words. */ +export const clearWhat = (a: Pick): string => + a.outputsOnly ? "tool outputs" : "tool calls and outputs"; + +/** The request in `/compact tools [outputs] [keep N]`; undefined for any other compaction. */ +export function clearRequest(instructions: string | undefined): ClearArgs | undefined { + const asked = /^\s*tools\b(.*)$/is.exec(instructions ?? ""); + const a = asked ? parseClearArgs(asked[1] ?? "") : undefined; + return a && !("error" in a) ? a : undefined; } -/** Clears what `kind` names, keeping the last `keep` of each; `note` is the toast that says what it did. */ -export function clearTools(messages: readonly SessionMessage[], kind: ClearKind, keep: number) { - const calls = kind === "tool-outputs" ? undefined : clearToolCalls(messages, keep); - const outputs = kind === "tool-calls" ? undefined : clearToolOutputs(calls?.messages ?? messages, keep); - const parts = [ - ...(calls ? [`${calls.cleared} tool calls`] : []), - ...(outputs ? [`${outputs.cleared} tool outputs`] : []), - ]; - const chars = (calls?.chars ?? 0) + (outputs?.chars ?? 0); - const kept = keep > 0 ? `, kept the last ${keep}` : ""; - return { - messages: outputs?.messages ?? calls?.messages ?? messages, - note: `Cleared ${parts.join(" and ")} (~${tokens(chars / 4)} tokens)${kept}`, - }; +const count = (n: number, what: string): string => `${n} ${what}${n === 1 ? "" : "s"}`; + +/** + * What the clear costs: the next call writes the whole new context to the cache again (estimated from the last + * call's input less the tokens removed), and each later call reads `saved` fewer tokens. The payback is that + * write's extra over a read, over the per-call read saving; the TTL is not known, so both are given. + */ +function costNote(saved: number, l: Pick): string { + if (l.lastCtx === 0) return " The cache re-write cost shows once a call is measured."; + const rewrite = Math.max(0, l.lastCtx - saved); + const usd5 = l.model === undefined ? undefined : cacheWriteUsd(l.model, rewrite); + const usd1 = l.model === undefined ? undefined : cacheWriteUsd(l.model, rewrite, CACHE_WRITE_1H); + const price = + usd5 === undefined || usd1 === undefined ? "" : ` (~${usd(usd5)} with the 5-min cache, ~${usd(usd1)} with 1 h)`; + const payback = (write: number) => Math.ceil((rewrite * (write - CACHE_READ)) / (saved * CACHE_READ)); + const back = + saved > 0 + ? `; it pays back after ~${payback(CACHE_WRITE_5M)} calls (5 min) or ~${payback(CACHE_WRITE_1H)} (1 h)` + : ""; + return ` The next call re-writes ~${tokens(rewrite)} tokens to the cache once${price}${back}.`; +} + +/** + * Clears what `a` asks for, keeping the last `a.keep` of each; `note` is the toast that says what it did and what + * it costs (from `l`, the live figures). A clear that would change nothing is a `skip`: no compaction at all. + */ +export function clearTools( + messages: readonly SessionMessage[], + a: ClearArgs, + l: Pick, +): { messages: SessionMessage[]; note: string } | { skip: string } { + const calls = a.outputsOnly ? undefined : clearToolCalls(messages, a.keep); + const outputs = clearToolOutputs(calls?.messages ?? messages, a.keep); + if ((calls?.cleared ?? 0) + outputs.cleared === 0) return { skip: "quantext: nothing to clear" }; + const parts = [...(calls ? [count(calls.cleared, "tool call")] : []), count(outputs.cleared, "tool output")]; + const chars = (calls?.chars ?? 0) + outputs.chars; + const kept = a.keep > 0 ? `, kept the last ${a.keep}` : ""; + const done = `Cleared ${parts.join(" and ")}${kept}: ${grouped(chars)} chars less (~${approx(chars)} tokens).`; + return { messages: outputs.messages, note: done + costNote(Math.round(chars / 4), l) }; } diff --git a/packages/mod/src/format.ts b/packages/mod/src/format.ts index 046470d..44128cd 100644 --- a/packages/mod/src/format.ts +++ b/packages/mod/src/format.ts @@ -1,5 +1,14 @@ /** Tokens, short: `950` → `1k`, `333_000` → `333k`, `1_234_567` → `1.2M`. */ export const tokens = (n: number): string => (n >= 1e6 ? `${(n / 1e6).toFixed(1)}M` : `${Math.round(n / 1000)}k`); +/** Tokens at ~4 characters each: `850`, `4.2k`, `12k`. */ +export const approx = (chars: number): string => { + const n = Math.round(chars / 4); + return n < 1000 ? `${n}` : n < 10_000 ? `${(n / 1000).toFixed(1)}k` : tokens(n); +}; + +/** A whole number with thousands commas: `11,950`. */ +export const grouped = (n: number): string => String(Math.round(n)).replace(/\B(?=(\d{3})+(?!\d))/g, ","); + /** US dollars, short: two decimals below $100, whole dollars from there. */ export const usd = (n: number): string => (n >= 100 ? `$${Math.round(n)}` : `$${n.toFixed(2)}`); diff --git a/packages/mod/src/prices.ts b/packages/mod/src/prices.ts index 1dfd2d8..02b2828 100644 --- a/packages/mod/src/prices.ts +++ b/packages/mod/src/prices.ts @@ -3,7 +3,10 @@ * (packages/engine/src/providers/anthropic/pricing.ts); a 5-minute cache write is 1.25 times the input price. * The mod cannot import the engine, so the input prices are copied here. */ -const CACHE_WRITE_5M = 1.25; +export const CACHE_WRITE_5M = 1.25; +/** A 1-hour cache write, and a cache read, as multiples of the input price (provider list prices). */ +export const CACHE_WRITE_1H = 2; +export const CACHE_READ = 0.1; /** US dollars per million input tokens; the first match wins, so a longer id comes before its prefix. */ const INPUT_PRICE: readonly [RegExp, number][] = [ @@ -18,8 +21,8 @@ const INPUT_PRICE: readonly [RegExp, number][] = [ [/haiku/, 1], ]; -/** US dollars for writing `tokens` to the 5-minute cache on `model`; undefined for a model with no known price. */ -export function cacheWriteUsd(model: string, tokens: number): number | undefined { +/** US dollars for writing `tokens` to the cache on `model` (5-minute by default); undefined with no known price. */ +export function cacheWriteUsd(model: string, tokens: number, write = CACHE_WRITE_5M): number | undefined { const price = INPUT_PRICE.find(([pattern]) => pattern.test(model))?.[1]; - return price === undefined ? undefined : (tokens * price * CACHE_WRITE_5M) / 1e6; + return price === undefined ? undefined : (tokens * price * write) / 1e6; } diff --git a/packages/mod/src/register.ts b/packages/mod/src/register.ts index 289c8d6..229319b 100644 --- a/packages/mod/src/register.ts +++ b/packages/mod/src/register.ts @@ -1,21 +1,22 @@ -/** The Quantext mod: a footer label and nudges from in-process usage, the /clear-tool-* commands, /quantext view. */ +/** The Quantext mod: a footer label and nudges from in-process usage, /clear-tools, /quantext view. */ import type { EngineInterface, Register } from "claude-code"; -import type { ClearKind } from "./clear-tools.ts"; -import { CLEAR_COMMANDS, clearRequest, clearTools, parseKeep } from "./clear-tools.ts"; +import type { ClearArgs } from "./clear-tools.ts"; +import { clearRequest, clearTools, clearWhat, compactArgs, parseClearArgs, USAGE } from "./clear-tools.ts"; import { EMPTY, inputOf, readLive, statusLabel, updateLive } from "./live.ts"; import { cacheExpiryNudge, contextNudge } from "./nudges.ts"; import { openViewer } from "./viewer.ts"; /** - * Queues `/compact keep N`, which the session.compact hook answers. `$.session.compact()` would + * Queues `/compact tools [outputs] keep N`, which the session.compact hook answers. `$.session.compact()` would * skip that hook and get Claude Code's summary; `$.command.run` queues the command as if typed. */ -async function clearNow($: EngineInterface, name: string, kind: ClearKind, keep: number): Promise { +async function clearNow($: EngineInterface, a: ClearArgs): Promise { + const args = compactArgs(a); try { - await $.command.run({ command: "compact", args: `${kind} keep ${keep}` }); + await $.command.run({ command: "compact", args }); } catch (error) { const why = error instanceof Error ? error.message : String(error); - $.ui.toast(`${name} failed: ${why}; try /compact ${kind}`); + $.ui.toast(`/clear-tools failed: ${why}; try /compact ${args}`); } } @@ -48,12 +49,12 @@ function remeasureSoon($: EngineInterface): void { export const register: Register = (on) => { on("session.start", async ($, e, next) => { const started = await next(e); - for (const c of CLEAR_COMMANDS) - await $.command.register({ - name: c.name, - description: `Clear old ${c.what} from the context and keep all else (also: /compact ${c.kind})`, - argumentHint: "[keep N]", - }); + await $.command.register({ + name: "clear-tools", + description: + "Clear old tool calls and outputs, no summary. outputs: only outputs; keep N: leave the last N (also: /compact tools)", + argumentHint: "[outputs] [keep N]", + }); await $.command.register({ name: "quantext", description: "Quantext: open the live context viewer (starts the engine when it is not running)", @@ -63,13 +64,13 @@ export const register: Register = (on) => { return started; }); - for (const c of CLEAR_COMMANDS) - on("command.run", { command: c.name }, async ($, e) => { - const keep = parseKeep(e.args); - if (keep === undefined) return { text: `Usage: /${c.name} [keep N]` }; - $.clock.after(0, () => void clearNow($, c.name, c.kind, keep)); // Claude Code refuses a compaction inside command.run - return { text: keep > 0 ? `Clearing ${c.what}, keeping the last ${keep}…` : `Clearing ${c.what}…` }; - }); + on("command.run", { command: "clear-tools" }, async ($, e) => { + const a = parseClearArgs(e.args); + if ("error" in a) return { text: `/clear-tools: ${a.error}. ${USAGE}` }; + $.clock.after(0, () => void clearNow($, a)); // Claude Code refuses a compaction inside command.run + const what = clearWhat(a); + return { text: a.keep > 0 ? `Clearing ${what}, keeping the last ${a.keep}…` : `Clearing ${what}…` }; + }); on("session.measure", async ($, e, next) => { const r = await next(e); @@ -98,6 +99,7 @@ export const register: Register = (on) => { input: l.input + u.input_tokens + u.cache_creation_input_tokens, lastEndAt: endedAt, lastCtx: inputOf(u), + model: u.model, })); if (nudge !== undefined) $.ui.toast(nudge); return r; @@ -114,26 +116,28 @@ export const register: Register = (on) => { return { text: "quantext: opening the viewer…" }; }); - // Answers the clear commands and `/compact [keep N]` in Claude Code's place: no summary. + // Answers `/compact tools [outputs] [keep N]` (queued by /clear-tools) in Claude Code's place: no summary. on("session.compact", async ($, e, next) => { const asked = e.agentId === undefined ? clearRequest(e.instructions) : undefined; let r: Awaited>; + let note: string | undefined; if (asked === undefined) r = await next(e); else { - const cleared = clearTools(e.messages, asked.kind, asked.keep); - $.ui.toast(cleared.note); + const cleared = clearTools(e.messages, asked, await readLive($)); + if ("skip" in cleared) return cleared; // no change: no compact boundary, no cache re-write + note = cleared.note; r = { messages: cleared.messages }; } if (e.agentId === undefined && e.trigger !== "precompute" && r.messages !== undefined) { await updateLive($, (l) => ({ ...l, bigCtxNudged: false })); remeasureSoon($); } + if (note !== undefined) $.ui.toast(note); // after the update, so a failed clear shows no "Cleared" return r; }).catch((_$, e, next) => { // A failed clear must not fall back to a paid summary; any other failure leaves it to Claude Code. const asked = e.agentId === undefined ? clearRequest(e.instructions) : undefined; - const what = CLEAR_COMMANDS.find((c) => c.kind === asked?.kind)?.what; - return what && !next.called ? { skip: `quantext: clearing ${what} failed` } : next(e); + return asked && !next.called ? { skip: `quantext: clearing ${clearWhat(asked)} failed` } : next(e); }); // A /clear ends this conversation and goes on with a new one: start its figures over. diff --git a/packages/mod/test/clear-tool-calls.test.ts b/packages/mod/test/clear-tool-calls.test.ts index 95e4dd8..556a51c 100644 --- a/packages/mod/test/clear-tool-calls.test.ts +++ b/packages/mod/test/clear-tool-calls.test.ts @@ -1,7 +1,6 @@ import type { SessionMessage } from "claude-code"; -import { expect, mock, test } from "claude-code/testing"; +import { expect, test } from "claude-code/testing"; import { clearToolCalls } from "../src/clear-tool-calls.ts"; -import { command, toasts } from "./kit.ts"; const call = (id: string, tool: string, input: Record): SessionMessage => ({ role: "assistant", @@ -78,55 +77,23 @@ test("small inputs, the last N calls and earlier stubs stay as they are", () => expect(again.messages[0]).toBe(once.messages[0]); }); -const chat: SessionMessage[] = [ - { role: "user", text: "write it", toolUses: [], handle: "p" }, - call("1", "Write", { file_path: "a.md", content: "a".repeat(8_000) }), - { ...result("1"), toolResults: [{ tool_use_id: "1", text: "o".repeat(4_000), isError: false }] }, -]; - -test("/clear-tool-calls and /clear-tools queue their /compact once the command has returned", async ($, on) => { - const clock = mock.clock(on, { now: 1_000_000 }); - const queued: string[] = []; - on("command.run", { command: "compact" }, (_$, e) => { - queued.push(e.args); - return { value: { text: "" } } as never; - }); - expect((await $.command.run(command("clear-tool-calls", "keep 2"))).text).toBe( - "Clearing tool calls, keeping the last 2…", - ); - expect((await $.command.run(command("clear-tools", ""))).text).toBe("Clearing tool calls and outputs…"); - expect((await $.command.run(command("clear-tool-calls", "all"))).text).toBe("Usage: /clear-tool-calls [keep N]"); - await clock.advance(1); - expect(queued).toEqual(["tool-calls keep 2", "tools keep 0"]); -}); - -test("/compact tool-calls stubs old tool inputs in place of a summary; tools does inputs and outputs", async ($, on) => { - mock.clock(on, { now: 1_000_000 }); - const seen = toasts(on); - let summarised = false; - on("session.compact", () => { - summarised = true; - return { messages: [] }; +test("inputs over 300 chars get a stub; fields up to 80 chars stay in it", () => { + const short = call("1", "Bash", { command: `echo ${"s".repeat(240)}`, description: "short" }); + const long = { command: `pnpm build ${"b".repeat(400)}`, description: "Build the plugin" }; + const r = clearToolCalls([short, call("2", "Bash", long)]); + expect(r.messages[0]).toBe(short); + expect(inputOf(r.messages[1])).toEqual({ + command: `[Old tool input cleared: Bash: pnpm build ${"b".repeat(49)} … (115 tokens)]`, + description: "Build the plugin", }); - const calls = await $.session.compact({ trigger: "manual", instructions: "tool-calls", messages: chat } as never); - expect(summarised).toBe(false); - expect(calls.messages?.[1]?.toolUses[0]?.input.content).toBe("[Old tool input cleared: Write a.md (2.0k tokens)]"); - expect(calls.messages?.[2]?.toolResults?.[0]?.text).toBe("o".repeat(4_000)); - const both = await $.session.compact({ trigger: "manual", instructions: "tools", messages: chat } as never); - expect(both.messages?.[1]?.toolUses[0]?.input.content).toBe("[Old tool input cleared: Write a.md (2.0k tokens)]"); - expect(both.messages?.[2]?.toolResults?.[0]?.text).toBe("[Old tool result content cleared]"); - expect(seen).toEqual(["Cleared 1 tool calls (~2k tokens)", "Cleared 1 tool calls and 1 tool outputs (~3k tokens)"]); + expect(r.cleared).toBe(1); }); -test("a failed clear-tool-calls skips the compaction rather than pay for a summary", async ($, on) => { - mock.clock(on, { now: 1_000_000 }); - let summarised = false; - on("session.compact", () => { - summarised = true; - return { messages: [] }; - }); - const broken = { role: "assistant", text: "", handle: "b" }; // no toolUses: it throws - const r = await $.session.compact({ trigger: "manual", instructions: "tool-calls", messages: [broken] } as never); - expect(summarised).toBe(false); - expect(r).toEqual({ skip: "quantext: clearing tool calls failed" }); +test("an input the stub would not make clearly smaller stays as it is", () => { + const fields = Object.fromEntries(Array.from({ length: 8 }, (_, i) => [`field${i}`, "f".repeat(40)])); + const m = call("1", "mcp__x__y", fields); + const r = clearToolCalls([m]); + expect(r.messages[0]).toBe(m); + expect(r.cleared).toBe(0); + expect(r.chars).toBe(0); }); diff --git a/packages/mod/test/clear-tool-outputs.test.ts b/packages/mod/test/clear-tool-outputs.test.ts index 6c7fb7d..f39077c 100644 --- a/packages/mod/test/clear-tool-outputs.test.ts +++ b/packages/mod/test/clear-tool-outputs.test.ts @@ -1,8 +1,6 @@ import type { SessionMessage } from "claude-code"; -import { expect, mock, test } from "claude-code/testing"; +import { expect, test } from "claude-code/testing"; import { clearToolOutputs } from "../src/clear-tool-outputs.ts"; -import { parseKeep } from "../src/clear-tools.ts"; -import { command, toasts } from "./kit.ts"; const CLEARED = "[Old tool result content cleared]"; const use = (id: string): SessionMessage => ({ @@ -46,93 +44,15 @@ test("keep N leaves the last N outputs whole; earlier stubs do not count toward expect(again.messages[6]?.toolResults?.[0]?.text).toBe("z".repeat(40)); }); -test("keep parses as N, keep N or nothing", () => { - expect(parseKeep("")).toBe(0); - expect(parseKeep(" 12 ")).toBe(12); - expect(parseKeep("keep 3")).toBe(3); - expect(parseKeep("KEEP 3")).toBe(3); - expect(parseKeep("all of it")).toBeUndefined(); - expect(parseKeep("keep -1")).toBeUndefined(); - expect(parseKeep("keep 1.5")).toBeUndefined(); -}); - -test("/clear-tool-outputs keep N queues /compact tool-outputs keep N once the command has returned", async ($, on) => { - const clock = mock.clock(on, { now: 1_000_000 }); - const queued: string[] = []; - on("command.run", { command: "compact" }, (_$, e) => { - queued.push(e.args); - return { value: { text: "" } } as never; - }); - const r = await $.command.run(command("clear-tool-outputs", "keep 2")); - expect(r.text).toBe("Clearing tool outputs, keeping the last 2…"); - expect(queued).toEqual([]); - await clock.advance(1); - expect(queued).toEqual(["tool-outputs keep 2"]); -}); - -test("/clear-tool-outputs with other words answers its usage and compacts nothing", async ($, on) => { - const clock = mock.clock(on, { now: 1_000_000 }); - const queued: string[] = []; - on("command.run", { command: "compact" }, (_$, e) => { - queued.push(e.args); - return { value: { text: "" } } as never; - }); - const r = await $.command.run(command("clear-tool-outputs", "everything")); - expect(r.text).toBe("Usage: /clear-tool-outputs [keep N]"); - await clock.advance(1); - expect(queued).toEqual([]); -}); - -test("/compact tool-outputs keep N stubs old tool outputs in place of a summary", async ($, on) => { - mock.clock(on, { now: 1_000_000 }); - const seen = toasts(on); - let summarised = false; - on("session.compact", () => { - summarised = true; - return { messages: [] }; - }); - const r = await $.session.compact({ - trigger: "manual", - instructions: "tool-outputs keep 1", - messages: chat, - } as never); - expect(summarised).toBe(false); - expect(r.messages?.map((m) => m.toolResults?.[0]?.text ?? m.text)).toEqual([ - "read them", - "", - CLEARED, - "", - CLEARED, - "", - "z".repeat(40), - ]); - expect(seen).toEqual(["Cleared 2 tool outputs (~1k tokens), kept the last 1"]); -}); - -test("any other compaction, and a subagent's, is left to Claude Code", async ($, on) => { - mock.clock(on, { now: 1_000_000 }); - const summary = [{ role: "user" as const, text: "summary", toolUses: [] }]; - on("session.compact", () => ({ messages: summary })); - const plain = await $.session.compact({ trigger: "manual", instructions: "keep the plan", messages: chat } as never); - expect(plain.messages).toEqual(summary); - const sub = await $.session.compact({ - trigger: "manual", - agentId: "a1", - instructions: "tool-outputs", - messages: chat, - } as never); - expect(sub.messages).toEqual(summary); -}); - -test("a failed clear-tool-outputs skips the compaction rather than pay for a summary", async ($, on) => { - mock.clock(on, { now: 1_000_000 }); - let summarised = false; - on("session.compact", () => { - summarised = true; - return { messages: [] }; - }); - const broken = { ...result("9", ""), toolResults: [{ tool_use_id: "9", isError: false }] }; // no text: it throws - const r = await $.session.compact({ trigger: "manual", instructions: "tool-outputs", messages: [broken] } as never); - expect(summarised).toBe(false); - expect(r).toEqual({ skip: "quantext: clearing tool outputs failed" }); +test("a ToolSearch result stays whole, so the deferred tools it loaded stay loaded", () => { + const search: SessionMessage = { + role: "assistant", + text: "", + toolUses: [{ tool_use_id: "s", tool: "ToolSearch", input: { query: "select:Monitor" } }], + handle: "as", + }; + const r = clearToolOutputs([search, result("s", "Monitor"), use("1"), result("1", "x".repeat(500))]); + expect(r.messages[1]?.handle).toBe("rs"); + expect(r.messages[3]?.toolResults?.[0]?.text).toBe(CLEARED); + expect(r.cleared).toBe(1); }); diff --git a/packages/mod/test/clear-tools.test.ts b/packages/mod/test/clear-tools.test.ts new file mode 100644 index 0000000..8357177 --- /dev/null +++ b/packages/mod/test/clear-tools.test.ts @@ -0,0 +1,180 @@ +import type { On, SessionMessage } from "claude-code"; +import { expect, mock, test } from "claude-code/testing"; +import { command, stepper, toasts, usage } from "./kit.ts"; + +const CLEARED = "[Old tool result content cleared]"; +const STUB = "[Old tool input cleared: Write a.md (2.0k tokens)]"; +const USAGE = "Usage: /clear-tools [outputs] [keep N]"; + +const write = (id: string): SessionMessage => ({ + role: "assistant", + text: "", + toolUses: [{ tool_use_id: id, tool: "Write", input: { file_path: "a.md", content: "a".repeat(8_000) } }], + handle: `a${id}`, +}); +const result = (id: string, text: string): SessionMessage => ({ + role: "user", + text: "", + toolUses: [], + toolResults: [{ tool_use_id: id, text, isError: false }], + handle: `r${id}`, +}); +const chat: SessionMessage[] = [ + { role: "user", text: "write it", toolUses: [], handle: "p" }, + write("1"), + result("1", "o".repeat(4_000)), + write("2"), + result("2", "z".repeat(40)), +]; + +/** Records each `/compact` the mod queues, and the clock that lets the timer run. */ +function compactions(on: On) { + const clock = mock.clock(on, { now: 1_000_000 }); + const queued: string[] = []; + on("command.run", { command: "compact" }, (_$, e) => { + queued.push(e.args); + return { value: { text: "" } } as never; + }); + return { clock, queued }; +} + +test("/clear-tools queues /compact tools once the command has returned", async ($, on) => { + const { clock, queued } = compactions(on); + const run = async (args: string) => (await $.command.run(command("clear-tools", args))).text; + expect(await run("")).toBe("Clearing tool calls and outputs…"); + expect(await run("keep 2")).toBe("Clearing tool calls and outputs, keeping the last 2…"); + expect(await run("outputs")).toBe("Clearing tool outputs…"); + expect(await run("OUTPUTS keep 3")).toBe("Clearing tool outputs, keeping the last 3…"); + expect(await run(" 4 ")).toBe("Clearing tool calls and outputs, keeping the last 4…"); + expect(queued).toEqual([]); + await clock.advance(1); + expect(queued).toEqual([ + "tools keep 0", + "tools keep 2", + "tools outputs keep 0", + "tools outputs keep 3", + "tools keep 4", + ]); +}); + +test("/clear-tools with bad words says what is wrong and compacts nothing", async ($, on) => { + const { clock, queued } = compactions(on); + const run = async (args: string) => (await $.command.run(command("clear-tools", args))).text; + expect(await run("keep -1")).toBe(`/clear-tools: keep needs a whole number, not "-1". ${USAGE}`); + expect(await run("keep 1.5")).toBe(`/clear-tools: keep needs a whole number, not "1.5". ${USAGE}`); + expect(await run("keep")).toBe(`/clear-tools: keep needs a whole number. ${USAGE}`); + expect(await run("calls")).toBe(`/clear-tools: unknown word "calls". ${USAGE}`); + expect(await run("keep 2 outputs")).toBe(`/clear-tools: unknown word "outputs". ${USAGE}`); + expect(await run("keep 1 keep 2")).toBe(`/clear-tools: unknown word "keep". ${USAGE}`); + await clock.advance(1); + expect(queued).toEqual([]); +}); + +test("the old clear commands are gone", async ($, on) => { + mock.clock(on, { now: 1_000_000 }); + const names: string[] = []; + on("session.start", (_$, e) => ({ cwd: e.cwd })); + on("command.register", (_$, e) => { + names.push(e.name); + return { value: { command: e.name } }; + }); + await $.session.start({ cwd: "/", surface: null, isInteractive: false }); + expect(names.filter((n) => n.startsWith("clear"))).toEqual(["clear-tools"]); +}); + +test("/compact tools stubs old tool calls and outputs in one pass, in place of a summary", async ($, on) => { + const { step } = stepper(on); + const seen = toasts(on); + let summarised = false; + on("session.compact", () => { + summarised = true; + return { messages: [] }; + }); + await step($, usage(100_000, 0)); + const r = await $.session.compact({ trigger: "manual", instructions: "tools keep 1", messages: chat } as never); + expect(summarised).toBe(false); + expect(r.messages?.map((m) => m.toolUses[0]?.input.content ?? m.toolResults?.[0]?.text ?? m.text)).toEqual([ + "write it", + STUB, + CLEARED, + "a".repeat(8_000), + "z".repeat(40), + ]); + expect(r.messages?.map((m) => m.toolUses[0]?.tool_use_id ?? m.toolResults?.[0]?.tool_use_id)).toEqual([ + undefined, + "1", + "1", + "2", + "2", + ]); + expect(seen).toEqual([ + "Cleared 1 tool call and 1 tool output, kept the last 1: 11,950 chars less (~3.0k tokens). " + + "The next call re-writes ~97k tokens to the cache once (~$0.49 with the 5-min cache, ~$0.78 with 1 h); " + + "it pays back after ~374 calls (5 min) or ~617 (1 h).", + ]); +}); + +test("/compact tools outputs stubs only old tool outputs and keeps the calls", async ($, on) => { + mock.clock(on, { now: 1_000_000 }); + const seen = toasts(on); + const r = await $.session.compact({ trigger: "manual", instructions: "tools outputs", messages: chat } as never); + expect(r.messages?.[1]).toEqual(chat[1]); + expect(r.messages?.[3]).toEqual(chat[3]); + expect(r.messages?.[2]?.toolResults?.[0]?.text).toBe(CLEARED); + expect(r.messages?.[4]?.toolResults?.[0]?.text).toBe(CLEARED); + expect(seen).toEqual([ + "Cleared 2 tool outputs: 4,040 chars less (~1.0k tokens). The cache re-write cost shows once a call is measured.", + ]); +}); + +test("the old /compact forms, any other compaction, and a subagent's are left to Claude Code", async ($, on) => { + mock.clock(on, { now: 1_000_000 }); + const summary = [{ role: "user" as const, text: "summary", toolUses: [] }]; + on("session.compact", () => ({ messages: summary })); + for (const instructions of ["tool-outputs", "tool-calls keep 1", "keep the plan", "tools that matter"]) { + const r = await $.session.compact({ trigger: "manual", instructions, messages: chat } as never); + expect(r.messages).toEqual(summary); + } + const sub = await $.session.compact({ + trigger: "manual", + agentId: "a1", + instructions: "tools", + messages: chat, + } as never); + expect(sub.messages).toEqual(summary); +}); + +test("a failed clear skips the compaction rather than pay for a summary", async ($, on) => { + mock.clock(on, { now: 1_000_000 }); + let summarised = false; + on("session.compact", () => { + summarised = true; + return { messages: [] }; + }); + const noUses = { role: "assistant", text: "", handle: "b" }; // no toolUses: clearing calls throws + const both = await $.session.compact({ trigger: "manual", instructions: "tools", messages: [noUses] } as never); + expect(both).toEqual({ skip: "quantext: clearing tool calls and outputs failed" }); + const noText = { ...result("9", ""), toolResults: [{ tool_use_id: "9", isError: false }] }; // no text: it throws + const outs = await $.session.compact({ + trigger: "manual", + instructions: "tools outputs", + messages: [noText], + } as never); + expect(outs).toEqual({ skip: "quantext: clearing tool outputs failed" }); + expect(summarised).toBe(false); +}); + +test("a clear that would change nothing installs no compaction and says so", async ($, on) => { + mock.clock(on, { now: 1_000_000 }); + const seen = toasts(on); + const once = await $.session.compact({ trigger: "manual", instructions: "tools", messages: chat } as never); + const again = await $.session.compact({ + trigger: "manual", + instructions: "tools", + messages: once.messages, + } as never); + expect(again).toEqual({ skip: "quantext: nothing to clear" }); + const short = await $.session.compact({ trigger: "manual", instructions: "tools keep 9", messages: chat } as never); + expect(short).toEqual({ skip: "quantext: nothing to clear" }); + expect(seen).toHaveLength(1); +}); diff --git a/packages/mod/types/index.d.ts b/packages/mod/types/index.d.ts index f5c8bc3..3baecec 100644 --- a/packages/mod/types/index.d.ts +++ b/packages/mod/types/index.d.ts @@ -10,6 +10,8 @@ export type Live = { /** When the last main-loop call ended (ms since the epoch), and its input size; 0 before the first. */ lastEndAt: number; lastCtx: number; + /** The model of the last main-loop call, to price a cache re-write; absent before the first. */ + model?: string; /** Whether the large-context nudge was shown since the last compaction or /clear. */ bigCtxNudged: boolean; };