refactor(skill): a small core with a dispatch table, flows in reference files, hooks for the hard rules

.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "km",
-  "version": "1.8.1",
+  "version": "1.9.0",
   "description": "Knowledge management for markdown knowledge bases: answer from it with sources, save notes, decisions and transcripts with validated frontmatter, lint, mount peer brains, contribute to a team brain by PR, and set up a shared team brain with CI (/km init --team).",
   "author": {
     "name": "Roman Passler"

.github/workflows/test.yml

@@ -16,7 +16,7 @@ jobs:
       - uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
         with:
           python-version: '3.11'
-      - run: pip install pyyaml==6.0.3
+      - run: pip install pyyaml==6.0.3 pre-commit==4.6.2
       - name: the package installs with its templates
         run: |
           python -m venv /tmp/wheel && /tmp/wheel/bin/pip install --quiet . && /tmp/wheel/bin/km --version
@@ -30,6 +30,8 @@ jobs:
       - run: bash tests/doctor_smoke.sh
       - run: bash tests/sync_smoke.sh
       - run: bash tests/list_smoke.sh
+      - run: bash tests/skill_smoke.sh
+      - run: bash tests/hook_smoke.sh
       - name: team_smoke.sh, with a Latin-1 locale for its file-name encoding test
         run: sudo localedef -i en_US -f ISO-8859-1 en_US.ISO-8859-1 && bash tests/team_smoke.sh
 
@@ -85,10 +87,10 @@ jobs:
           source tests/lib.sh
           windows_like "" || { echo "::error::the runner's encoding is UTF-8, this job tests nothing"; exit 1; }
           with_env "" bash tests/encoding_smoke.sh    # as checked: UTF-8 mode off
-      - name: km doctor and km sync on Windows
+      - name: km doctor, km sync and the plugin's edit guard on Windows
         env:
           PYTHON: python
-        run: bash tests/doctor_smoke.sh && bash tests/sync_smoke.sh
+        run: bash tests/doctor_smoke.sh && bash tests/sync_smoke.sh && bash tests/hook_smoke.sh
 
   # Merging a version bump is the release: once main is green, tag v<__version__> if that tag is new.
   # Brains pin these tags, and `km init` writes the tag of the km that runs it.

.gitignore

@@ -3,3 +3,4 @@ __pycache__/
 .DS_Store
 build/
 *.egg-info/
+evals/results/

CLAUDE.md

@@ -19,3 +19,4 @@ brains. See README.md for the layout.
   templates pin the tag of the km that wrote them.
 - Run all the smoke tests before pushing; add a case for every behaviour change and check that it
   fails without the change.
+- A change under `skills/km/` runs the plugin evals before merge (CI does not run them, they cost money); `evals/README.md` says which cases and how to report them.

README.md

@@ -90,9 +90,9 @@ signed in and whether you push to the brain or through a fork. Each check that f
 fix; it needs no PyYAML, so it also runs on a computer that is half set up.
 
 Every command takes `--root DIR`; the default is the git work tree around the current directory.
-Install a release by its commit (`git ls-remote https://github.com/pasrom/km refs/tags/v1.8.1` shows
-it), not by the tag: `pip install git+https://github.com/pasrom/km@<commit>  # v1.8.1`, or run it
-without installing: `uvx --from git+https://github.com/pasrom/km@<commit> km validate  # v1.8.1`.
+Install a release by its commit (`git ls-remote https://github.com/pasrom/km refs/tags/v1.9.0` shows
+it), not by the tag: `pip install git+https://github.com/pasrom/km@<commit>  # v1.9.0`, or run it
+without installing: `uvx --from git+https://github.com/pasrom/km@<commit> km validate  # v1.9.0`.
 
 ## What a brain gets
 
@@ -101,13 +101,13 @@ Brains carry no km code. `km init` writes `CONVENTIONS.md`, `CLAUDE.md`, `schema
 generated indexes, a README, and CI that installs km through this repository's GitHub Action:
 
 ```yaml
-- uses: pasrom/km@<commit> # v1.8.1, installs the km CLI at that commit
+- uses: pasrom/km@<commit> # v1.9.0, installs the km CLI at that commit
 - run: km validate
 - run: km gen-index --check
 ```
 
 The pre-commit hook works the same way (`repo: https://github.com/pasrom/km`,
-`rev: <commit>  # frozen: v1.8.1`, `id: km-validate`). Every pin names a release by its commit, so a
+`rev: <commit>  # frozen: v1.9.0`, `id: km-validate`). Every pin names a release by its commit, so a
 tag moved later cannot change what a brain runs; the version rides along as a comment, and
 `km upgrade` checks that the commit is that release's. km's own dependencies are pinned to exact
 versions. `km init` and `km upgrade` look a release's commit up by its tag once, so a release tag here
@@ -117,27 +117,37 @@ a brain from the copy-in days into one that pins km.
 
 A team brain also keeps direct pushes off `main`, which the host may not enforce for a private
 repository on a free plan: its pre-commit config adds `no-commit-to-branch` for `main` from
-`pre-commit/pre-commit-hooks`, and a `main-guard` workflow fails and opens an issue when a commit
-reaches `main` without a merged pull request. `km upgrade` adds whichever a team brain lacks.
+`pre-commit/pre-commit-hooks` and a pre-push hook that refuses `git push origin main` (`pre-commit install`
+installs both, the config names both hook types and needs pre-commit 3.2 or newer; pre-commit checks
+the first ref of a push only and skips a push whose commits the remote already has on another ref, so
+a push of several refs at once or a first push of `main` gets past it), and a `main-guard` workflow fails and opens an
+issue when a commit reaches `main` without a merged pull request. `km upgrade` adds whichever a team
+brain lacks. The plugin itself refuses the file-edit tools (Edit, Write, MultiEdit, NotebookEdit) inside
+a mounted brain (a brain checked out as a submodule of a brain, found through git, not by path) and
+in a mount that is not checked out yet: a correction goes through a pull request against that
+brain, in a throwaway clone, as the skill describes; a shell edit is not caught.
 
 ## Layout
 
 ```
 .claude-plugin/        plugin and marketplace manifests
-hooks/                 the plugin's SessionStart hook: km sync in a brain
+hooks/                 the plugin's hooks: km sync when a session starts in a brain, no edit in a mounted brain
 action.yml             GitHub Action: installs km at the action's version
 .pre-commit-hooks.yaml pre-commit hook km-validate
 src/km/                the package: cli, validate, gen_index, demote, serve, promote, approve, repin, cite, cites, init, upgrade, doctor, sync, list_docs
 src/km/templates/      what `km init` writes
-skills/km/SKILL.md     the skill; skills/km/bin/km runs the package from the installed plugin
+skills/km/SKILL.md     the skill's core: rules, a dispatch table, what every flow shares
+skills/km/reference/   the flows the core dispatches to, read on demand; skills/km/bin/km runs the package from the installed plugin
 tests/                 smoke tests
 ```
 
 ## Development
 
 The smoke tests need Python 3.11+, PyYAML and git: `bash tests/gate_smoke.sh`, `index_smoke.sh`,
-`team_smoke.sh`, `doctor_smoke.sh`, `sync_smoke.sh`, `list_smoke.sh`. CI runs them plus the GitHub Action on every pull request, and
-`encoding_smoke.sh`, `doctor_smoke.sh` and `sync_smoke.sh` on Windows (team_smoke runs the encoding test on macOS under non-UTF-8 locales). A release bumps
+`team_smoke.sh`, `doctor_smoke.sh`, `sync_smoke.sh`, `list_smoke.sh`, `skill_smoke.sh` (the split of the
+skill lost nothing, against the frozen pre-split SKILL.md) and `hook_smoke.sh` (the plugin's edit guard;
+the pre-push hook too where pre-commit is installed). CI runs them plus the GitHub Action on every pull request, and
+`encoding_smoke.sh`, `doctor_smoke.sh`, `sync_smoke.sh` and `hook_smoke.sh` on Windows (team_smoke runs the encoding test on macOS under non-UTF-8 locales). A release bumps
 `__version__` in `src/km/__init__.py`, `version` in `.claude-plugin/plugin.json` and the version in
 this README together; merging that to main is the release, CI tags `vX.Y.Z` once main is green.
 

evals/README.md

@@ -0,0 +1,9 @@
+# Evals
+
+One case per row of the dispatch table in `skills/km/SKILL.md`, plus a negative case. Each case scaffolds a brain with `km init` from this checkout (`lib/brain.sh`), so run with `--scaffold`; the write and shell cases need `--allow-tools Bash Write Edit`, and every shell command then runs in Claude Code's sandbox inside the fixture brain.
+
+Each case whose row names a reference file has a `reads-reference` grader (that file was read); `help` and `save content` name only the core, so they have none. Every case has one grader on the flow's key step (the `km` command it runs, the file it writes, the line it answers), which the pre-split skill passes as well. Graders are `tool_used` and `regex`, except one `llm` grader in `fix-in-mounted-brain`.
+
+Run from the plugin root: `claude plugin eval . --scaffold --no-publish --ablation none --allow-tools Bash Write Edit`; `--tag batch2` selects the nine dispatch cases, and the no-plugin baseline (`--ablation with-without`) is worth its cost only when a grader changes. `results/` is ignored.
+
+Before merging a change under `skills/km/`, run the cases whose dispatch-table row it touches, all of them for a change to the core or the table. Put the results in the PR body, with the baseline where a grader changed. A case that fails with the new skill and passes with the old one is a regression: fix the skill, not the grader.

evals/brains-list-remove/case.yaml

@@ -0,0 +1,4 @@
+schema_version: "1.1"
+name: brains-list-remove
+context:
+  scaffold_script: fixture.sh

evals/brains-list-remove/fixture.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+source "$(dirname "${BASH_SOURCE[0]}")/../lib/brain.sh"
+brain_with_peer "$PWD"

evals/brains-list-remove/graders/lists-peer.md

@@ -0,0 +1,4 @@
+---
+type: regex
+pattern: 'peer'
+---

evals/brains-list-remove/graders/reads-reference.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Read
+input_match: 'reference[/\\]+brains\.md'
+---

evals/brains-list-remove/graders/removes-the-submodule.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Bash
+input_match: 'git submodule deinit|git rm'
+---

evals/brains-list-remove/prompt.md

@@ -0,0 +1,7 @@
+---
+max_turns: 10
+tags: [batch2]
+allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]
+---
+
+/km brain list, then remove the brain peer. I confirm the removal already, do not ask me and do not wait.

evals/doctor/case.yaml

@@ -0,0 +1,4 @@
+schema_version: "1.1"
+name: doctor
+context:
+  scaffold_script: fixture.sh

evals/doctor/fixture.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+source "$(dirname "${BASH_SOURCE[0]}")/../lib/brain.sh"
+review_brain "$PWD"

evals/doctor/graders/reads-reference.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Read
+input_match: 'reference[/\\]+setup\.md'
+---

evals/doctor/graders/runs-km-doctor.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Bash
+input_match: 'km doctor'
+---

evals/doctor/prompt.md

@@ -0,0 +1,7 @@
+---
+max_turns: 10
+tags: [batch2]
+allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]
+---
+
+prüfe meine Einrichtung

evals/fix-in-mounted-brain/case.yaml

@@ -0,0 +1,4 @@
+schema_version: "1.1"
+name: fix-in-mounted-brain
+context:
+  scaffold_script: fixture.sh

evals/fix-in-mounted-brain/fixture.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+source "$(dirname "${BASH_SOURCE[0]}")/../lib/brain.sh"
+brain_with_peer "$PWD"

evals/fix-in-mounted-brain/graders/no-edit-of-the-mount.md

@@ -0,0 +1,6 @@
+---
+type: llm
+---
+
+PASS if the reply says the correction is made as a pull request against the peer brain's own repository, in a throwaway clone (or a separate clone), and does not propose editing the file under brains/peer directly.
+FAIL if the reply proposes editing brains/peer/docs/pump-timing.md in place, or committing in the mounted submodule, or gives no pull-request-based route at all.

evals/fix-in-mounted-brain/graders/read-brains-reference.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Read
+input_match: 'reference[/\\]+brains\.md'
+---

evals/fix-in-mounted-brain/graders/skill-fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:km:)?km"'
+---

evals/fix-in-mounted-brain/prompt.md

@@ -0,0 +1,6 @@
+---
+max_turns: 15
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+fix this in @peer: docs/pump-timing.md says the coolant pump starts 850 ms after the contactor closes, but the measurement from last week says 950 ms. Tell me how you would go about it and what the first steps are; do not ask me anything, just lay the steps out.

evals/help/case.yaml

@@ -0,0 +1,4 @@
+schema_version: "1.1"
+name: help
+context:
+  scaffold_script: fixture.sh

evals/help/fixture.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+source "$(dirname "${BASH_SOURCE[0]}")/../lib/brain.sh"
+review_brain "$PWD"

evals/help/graders/lists-brain-fix.md

@@ -0,0 +1,5 @@
+---
+type: regex
+pattern: 'brain fix'
+flags: i
+---

evals/help/graders/lists-queue.md

@@ -0,0 +1,5 @@
+---
+type: regex
+pattern: 'queue'
+flags: i
+---

evals/help/prompt.md

@@ -0,0 +1,7 @@
+---
+max_turns: 10
+tags: [batch2]
+allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]
+---
+
+/km help

evals/km-queue/case.yaml

@@ -0,0 +1,4 @@
+schema_version: "1.1"
+name: km-queue
+context:
+  scaffold_script: fixture.sh

evals/km-queue/fixture.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+source "$(dirname "${BASH_SOURCE[0]}")/../lib/brain.sh"
+review_brain "$PWD"

evals/km-queue/graders/read-queue-reference.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Read
+input_match: 'reference[/\\]+queue\.md'
+---

evals/km-queue/prompt.md

@@ -0,0 +1,6 @@
+---
+max_turns: 15
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+/km queue

evals/lib/brain.sh

@@ -0,0 +1,32 @@
+# Sourced by the eval fixtures: a brain made by `km init` from this checkout, in the empty workspace
+# the run starts in. tests/lib.sh gives `km` from this checkout and `fake_km_remote`, a local bare
+# repository tagged with this km's version that stands in for the km repository (KM_REPO_URL), since
+# km init pins a release by its commit.
+set -eu
+source "$(dirname "${BASH_SOURCE[0]}")/../../tests/lib.sh"
+# brain DIR INITIALS: a personal brain in DIR, committed
+brain(){
+  local remote; remote="$(mktemp -d)/km-remote"; fake_km_remote "$remote"
+  KM_REPO_URL="file://$remote" km init "$1" --initials "$2" >/dev/null
+  git init -q -b main "$1" && git -C "$1" "${G[@]}" add -A && git -C "$1" "${G[@]}" commit -q -m "chore: initialize knowledge base conventions"
+}
+# doc DIR PATH TITLE STATUS BODY: a doc with km frontmatter
+doc(){
+  mkdir -p "$(dirname "$1/$2")"
+  printf -- '---\ntitle: %s\ntype: note\nstatus: %s\nauthor: EV\ndate: 2026-01-15\ntags: [eval]\n---\n\n# %s\n\n%s\n' "$3" "$4" "$3" "$5" > "$1/$2"
+}
+# review_brain DIR: a brain with one doc in review, committed
+review_brain(){
+  brain "$1" EV
+  doc "$1" inbox/thermal-derating.md "Thermal derating curve" review "Above 45 degrees Celsius the charge current is reduced by 2 percent per degree."
+  git -C "$1" "${G[@]}" add -A && git -C "$1" "${G[@]}" commit -q -m "docs(inbox): thermal derating"
+}
+# brain_with_peer DIR: a brain that mounts a peer brain under brains/peer; the peer holds one accepted doc
+brain_with_peer(){
+  local T; T="$(mktemp -d)"
+  brain "$T/peer" PE
+  doc "$T/peer" docs/pump-timing.md "Pump timing" accepted "The coolant pump starts 850 ms after the main contactor closes."
+  git -C "$T/peer" "${G[@]}" add -A && git -C "$T/peer" "${G[@]}" commit -q -m "docs: pump timing"
+  brain "$1" EV
+  git -C "$1" "${G[@]}" submodule add -q "$T/peer" brains/peer && git -C "$1" "${G[@]}" commit -q -m "chore(brains): mount peer"
+}

evals/lint/case.yaml

@@ -0,0 +1,4 @@
+schema_version: "1.1"
+name: lint
+context:
+  scaffold_script: fixture.sh

evals/lint/fixture.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+source "$(dirname "${BASH_SOURCE[0]}")/../lib/brain.sh"
+review_brain "$PWD"

evals/lint/graders/reads-reference.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Read
+input_match: 'reference[/\\]+validate-and-gate\.md'
+---

evals/lint/graders/runs-km-validate.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Bash
+input_match: 'km validate'
+---

evals/lint/prompt.md

@@ -0,0 +1,7 @@
+---
+max_turns: 10
+tags: [batch2]
+allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]
+---
+
+/km lint

evals/lookup-question/case.yaml

@@ -0,0 +1,4 @@
+schema_version: "1.1"
+name: lookup-question
+context:
+  scaffold_script: fixture.sh

evals/lookup-question/fixture.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# A brain with one doc that holds a distinctive fact the prompt asks for.
+source "$(dirname "${BASH_SOURCE[0]}")/../lib/brain.sh"
+W="$PWD"
+brain "$W" EV
+doc "$W" inbox/pump-startup-delay.md "Coolant pump start-up delay" accepted \
+  "The coolant pump starts 850 ms after the main contactor closes, so that the inrush of the pump never overlaps the precharge phase. Measured on the bench rig in January 2026."
+git -C "$W" "${G[@]}" add -A && git -C "$W" "${G[@]}" commit -q -m "docs(inbox): pump start-up delay"

evals/lookup-question/graders/answer.md

@@ -0,0 +1,5 @@
+---
+type: regex
+pattern: '850\s*ms'
+target: last_message
+---

evals/lookup-question/graders/cites-source.md

@@ -0,0 +1,5 @@
+---
+type: regex
+pattern: 'pump-startup-delay\.md'
+target: last_message
+---

evals/lookup-question/graders/skill-fired.md

@@ -0,0 +1,5 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:km:)?km"'
+---

evals/lookup-question/prompt.md

@@ -0,0 +1,6 @@
+---
+max_turns: 15
+allowed_tools: [Read, Glob, Grep, Skill]
+---
+
+What do we know about the coolant pump start-up delay? Answer from this knowledge base and cite the source.

evals/not-a-brain-task/graders/finds-the-bug.md

@@ -0,0 +1,7 @@
+---
+type: regex
+pattern: '<=|off[- ]by[- ]one|out[- ]of[- ]bounds|n - 1|past the end'
+flags: i
+target: last_message
+arm: both
+---

evals/not-a-brain-task/graders/skill-not-fired.md

@@ -0,0 +1,8 @@
+---
+type: tool_used
+tool: Skill
+input_match: '"skill"\s*:\s*"(?:km:)?km"'
+min: 0
+max: 0
+arm: both
+---