Skip to content

fix(host): read Markdown under .github/workflows as instructions, not workflows (#1110) - #1139

Merged
pengfei-threemoonslab merged 6 commits into
mainfrom
claude/issue-1110-workflow-markdown
Nov 15, 2026
Merged

pengfei-threemoonslab merged 6 commits into
mainfrom
claude/issue-1110-workflow-markdown

Conversation

@pengfei-threemoonslab

Copy link
Copy Markdown
Contributor

What was wrong

The repository reader behind diff, verify's host comparison, check's rows and audit --host --drift filed every source the boundary registry finds under .github/workflows/ as a GitHub workflow, before its Markdown check ran:

  • _source_kind returned workflow for any path under .github/workflows/;
  • _audit_hosts forced host github for the same prefix.

The registry enumerates .github/workflows/AGENTS.md (shared adapter, **/AGENTS.md) and .github/workflows/CLAUDE.md (Claude Code adapter, **/CLAUDE.md), because Codex and Claude Code read nested instruction files. The reader then parsed that Markdown as a workflow document and recorded a blocking parse_failed under github.

  • Public case. pydantic/pydantic-ai PR #12653 (base e2332baa, head b86b6ff9) came back incomparable (base_inventory_incomplete; head_inventory_incomplete) with zero rows, hiding the PR's changes to .github/workflows/bots.yml and ci.yml.
  • Unchanged Markdown. The comparison held, but listed Not compared … github .github/workflows/AGENTS.md — parse_failed, and check added BOUNDARY-INPUT-INCOMPLETE.
  • Baselines. audit --host --save-baseline refused any repository with Markdown in that directory ("inventory is incomplete").

check's own evaluator (_is_workflow_path) and the Stop hook's host_config_change_patterns already took only .yml/.yaml workflows in the root directory. The diff/verify/drift reader was the odd one out.

The fix

  • One predicate. core/boundary_registry.py gains GITHUB_WORKFLOW_GLOBS (.github/workflows/*.yml, .github/workflows/*.yaml) and is_github_workflow_path, which is exactly a case-folded match against those globs. The shared adapter now splats the same constant into its globs, so its values and order are unchanged.
  • Both classifiers use it. _source_kind and _audit_hosts in core/host_grants.py call that one predicate, so a source is github exactly when it is read as a workflow.
  • Everything else in the directory. Whatever else the registry enumerates there is read by its own adapter, as it would be in any other directory:
    • AGENTS.md under claude-code, codex and cursor; CLAUDE.md under claude-code. Both are plain instruction (guidance) sources, so a prose edit is a changed_without_grant_change source with no row, exactly as for docs/AGENTS.md.
    • .mcp.json, .codex/config.toml and .claude/hooks/hooks.json below the directory had been published as empty github workflow grants. They are now read as MCP, Codex and hook configuration.
  • Unchanged. Case is still folded (.GitHub/Workflows/CI.YML is a workflow). A nested copy (samples/x/.github/workflows/ci.yml) keeps today's handling. Reusable-workflow resolution already refused a non-.yml/.yaml target, so nothing starts treating Markdown as a reusable workflow; a test now pins that.

check's _is_workflow_path is not changed; see Out of scope below.

Public result after the fix

./shipgate diff --workspace <pydantic-ai clone> --base e2332baaba4feb2c92f5ed47b7343374c2ba3546 returns comparable, with no incomparable_reasons and no unchanged_limits. It has two rows:

  • critical changed github .github/workflows/bots.yml
  • high changed github .github/workflows/ci.yml

The coverage items for the instruction files:

Nothing blocks the comparison. The #1004 directory-link rule is not involved.

Published changes

  • Changes, for non-workflow sources under .github/workflows/ only. Their artifact host, kind and artifact_id change, and their parse_failed issues and bogus github workflow grants are gone. So do the Not compared line and check's BOUNDARY-INPUT-INCOMPLETE that they caused.
  • Stable. Root workflow grant ids and config_sha256 are unchanged; ci.yml's host_grant_a85864… is the same on main and here. sources_expected, the registry's glob values, the Stop hook patterns, goldens and fingerprints are unchanged too.
  • Old baselines. A baseline saved by main on a repository without such files compares with no drift (checked: main-saved baseline, drift on this branch, no changes). Main could not save a baseline with Markdown there at all. The only main-saved baseline that drifts is one holding one of those JSON/TOML files: its former github grant reads as removed and its own grants as added. The CHANGELOG says so.
  • No schema, contract or check ID moves. No STABILITY migration note: no documented meaning changes. The support table's .github/workflows/* cell now says .github/workflows/*.yml and *.yaml, which is what was always meant. No distribution-surface claim changes (docs/distribution-surfaces.md untouched).

Tests

tests/test_workflow_directory_instructions.py has 23 cases. Its shard weight is in tests/shard_seconds.json, appended at the end, away from #1138's lines. All its route cases failed on main before the fix.

  • Classification table.
    • Root .yml/.yaml (case-folded) are workflows.
    • AGENTS.md, CLAUDE.md, Agents.md, a subdirectory AGENTS.md and a .mcp.json in the directory keep their adapters' kinds and hosts.
    • A nested samples/x/.github/workflows/ci.yml keeps kind config and the shared hosts.
  • check agreement. diff's classification agrees with check's _is_workflow_path on those files.
  • Reusable workflows. A Markdown reusable target is never followed.
  • Routes. An unchanged and a changed AGENTS.md (beside ci.yml) and CLAUDE.md (beside release.yaml), each next to a widened workflow, go through diff text and JSON, verify --preview, audit --host --drift against a baseline saved at the base, check and audit --host. Each must be:
    • comparable;
    • with the workflow row on every route and workflow_write_changed from drift;
    • with no parse_failed, Not compared or BOUNDARY-INPUT-INCOMPLETE;
    • with the instruction artifact parsed under the adapter's hosts and its coverage item under those hosts.
  • Same as anywhere else. A prose edit to .github/workflows/AGENTS.md produces the same coverage item as to docs/AGENTS.md.
  • The public link shape. CLAUDE.md is an unchanged link to AGENTS.md: comparable, read through, and the same as a docs/ link.
  • Markdown alone. A Markdown-only edit beside an unchanged workflow is comparable, with no rows.

Checks.

  • Full suite (-m "not perf" -n auto): 24088 passed, 2 failed, 16 skipped, 120 xfailed in 36 min. The 2 failures are the known environmental tests/test_check_unmodelled_host_config_keys.py[local_settings_enabled_plugins-worktree|git_range]: the fixture's git commit -qm base exits 1 because this machine's global git-ignore excludes the local settings file.
  • ruff check .: clean.
  • python scripts/generate_schemas.py --check: clean.

Out of scope (noted, not fixed)

check's _is_workflow_path (core/host_boundary.py) has a latent bug of its own. It never checks the .github/workflows/ prefix; it slices path[len(".github/workflows/"):] after the shared adapter matched. So policies/agent-boundary.shipgate.yaml and any policies/<name>.shipgate.yaml long enough return True, and check evaluates a changed policy pack as a workflow. Verified on this branch and main alike: a malformed policies/foo.shipgate.yaml gets HOST-CONFIG-PARSE-FAILED (workflow_yaml_parse_failed), and one with a top-level permissions: write-all gets HOST-WORKFLOW-WRITE-ALL and block. It also compares the extension case-sensitively, so .github/workflows/CI.YML is not evaluated, while the registry and this reader fold case. Making it call is_github_workflow_path would fix both. That change touches check's decisions and host_boundary.py's import block (PR #1133's area), so it is left for a follow-up.

Surface discipline

No new public surface. The fix moves blocked-recall: a workflow permission change was hidden behind an incomparable result.

Notes

Closes #1110

🤖 Generated with Claude Code

… workflows (#1110)

The repository reader filed every source the boundary registry found
under `.github/workflows/` as a GitHub workflow (`_source_kind`) under
host `github` (`_audit_hosts`), before the Markdown check ran. The
`AGENTS.md` Codex reads there and the `CLAUDE.md` Claude Code reads were
parsed as workflow documents and recorded as blocking `parse_failed`, so
pydantic/pydantic-ai PR #12653 came back `incomparable` with no rows,
hiding its changes to `bots.yml` and `ci.yml`. With the Markdown
unchanged the comparison held but listed it under `Not compared`,
`check` added `BOUNDARY-INPUT-INCOMPLETE`, and `--save-baseline`
refused the repository.

Both classifiers now ask one registry predicate,
`is_github_workflow_path`: exactly the shared adapter's root workflow
globs (`.github/workflows/*.yml|*.yaml`), case-folded as before, which
is also what `check` evaluates for these files. Anything else the
registry enumerates there is read by its own adapter, as in any other
directory. A nested copy keeps its handling; root workflow grant ids,
`config_sha256` and `sources_expected` do not move.
Conflicts: CHANGELOG.md keeps both Unreleased entries (Host comparison,
then #1130's Changes); tests/shard_seconds.json keeps both files' weights
and both supplemental measurements.
Conflict: tests/shard_seconds.json keeps both files' weights and both
supplemental measurements. CHANGELOG.md auto-merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Workflow reader parses Markdown instruction files under .github/workflows and refuses the whole diff

1 participant