Repository navigation
Description
Observed
The 2026-09-14 review at e2ab00078b2ca5b9e73405109a6f09b820f5fc9e found a gap between the shipped host-review engine and the integration a first team is told to copy:
examples/github-actions/README.md:3describes its recipes as requiring a rootshipgate.yaml; the recommended first recipe is presented as adding a gate.docs/integrations.md:22–31starts withconfig: shipgate.yaml, although the released Route H measurement indocs/design-partner-pilot-results.md:95–97records successful manifest-freeverifyand six advisory change rows.- The host-grant recipe is scheduled baseline/drift, a different job from reviewing the current PR against Git history. Local
diffalready needs neither a manifest nor a committed baseline.
The product has a manifest-free per-change path, but a team that got value locally has no equally clear, exercised first PR recipe. A GitHub App is not required to fix that gap.
Why it matters
Repeated use should arrive on the next relevant PR without a developer remembering another audit command. Requiring declarations or baseline maintenance before that happens creates adoption work unrelated to the selected host-review job. The first integration must show what the current PR changed, explain unread coverage and let the team's reviewer decide.
Proposed
Add one copyable host-only advisory PR recipe to the existing GitHub Actions examples, using the current composite Action and existing manifest-free verifier/PR projection. Pin the Action source ref and package version from the tested released provenance rather than maintaining another handwritten version constant. Do not introduce a new renderer, Action wrapper, hosted service or gate vocabulary.
Make the base/head checkout explicit and available locally (fetch-depth: 0 or an equivalently exercised bounded fetch). The engine never fetches. Exercise the actual composite Action, not a script that merely reconstructs its expected CLI arguments. A manifest or pre-existing host baseline must be absent in the tested repository.
Default to advisory output, with no required check, fail-on-verdict policy or generated host instruction. Show one human PR summary naming changed capabilities, evidence, coverage limits and next action. Keep detailed machine output in the existing artifacts. Explain the minimum contents/PR-comment permissions, and preserve the workflow-summary/artifact fallback when a fork or token cannot post a comment. Do not ask for broad write permissions or use an untrusted-code execution path to make a fork appear supported.
Let the existing in-job trigger/comparison decide applicability; do not add a separate path allowlist that can silently miss the engine's supported cases. A correct no-change run stays quiet or gives a compact no-change status. An incomplete comparison, missing base or execution error is visible and distinguishable from no change, even though findings do not make this default advisory integration a merge requirement.
Acceptance
- The new complete recipe runs in a repository with supported host configuration and no
shipgate.yamlor committed baseline. README/quickstart link it as the next step after local host-diff value, while the existing builder/strict recipes retain their own scope. - A real GitHub-hosted run exercises the pinned public Action and package on exact base/head refs. Preserve the run URL, version/ref, outputs and test inputs; a source-only unit test cannot stand in for this observation.
- Changed permission/MCP examples agree with the same-version local CLI on subject, direction, expansion semantics and coverage; include a narrowing case so every row cannot become a widening. No-change and malformed known input have distinct outcomes.
- A second update to the PR produces the current result without accumulating duplicate actionable comments. An unrelated or covered no-change update does not create a new warning; missing history and unread inputs never disappear as quiet success.
- Advisory defaults do not install a blocking policy or require the check. Execution/setup failure remains explicit and the documentation distinguishes it from a review finding or merge decision.
- The recipe uses the minimum demonstrated permissions and records the tested same-repository/fork behavior. With comment permission absent, a reviewer can still find the same evidence and the limitation is explicit.
- No user code, agent or declared MCP server is executed to analyze the PR. Installation uses the pinned trusted release, not package code selected from an untrusted PR checkout.
- Run the published advisory host-diff pilot: first reviewer value and the next eligible change #571 records actual setup time, assistance, continued enablement and second-reviewer use when a participant opts into this recipe. The hosted test proves integration behavior, not external adoption.
Sequencing
The same hosted run should satisfy #570's still-open literal post-publication tag/Action replay and be linked there; record tag resolution as well as the generated immutable-SHA default. No second harness is needed.
P1 in #778, alongside #779 and #781 in the first week. #779 owns the first local route; this issue owns the concrete team-workflow handoff. #653/#571 can begin human local observations before this recipe is finished. Reuse #684/#662's delivered host comparison and PR projection; do not reopen their completed implementations unless this exercised recipe reveals a bounded defect.
#664 remains conditional on useful repeat use and an observed obstacle that survives this working Action route. #690 stays paused. Baseline governance, authenticated approval (#337/#555), required branch checks and organization policy are separate choices, not default setup for this recipe.
Journey delivery - 2026-09-16
Program: #778. Delivery milestone: Adoption Phase 2 - Decide and verify the correction. Accountable role: GitHub integration/release engineering.
The recipe and source repair in #786 are delivered. Finish only remaining evidence against the exact distributed Action/package: actual hosted PR output, fork/minimal-permission fallback, second-push sticky update, artifact/summary agreement and safe install imports. Reuse this hosted observation for #570 rather than building a second harness. The owner's existing hosted-run deferral is retained until an authorized execution is selected; a milestone is not a completed hosted run. #811's local journey and human attempts continue independently. Offer CI through #796 after first value, not as a prerequisite.
Planning checkpoints preserve #778's original 2026-09-14 start: Oct 14 / Nov 13 / Dec 13. They are evidence-review dates, not promises about external replies. Phases overlap; a later milestone never delays supported research, an already-possible correction or an individual four-week opportunity window. A named accountable owner is required before execution; the role here is not an assignment to an uninvolved person.
Activity
pengfei-threemoonslab commentedon Sep 15, 2026
Status — 2026-09-14. #786 adds the recipe, docs and a test that runs the recipe's inputs through
action.yml's own step on manifest-free PR branches, plusagents-shipgate==1.0.0runs of the sameverifyinvocation. The owner decided to skip the GitHub-hosted run for now, so the hosted acceptance items (run URL and outputs on exact refs, second-push comment behaviour, fork/permission observation) remain open and unclaimed here. A defect found while isolating the route is filed separately as #785.Address review cycle 1 on the host-only advisory recipe (#780)
Add a host-only advisory PR recipe with no manifest or baseline (#780)
Address review cycle 1 on the host-only advisory recipe (#780)
Address review cycle 2 on the host-only advisory recipe (#780)
Address review cycle 3 on the host-only advisory recipe (#780)
Add a host-only advisory PR recipe with no manifest or baseline (#780) (
pengfei-threemoonslab commentedon Sep 15, 2026
PR #786 was squash-merged as
3131c9c30c47e98d5eebde9e08f8038f209c4afaafter three independent review/address cycles and green required checks.Delivered
examples/github-actions/14-host-only-advisory-pr.yml: no manifest or baseline,ci_mode: advisory,diff_base: target, sticky PR comment.python -P, so a PR cannot plantpip/oragents_shipgate/modules in the workspace and have them run before the engine installs. Non-GitHub CI recipes run python -m pip from the untrusted checkout, so a PR-supplied pip/ package can execute before install #787 tracks the other recipes.diff.Still open (as of 2026-09-14)
pengfei-threemoonslab commentedon Sep 22, 2026
Hosted run of the host-only advisory PR recipe — published v1.1.0, 2026-09-22
The owner authorized one run on 2026-09-22, against the published 1.1.0 Action and package only. v1.0.0 was not run. Nothing in this repository was changed: contents, tags, releases and settings are as they were.
main6412a5529ed7a60256d77e061c58322d6674ee0e). It holdsREADME.md,.claude/settings.json,.mcp.jsonand the workflow. It has noshipgate.yamland no saved baseline.examples/github-actions/14-host-only-advisory-pr.ymlas published at tagv1.1.0, with only its two pins changed:@v1.0.0→@v1.1.0andshipgate_version: '1.0.0'→'1.1.0'. The recipe at the tag still pins 1.0.0; the pin move is Move the published-release pins to v1.1.0 #853 (open).Release under test
ThreeMoonsLab/agents-shipgate@v1.1.0in 9 runs;@e3c6cb0c7657d9c53d4e29b2061d04dcf99a4e9bin run 35774401670e3c6cb0c7657d9c53d4e29b2061d04dcf99a4e9b(log line:Download action repository … (SHA:e3c6cb0c…))refs/tags/v1.1.0→ annotated tag39514a0a4e2aa8c3bab7c41c46db0997b980c8a1→e3c6cb0c7657d9c53d4e29b2061d04dcf99a4e9bpython -P -m pip install agents-shipgate==1.1.0from PyPI (Downloading agents_shipgate-1.1.0-py3-none-any.whl (2.6 MB), not cached);tool.version1.1.0; runner CPython 3.12.14038bdb4650d45d9c81996f60d33781b5671bfb57f006233f80e74db8a7377d33. The run does not record it (#855). The value comes from PyPI's only 1.1.0 file, the matchingv1.1.0GitHub Release asset, and a local download whose 609 installed files match its RECORDScenarios
Every job ended
success. "Local 1.1.0" isagents-shipgate diff --workspace <clone at the run's merge commit> --base <the run's base> --jsonfrom the PyPI 1.1.0 wheel (local CPython 3.13.12). It was compared as a whole ordered object withverifier.jsonhost_comparison: status, reasons, base, unchanged limits, rows, coverage and review.Bash(npm test *)→Bash(npm *);Bash(git push*)deny → allow⚠ medium / moved,⚠ medium / widened, coverage block (settings compared, 4 rows; workflow and.mcp.jsonunchanged), question "these 2 declared capability changes (from 4 rows)", Compared/Reproduce/AdvisoryRead(src/**))Bash(npm:*)→Bash(npm test:*)medium / narrowed, "the new rule matches only what the old rule matched", no⚠anywhere, widenings 0⚠ high / added,url https://mcp.example.com/<redacted-path>; header keys X-Fixture-Key. The path and header value appear in no log, artifact, comment or local output.mcp.jsonhead_inventory_incomplete" names.mcp.jsonparse_failed;Inputs:line, notCompared:; no question; job not failedfetch-depth: 1plus a wideningshallow_history", next actionfetch_base; job not faileddiff --base origin/mainrefuse with exit 2config_error(JSON on stderr)PullRequests: read; comment POST returned 403; warning annotation raised; PR has 0 comments. The same review is in the job summary and the artifact (pr-comment.md,verifier.json); see #854, #856pip/__init__.pyandagents_shipgate/__init__.py(each prints a marker only)changed_files; no MCP server ornpxranpython -m pipandpython -without-Pprint both markersuses:ref, same settings change as 1e3c6cb0c. Settings rows equal scenario 1's. The extrahigh / changedworkflow row says "adds no scope" and has no⚠Downloaded logs, artifacts, comment edit history and local CLI outputs are kept with the run evidence.
Acceptance
shipgate.yamland no baseline, and README/quickstart/integrations link it. The caveat: at tagv1.1.0the published recipe still pins 1.0.0 and says the hosted run is outstanding, so the run used it with the pins moved. That pin move is Move the published-release pins to v1.1.0 #853, still open.Status:andCritical: 0 - High: 0 - Medium: 0on every run (Host-only Action runs print an empty status, zero severity counts and nonexistent artifact paths in the job summary #854).contents: readandpull-requests: write. Same-repo PRs got a sticky comment. For the fork PR, see scenario 8. Not exercised: the first-time-contributor approval path, because the fork belongs to the owner's account (the run started with no approval wait).python -P -m pip install agents-shipgate==1.1.0from PyPI. As documented, thepull_requestworkflow file itself is PR-controlled.Defects filed from this run (milestone 11)
shipgate_versionrun records no wheel digest.## Agents Shipgateliterally.control.reasoncounts rows but calls them changes.addedplus a misleadingremoved.#780 stays open. Items 4 and 5 still need the three unexercised cases above: a no-change push to a PR that already has a comment, the "Not compared" case, and a hosted setup or execution failure. Item 1's caveat closes when #853 merges. Item 8 is recorded as moved to #571 and does not block this issue. The same run is recorded against #570's acceptance on #570.
pengfei-threemoonslab commentedon Sep 24, 2026
Days 1–5 reconciliation (PR #869): retain the ten published 1.1.0 hosted runs already recorded here. #853 is merged, removing the source-pin caveat. Remaining hosted scenarios are (1) no-change push to a PR with an existing comment, (2) Not compared for unread unchanged input, (3) explicit setup/execution failure. Milestone 11 no longer says all hosted execution is deferred. #854–#859 retain their defects; participant behavior stays in #571. Issue stays open; no redundant rerun of the original ten cases is selected.
pengfei-threemoonslab commentedon Oct 7, 2026
From #855: the same hosted 1.2.0 session must show the installed-engine digest line from scripts/action_engine_identity.py.