Skip to content

Epic: earn the complete permission-review journey, from a real change to voluntary reuse #778

Open

Description

@pengfei-threemoonslab

PM backlog review — 2026-10-01

Application-agent PR proof (#868) remains the primary product track. Host correctness is bounded maintenance. This review organizes the 100 open issues; it does not establish delivery, user adoption or release qualification.

Workstream Current focus Exit evidence
Application comparisons #908 measurement, then #909 + #874, #910, #913 + #914; #911/#912 next #868's 10 useful, correct, complete cases; development and holdout counts reported separately on the released build
Host maintenance P1 #918, #920, #921; P1 #698 diagnosis Correct expansion/scope wording on each pinned reproduction; measured large-checkout latency with timeouts retained
Control correctness P0 #610 Empty planning completion cannot confer verifier publication, merge or completion authority; compatible runtime/schema/consumer behavior
Workflow and adoption #915 application delivery; remaining #780/#855 hosted evidence; #839 → #840 for the host decision/correction journey Actual hosted results and reviewer interpretation; voluntary use stays a separate observed outcome
Qualified claims #572 and its evidence dependencies Deferred, explicit qualification obligations; no substitution of advisory or engineering evidence
Shared maintenance Existing build, reader and diagnostics backlog Bounded acceptance and a reproduced current need before pickup

Evidence, not a delivery forecast. The latest #868/#908 ledger (2026-09-30, main 6ced6f70) reports 1 Q2 case / 49 development PRs, 9 with rows and zero false-complete answers in that sample. These are reported development measurements, not a fresh replay, adoption counts or representative market rates. The Oct 14 / Nov 13 / Dec 13 checkpoints remain evidence reviews.

Execution order. Resolve #908's corpus preservation and holdout-freeze prerequisite before merging the reader changes it gates. #909 owns list-expression evaluation; #874 owns parameter/constructor/instance flow. #910 owns object/built-in reader work, with #866 retaining its specific regression acceptance. #913 supplies richer observed effects; #914 supplies readable findings. #915 can prepare integration and ship with existing truthful text rather than waiting for #914. #916 is a bounded framework-selection study, not authorization to add a reader.

Avoid duplicate work. #584 is covered by #909 but stays open until its acceptance is demonstrated. #655/#867 need an acceptance-to-evidence reconciliation against the delivered application route before new implementation is selected. #795/#812 have delivered slices; #780 has ten recorded hosted runs; #855 still owes hosted digest proof. Do not repeat completed work or close residual obligations solely because a PR merged.

Queue policy. Every issue has one workstream:* label and one P0/P1/P2 label. Priority describes importance within its workstream; it is not a commitment to start every P1. status:deferred preserves an explicit prior deferral or conditional decision. #563 retains its historical P0 qualification significance with that status; it is not promoted into the advisory application sprint. Existing assignees are preserved; name an accountable implementer when work is picked up rather than assigning the entire backlog to one person.

Next product decisions. #908 must record a feasible freeze sequence for its >=30 holdout PRs created after Oct 1; no backdating or silent threshold change. #772 remains owner-deferred on privacy/noise. #502 owns contribution policy; #917 is a proposed example requiring version-pinned evidence, not a verified product defect. The recorded #830 and per-note outreach requirements remain in their owning issues; this triage sends no outreach and changes none of those requirements.

The dated selection above supersedes older execution order where they differ. Historical observations and acceptance below remain intact.


Selected execution — 2026-09-24

Days 1–5 execution is selected and recorded in PR #869. Application-agent proof (#868, milestone 13) is primary; host correctness is bounded maintenance. Accountable product owner: @pengfei-threemoonslab; technical executor: Codex current task. #580/#655 design precedes #867 and the reproduced #864/#865/#866 readers. #610 is reproduced; #787 is the bounded recipe repair. Milestones 9–11 were reconciled, retaining Oct 14 / Nov 13 / Dec 13 as evidence checkpoints. #830 and external outreach holds remain unchanged. Older recruitment-first and hosted-deferred text is superseded by this dated selection; no external value or ten-case result is claimed.


Earlier sequencing below is retained as history and superseded where it conflicts with this dated execution selection.

Product outcome

A reviewer can turn a real agent-permission change into an explicit, evidence-grounded decision, carry out the next step, verify the selected correction and choose to use the workflow on the next eligible change.

Real change → understand → decide → verify correction → voluntary next use.

This is the sole product program epic. #811 owns the selected experience; its implementation children #839 and #840 close the fact-to-decision and correction-verification gaps. #791 remains discovery coordination. No separate dashboard, control plane, research cohort or approval engine is introduced.

Current sequence — 2026-09-18

The owner's 2026-09-16 decision controls execution: build the selected work in milestones 10–12, run #830 on a released build, and begin external recruitment/outreach only after that gate passes and an explicit owner decision permits it. Internal preparation, fixture replay and product design may proceed now. This update performs no outreach and does not change #830's thresholds or per-note approval.

The original 2026-09-14 program clock and Oct 14 / Nov 13 / Dec 13 checkpoints remain. A missing participant is a dated shortfall, not value. Each participant's existing four-week observation window starts at first value; preserve any prior observations and never reset a window for this plan. #811's known-change decision-support hypothesis and #830's novel automatic author-actionable finding hypothesis retain separate measures and denominators.

Why this is the next product work

The evidence chain has useful foundations, but a generic "Does the team intend this declared capability change?" leaves the reviewer to infer the available choices and the condition a rerun would verify. More findings or severity labels do not supply business intent.

The selected product question is: Does the result let a non-author reviewer make and verify a concrete decision without a maintainer translating it? A justified decision to retain a known expansion is eligible value. Correctly identifying missing evidence is useful recovery; an incomplete comparison remains outside #571's first-valid-result definition.

Current facts — observed 2026-09-18

One decision-ready result

Reviewer question Product answer Owner
What changed? Exact before/after, source, original refs #795 / #819
What does this establish? Supported static semantics and direction; no invented runtime or business outcome #816 delivered; #820 / #827 and bounded readers
What is still unknown? Compared scope, partial/unread input, named recovery where established #812 / #821 / #808 / #822
What must I decide? One case-specific missing-intent question and relevant conditional options #839
Who acts, and what happens next? Existing operational control or advisory owner-to-be-assigned; exact verification target #839, existing fix/control contracts
Did the correction address that change? Original B→C and B→F, selected correction plus remaining/new deltas and current coverage #840
Will this help on the next change? Actual eligible opportunity and voluntary review consumption #571 / #796, after #830

Start with the supported shell-permission route. A refund demonstration from an application tool surface is a separate scenario and cannot be inferred from a host allow rule. Agent/framework breadth follows demonstrated value and the existing support boundaries.

Roadmap and phase exits

Phase / checkpoint Selected result Engineering proof Product evidence
Phase 1, 2026-10-14 Truthful, concrete facts a reviewer can use Close remaining release/presentation proof around #795/#812/#792/#781; distinguish no-change from unknown; first #798 case Internal cold-reader readiness and explicit shortfalls only; #653 recruitment remains gated
Phase 2, 2026-11-13 One supported change supports a decision and a verified correction Existing fields/direction/coverage work, then #839 + #840 inside #811; B/C/F/R and negative controls; source/package/hosted evidence separately Record whether independent internal readers identify fact, limits, decision, action and correction correctly; no external-adoption claim
Phase 3, 2026-12-13 Establish automatic value, then observed reviewer value and reuse when permitted #796 exercised route; #830 on released Phase 2 build, retaining V1 unchanged Only after #830 and owner approval: #653/#571 existing cohort, concrete decisions, real corrections and next eligible use; otherwise dated shortfall and bounded next decision

Dates are evidence-review checkpoints, not promised releases, recruitment dates or automatic permission to start outreach. Phase 3 engineering/gate readiness precedes outreach; its external-value outcome is necessarily observed afterwards. Do not require an observation produced only by outreach before allowing the prerequisite gate to run. Do not close the user outcome as achieved merely because the readiness work or V1 is complete.

Delivery order and capacity

Preserve the owner's Sept 16 engine order. Phase 1's merged fixes and #837/#838 are delivered inputs; remaining installed proof and owner-reviewed publication remain. Proposed 1.1.0/1.2.0 in the prior plan are release targets, not already published versions.

Phase 2: #819 → #820 → #821 → #808/#822 → #809 → #827 → #824 → #823 → #825 → #826 → #807 → #702 → #811, including #839 then #840 → #780/#570 owner-approved hosted evidence → #828 decision → conditional #829 → owner-reviewed release. A supported shell prototype may be designed earlier without expanding coverage claims or silently changing this delivery order.

Keep the capacity assumption of two engineers plus product/research ownership and about 20% engineering reserve. The new implementation slices are provisionally 3–5 and 5–8 engineering days after prerequisites; they are part of #811, not an additional platform project. Validate those estimates at pickup. If the date is missed, name delivered scope and the remaining dependency; do not drop an identity/coverage test or manufacture user evidence.

Measurement: three distinct questions

  1. Engineering readiness: are facts, direction, conditional guidance and correction explanations correct and reproducible? Require zero false correction confirmations across the selected adversarial matrix. Internal task timing measures readability, not adoption.
  2. Automatic discovery: Value gate before any outreach: measure automatic author-actionable value on a released build #830's M1–M6, strata, novelty, yield, precision/noise, independent adjudication and released-build requirements stay unchanged. Manual examples and known intentional changes cannot raise M4.
  3. Reviewer decision value: Run the published advisory host-diff pilot: first reviewer value and the next eligible change #571's existing first-value definition, full denominators and four-week window stay unchanged. Add decision/rationale, assistance, missing evidence, next owner/action, requested static correction and whether it was correctly verified. Count a justified retention, a correction request and useful recovery separately. A dispute never suppresses a finding.

Use the same cohort/ledger. Preserve invited, attempted, first_valid_result, first_value, second_change_eligible, second_change_observed. Compare the ordinary workflow and Agents Shipgate with balanced ordering where practical; record total reading/decision/correction time, errors and assistance. Scanner latency and positive comments are not saved review time.

Evaluate #571's existing ladder in order, on the same route: Continue at ≥2 unaided first values and ≥1 observed second eligible change; Stop at zero first value with ≥1 first valid result; Narrow otherwise. Report all denominators. No second eligible change means repeat use remains unobserved. Voluntary use is reported separately from automated CI execution and from the existing second-change threshold.

Strategic caution: a known-change result can improve a real decision without finding a novel problem. #830 and #811 therefore test different propositions. This roadmap preserves the owner-selected outreach gate; changing it later requires an explicit decision, not a metric substitution.

Authority, privacy and conditional expansion

Program acceptance

Activity

  1. added
    P1Picked up inside the current milestone window; keep about 15 open
    epicTracking issue coordinating a group of related issues
    area:releaseRelease pipeline, packaging, and safety qualification
    on Sep 15, 2026
  2. pengfei-threemoonslab commented on Sep 15, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Backlog execution applied — 2026-09-14

    The reviewed issue changes are live. Roadmap PR #782 updates the plan of record and archives the old pre-publication sequence. Its independent coding-agent review/address cycle is recorded in GitHub; it is not a human approval or qualification claim.

    The first execution wave is #779/#780/#781 alongside #653/#571, not ahead of all user contact. Each selected implementation follows its own acceptance and one-to-three PR review/address cycles. This planning pass has not performed product fixes, recruited participants or produced adoption counts.

  3. pengfei-threemoonslab commented on Sep 15, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Execution checkpoint — 2026-09-14/15 (engineering)

    What happened, what did not, and the next bounded decisions. Every implementation PR runs independent coding-agent review/address cycles, recorded on the PR, with required CI before merge. These are not human approvals.

    Merged

    Open, in review

    Decided / deferred

    Filed from this work

    Triaged (still reproduce on 45d6ca7a): #575, #577, #638.

    Not done / not claimed

    🤖 Generated with Claude Code

  4. pengfei-threemoonslab commented on Sep 15, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Checkpoint — 2026-09-14. Implementation PRs are squash-merged only after independent review/address cycles with no open P0–P2 and green required checks.

    Days 0–7: all three merged

    Days 8–30: in progress

    Reliability reserve

    Not started by design

  5. pengfei-threemoonslab commented on Sep 16, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Checkpoint — 2026-09-15. Every implementation PR is squash-merged only after independent review/address cycles with no open P0–P2 and green required checks.

    Merged

    Issue PR Merge Cycles
    #781 installed-kit pins #783 66665493 3
    #779 diff-first entry docs #784 67a3a5e7 3
    #780 host-only advisory recipe #786 3131c9c3 3
    #577 non-regular evidence readers #790 7ac1c456 2
    #771 workflow step action references #788 42110535 3
    #714 hook loading evidence #789 6030e706 4
    #575 + #785 workspace reports directory #801 ba10b486 3
    #638 base-cache namespace #805 ff300e2d 2

    In flight

    Open by design

    Filed from this pass: #785 (fixed in #801), #787, #802, #803 (rescoped to Windows), #804, #807, #808, #809, #810, #813.

  6. pengfei-threemoonslab commented on Sep 16, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Plan complete — 2026-09-16. Every implementation issue in this plan of record is delivered and closed. Each PR was squash-merged only after independent coding-agent review/address cycles with no open P0–P2 and green required checks.

    Issue PR Merge Cycles
    #781 installed-kit pins #783 66665493 3
    #779 diff-first entry docs #784 67a3a5e7 3
    #780 host-only advisory recipe #786 3131c9c3 3
    #577 non-regular evidence readers #790 7ac1c456 2
    #771 workflow step action references #788 42110535 3
    #714 hook loading evidence #789 6030e706 4
    #575 + #785 workspace reports directory #801 ba10b486 3
    #638 base-cache namespace #805 ff300e2d 2
    #693 reusable-workflow secret remapping #806 9df307ab 2

    Defects the loop caught before merge, each reproduced against published 1.0.0:

    Still open by design

    Filed from this pass: #785 (fixed in #801), #787, #802, #803 (rescoped to the Windows fallback), #804, #807, #808, #809, #810, #813.

  7. changed the title [-]Epic: post-1.0 adoption — make one real PR useful, then earn the next review[/-] [+]Epic: earn the complete permission-review journey, from a real change to voluntary reuse[/+] on Sep 16, 2026
  8. pengfei-threemoonslab commented on Sep 16, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Owner decision — 2026-09-16

    Build milestones 10, 11 and 12 first. Reach out to OSS maintainers or potential customers only afterwards, and only with value the tool produces automatically, gated by #830.

    The public pilot ledger, dated 2026-09-14, still records zero invitations, attempts, first values and second-change observations. That is the dated experiment count, not a census of users. This comment changes order and scope only. It performs no release, recruitment or outreach.

    Why: scan baseline (aggregated)

    A 2026-09-15 scan of 23 public PRs, re-run on main 9df307ab on 2026-09-16:

    Measure Result
    Host diff comparable 18/23
    Comparable, 0 rows, no named limit, ≥1 relevant changed file unread (silent) 9/18
    Incomparable results naming the blocking source 0/5
    Relevant changed files covered 15/120 (partial 7, not covered 86, unclear 12)
    Automatic author-actionable items 0/23 (one-sided 95% upper bound 12.2%)

    Caveats: this is one host-heavy sample, and the file denominators were reconstructed by hand. The run executed no repository code.

    Milestones 10–12 as originally scoped target reviewer comprehension in consented pilots: #811 keeps novel findings outside its success gate. They do not target automatic findings. On this sample they would leave automatic author-actionable items at about 0/23. The issues below add row precision, coverage honesty and bounded readers, followed by a value gate.

    New issues

    Phase Issue
    1 — milestone 10 #816 :* narrowing reported as widening; moved rule blocks pairing; single MCP tool treated as wildcard (P1)
    #817 diff traceback in a partial clone missing base blobs (P2)
    #818 relative --out resolution differs across commands and is undocumented (P2)
    2 — milestone 11 #819 hook matcher/command/timeout and MCP args as engine fields; #795 slice 2 (P1)
    #820 every hook/MCP/plugin/mode/sandbox edit labelled a widening (P2)
    #821 name unread candidate inputs so a zero-row result is never read as covered; #812 slice 2 (P1)
    #822 unchanged instruction limit through an in-tree link refuses the whole comparison (P2)
    #823 agent launches in CI workflows unread (P1)
    #824 exec-equivalent Bash allow rules rated like scoped rules (P2)
    #825 unpinned MCP launch source note (P2, after #819)
    #826 inline unconditional allow hook note (P2, after #819)
    #827 check and diff disagree on prompt-disabling settings (P2)
    #828 decision: agent-launch argv in repository scripts; default is keep excluded (P2)
    #829 residual reach of a prefix allow narrowed only by specific denies (P2, conditional)
    3 — milestone 12 #830 value gate before any outreach (P1)

    Other changes:

    Revised engineering order

    Phase 1

    1. Token-shaped workflow job names are published raw in host grants #802 (PR Redact token-shaped workflow job ids and step labels in host grants (#802) #815 in review)
    2. check crashes with a ValidationError when settings only add enabledPlugins #810 (PR Keep check coherent when settings add keys outside the host-boundary rule allow-list (#810) #814 in review)
    3. Host diff misreads permission direction: :* narrowing reported as widening, a moved rule blocks pairing, a single MCP tool treated as wildcard #816
    4. verify --out set to a tracked directory hides that directory's changes from the decision and currency check #804 → workspace_unverifiable does not say why the worktree could not be read #813 → Relative --out resolves differently across commands: verify against the Git root (undocumented), audit --host against cwd; printed artifact paths do not resolve from the caller #818
    5. diff exits 1 with an uncaught ConfigError traceback in a partial clone missing base blobs #817
    6. Show concrete field changes so an unfamiliar reviewer can act on the first advisory result #795 slice 1
    7. Show what each host comparison checked, partially checked and could not establish #812 slice 1
    8. Make existing discovery copy explain the host-review task without restoring the legacy skill #792
    9. Next release (proposed 1.1.0). The owner reviews it before publishing.
    10. A published 1.0.0 adoption kit installs 0.15.0 and falsely stops at the contract floor #781 (release-time check)

    Phase 2

    1. Publish hook matcher, command summary and timeout, and MCP launch arguments, as bounded engine fields (#795 slice 2) #819
    2. Hook, MCP server, plugin, permission-mode and sandbox edits are labelled widenings even when they narrow or disable #820
    3. Name relevant inputs the host entry does not read, so a zero-row result is never read as covered (#812 slice 2) #821
    4. Preserve independently established host changes when a plugin scope is incomparable #808 with An unchanged instruction limit reached through an in-tree link refuses the whole comparison instead of being named unchanged #822
    5. An enabled in-repo plugin's hook at a non-registry path gets check allow #809
    6. check and diff disagree on prompt-disabling settings: enableAllProjectMcpServers as CONFIG-PARSE-FAILED, dontAsk/bypassPermissions medium vs blocking wildcard, setting names missing #827
    7. Rate exec-equivalent Bash allow rules (interpreter -c/-e, npx, uv run, docker exec, xargs, env) as reaching arbitrary code #824
    8. Read agent launches in CI workflows: known agent action inputs, literal agent CLI run steps and head-ref checkouts #823
    9. Note an unpinned MCP launch source (@latest, no version, git without ref, image without digest) after launch arguments are published #825
    10. Note an inline hook that unconditionally returns permissionDecision allow for a broad matcher #826
    11. verify --preview in a configured repo publishes human_review_required: input directory capture is unavailable #807
    12. A hook command that runs an in-repo script: editing the script changes what runs, with no row #702
    13. [Feature]: Make deterministic permission review tangible through a change–correction–recurrence loop #811
    14. Add an exercised host-only advisory PR recipe with no manifest or saved baseline #780/Complete the published v1.0.0 tag/Action replay after the ordinary CI pin repair #570. The hosted run needs owner approval.
    15. Decide: read agent-launch arguments in repository scripts, or keep them a named unread surface #828 decision
    16. Conditional: show the residual reach of a prefix allow narrowed only by specific deny rules #829 (conditional)
    17. Next release (proposed 1.2.0)

    Phase 3

    1. Offer optional workflow retention after first value; repair the existing pre-commit example pin #796
    2. Value gate before any outreach: measure automatic author-actionable value on a released build #830 value gate
    3. Then, only if Value gate before any outreach: measure automatic author-actionable value on a released build #830 passes, Run the published advisory host-diff pilot: first reviewer value and the next eligible change #571/Design-partner conversations: two waves of five and at least three real host diffs #653 outreach, limited to individually human-written notes that the owner approves before sending

    Issues in these milestones that are not listed keep their current milestone and priority. Checkpoint dates stay evidence-review dates. At 2026-10-14, #653 records a dated shortfall rather than extending its window.

    Rules kept

  9. pengfei-threemoonslab commented on Sep 24, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Days 1–5 execution is selected and recorded in PR #869. Application-agent proof (#868, milestone 13) is primary; host correctness is bounded maintenance. Accountable product owner: @pengfei-threemoonslab; technical executor: Codex current task. #580/#655 design precedes #867 and the reproduced #864/#865/#866 readers. #610 is reproduced; #787 is the bounded recipe repair. Milestones 9–11 were reconciled, retaining Oct 14 / Nov 13 / Dec 13 as evidence checkpoints. #830 and external outreach holds remain unchanged. Older recruitment-first and hosted-deferred text is superseded by this dated selection; no external value or ten-case result is claimed.

  10. pengfei-threemoonslab commented on Sep 24, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Days 1–5 validation closeout

    The two coding-agent review/address rounds are recorded in this PR. Full hosted CI passed on the published head; the release-tag-only check is intentionally skipped on PRs. The PR is ready for maintainer review.

    Local validation: lint passes; the initial sandboxed wheel-build setup needed network. A separate fixture failure was traced to the existing global Git ignore rule for .claude/settings.local.json; all 41 tests in that file pass when only core.excludesFile is isolated to /dev/null for that process. No user Git setting was changed and no local full-suite pass is claimed.

    #869 records the baseline and design; #870 delivers #787's bounded repair with 14 focused tests and a current, complete verifier result. #610 remains a reproduced contract defect; #580/#655/#867 implementation and the ten-case proof remain in the next phase. Selected issues and milestones have been reconciled without closing those outstanding acceptance criteria.

  11. pengfei-threemoonslab commented on Oct 7, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Re-parented from #791: #800 (MCP tool descriptions), #696 (scorer result recognition), #799 (discovery measurement, deferred until funded), #793/#794/#797 (deferred until #830 passes).

  12. added
    P2Queued; valuable but not blocking
    and removed
    P1Picked up inside the current milestone window; keep about 15 open
    on Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

P2Queued; valuable but not blockingarea:releaseRelease pipeline, packaging, and safety qualificationepicTracking issue coordinating a group of related issuesworkstream:adoptionReviewer journey, distribution, workflow delivery and observed reuse.

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions