Skip to content

Verifier: bind absent and relocated scoped base inputs without inventing an empty agent surface #580

Closed

Description

@pengfei-threemoonslab

Selected execution — 2026-09-24

Days 1–5 design is in PR #869, docs/research/application-days1-5/comparison-design.md. Bind independent base/head scopes and exact compared trees; distinguish Git-proven path absence, unconfigured existing app, relocation and unread input. A rename alone is not agent identity. Preserve reviewed-subject requirements for verifier/qualification; generated advisory extraction cannot satisfy them. Implementation and the fixed #1975/#1977 acceptance remain open for Weeks 2–3. Selected milestone 13.


The earlier advisory-release deferral below is historical. This issue is now selected for the application proof milestone; its implementation is not yet complete.

First advisory release scope — 2026-09-11

Owner decision: adoption > completeness; make v1.0 ready with repository-independent work, then expand adoption. Historical scoped tool-source inputs are deferred while the host workflow is made ready. The partial red test is not an implementation or a passing result.

Removed from the selected v1.0 milestone; kept open and deferred, not resolved. #643/#644/#645/#646 define current product acceptance; #572 preserves the historical qualified no-go and #648 owns the actual publication contract. The prior evidence and acceptance below remain a record of the deferred work, not newly satisfied conditions.


Problem

Found while implementing #564. The miner's missing-base-directory return hides an engine input-contract gap: a committed verifier comparison uses one repository-relative config_relative for both trees. A real PR-added manifest can already produce a terminal result with base_status=missing_manifest, but that does not prove an evaluated empty agent surface. A renamed scoped project has no separately bound base-manifest path.

Evidence

At main ded4be37f, cli/verify/orchestrator.py materializes the base tree and reads base_tree_dir / config_relative; absence returns missing_manifest. --diff-from can consume a separately produced old-scope report, but marks diff_from_provided rather than an independently scanned base and lacks the normal base subject/config join. The qualification scorer requires a successful, tree-bound base run. Neither workaround may be represented as that evidence.

The unchanged fixed public history supplies both concrete cases:

Those observations establish paths and file identities, not reviewed business authority or whole-agent identity. #564 will preserve them in a typed unsupported-input obligation instead of copying head sources into base or fabricating a verifier catch.

Required design and implementation

Define how the existing verifier plan/receipt binds independent base and head scoped inputs. A renamed scope needs the real base source path, head path and their evidence; an absent path needs an explicit, reviewed comparison subject with a precise absence claim. Absence of a manifest must never by itself mean absence of tools or runtime authority. Keep genuine PR-added manifests and all trust-root changes visible.

Prefer a bounded extension of existing manifest/diff/verification identities rather than a parallel verdict engine. State how this affects base status, capability coverage, qualification eligibility and report compatibility before implementing. Preserve fail-closed behavior until the input can actually be supported.

Acceptance

  • Real PR-added manifests continue through the existing terminal verifier route with their trust-root delta intact.
  • Renamed scopes bind original base/head trees and their actual input locations; partial/ambiguous renames cannot silently choose an agent identity.
  • A new scope distinguishes a Git-proven absent path from an unconfigured existing surface and an unreadable tree; no head inventory is manufactured at base.
  • Plan, receipt, report and qualification consumers agree on the new subject/status semantics without weakening successful-base or policy requirements.
  • Fixed #1975/#1977 refs and new/rename/unrelated-deletion fixtures demonstrate truthful evidence or precise refusal.
  • Cold-start and reviewed-scope measurements remain separate; no relabeling, invented effect/authority/binding declaration or scan fallback is used as a successful PR catch.

Sequencing

Split from #564 as its existing implementation direction requires; defer this newly discovered contract work under the user's instruction. Coordinate #563's reviewed scope, #559's exact-byte scorer, #569's compatibility freeze and #572's release decision. A typed miner obligation does not satisfy the historical catch bars or qualify v1.0. No milestone or release eligibility is granted by filing this issue.

Activity

  1. added
    P1Picked up inside the current milestone window; keep about 15 open
    area:identityVerification identity, receipts, reproducibility
    area:benchmarkLabeled corpus, accuracy measurement
    on Sep 8, 2026
  2. pengfei-threemoonslab commented on Sep 10, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Selecting this existing P1 after #596 for the core input-contract work needed by #563 and #515. The current source still has one manifest location for both trees; a synthetic relocated-project CLI regression now exits 2 because --base-config is unsupported. This does not change #515's outstanding default-diff requirement.

    Implementation contract before production edits:

    • Add an explicit Git-root-relative verify --base-config alongside head --config, requiring an explicit --base. Omission keeps the same-path behavior. Do not infer project/agent identity from Git rename similarity, and reject incompatible preview/no-base/supplied-report combinations.
    • Bind the selected base manifest location and its observed Git tree/blob or precise absence in existing plan input options. The head manifest remains the existing config blob. Existing plan/request/receipt identities bind this comparison; worker validation must reconfirm it. Rerun commands retain the original base path.
    • Preserve existing base_status: only an actual successful base scan is succeeded. missing_manifest additionally distinguishes an absent project directory from an existing directory without the requested manifest; unreadable/unsafe inputs remain failures. These path observations assert no empty tool inventory, deployment identity or business authority.
    • Keep base effective-policy comparison and all actual trust-root edits visible, including a relocation that also weakens CI. Neither a missing manifest nor a new CLI flag clears capability/dependency coverage gaps.
    • No automatic historical manifest overlay is introduced. Fixed ADK #1975/#1977 source trees without committed reviewed manifests receive precise evidence/refusal; they are not successful qualification catches. The exact-byte scorer retains its successful-base, policy and provenance requirements. Cold-start and reviewed-scope evidence remain separate.
    • Use existing JSON/receipt surfaces, add paired new/rename/unrelated-deletion and unsafe-path fixtures, and document compatibility. verification prepare/worker remain decision-free; they may bind/validate a subject but cannot manufacture a successful base scan.

    #640 separately retains the cal-1 entrypoint/extension-binding uncertainty; path binding here cannot settle it. No labels or declarations are changed to make a historical case pass.

  3. pengfei-threemoonslab commented on Sep 24, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Days 1–5 design is in PR #869, docs/research/application-days1-5/comparison-design.md. Bind independent base/head scopes and exact compared trees; distinguish Git-proven path absence, unconfigured existing app, relocation and unread input. A rename alone is not agent identity. Preserve reviewed-subject requirements for verifier/qualification; generated advisory extraction cannot satisfy them. Implementation and the fixed #1975/#1977 acceptance remain open for Weeks 2–3. Selected milestone 13.

  4. added 2 commits that reference this issue on Sep 27, 2026
    523725f
    36c061d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

P1Picked up inside the current milestone window; keep about 15 openarea:benchmarkLabeled corpus, accuracy measurementarea:identityVerification identity, receipts, reproducibility

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions