Skip to content

harness: add bounded VATS/GVO runtime and benchmark adapters - #488

Draft
amy-why-3459 wants to merge 9 commits into
mainfrom
feat/specrsi-offline-search
Draft

amy-why-3459 wants to merge 9 commits into
mainfrom
feat/specrsi-offline-search

Conversation

@amy-why-3459

@amy-why-3459 amy-why-3459 commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Problem and resulting behavior

The serial lifecycle cannot edit incomplete/failed candidates, select bounded verification bundles, or train search roles from independent search returns. This PR adds a synchronous VATS/GVO runtime: retain real edits, verify selected candidates, atomically settle evidence and costs, update separate CPU Drafter/Policy/Scorer parameters, and search again with the new frozen checkpoint. A repaired child can win independently of its parent's quality outcome, with its own complete evidence.

Refs #363

This is an experimental local implementation toward the full proposal. The support and remaining-acceptance matrix identifies its boundaries. The full RFC is not claimed complete; the PR remains draft.

Implementation

  • Immutable candidate/evidence trees and exact bounded completion planning, with an independent exhaustive oracle and finite-objective overflow rejection.
  • Synchronous DRAFT/VERIFY/ACCEPT blocks, Scorer refinement independent of Policy selection, frozen roles, retained block cursors, complete-candidate fallback, and atomic dependency invalidation/incumbent changes.
  • WorkControl transactions in the existing shared RunStore. Claims reserve elapsed-time caps and domains before launch; settlement commits evidence and accounting together. Unknown work/cleanup retains reservations and known usage lower bounds; known overruns block new work immediately, even before cleanup. Recovery reconciles conclusively unstarted expired work without dispatch, and never blindly relaunches uncertain execution. Subprocess execution uses the existing executor.
  • Actual finite-edit softmax Drafter and separate linear Policy/Scorer updates: independent groups with identical initial evidence/history, leave-one-out advantage, recorded behavior probabilities, PPO-style clipping/KL, actual paid suffix targets, metered forced continuations, checkpoints and Reset/Carry recovery. This is a constrained CPU operator model, not LLM finetuning. Apps injects role operations; Harness does not import Model.
  • Engine evaluation uses the packaged full-task protocol through the shared isolated protocol process and passes its remaining absolute deadline. Skill uses an explicitly configured comparator, existing private Git-index materializer, independent grader and guarded provider. Quality precedes token use; unknown usage remains INCONCLUSIVE. Frozen resources are checked before and after execution.
  • One-time final-scope claims before holdout access, immutable results, receipt-only recovery/export and task promotion only after accepted final evidence. Skill scope is tied to the frozen data digest. Role checkpoints are not automatically promoted to a default improver.
  • inferrsi search engine/status/export and Python composition. Benchmark is loaded only by the explicit Engine entry; ordinary project CLI/session behavior keeps its existing dependency boundary.

Validation

The software regression results below were recorded at earlier revision dc7292a3cd3d624794ed280a0d677d20e03fff29, integrating main cf671a880ac8037e01922e1be0b701771fb9922f. The current PR head is b84989494dfa69628aa918ed85653afc936a5493; this description update does not claim a fresh full regression run at that head.

  • Earlier-revision local regression: 216 passed, 54 subtests passed, covering WorkControl/VATS/GVO, planner/tree, role checkpoints, Engine/Skill adapters, isolated task protocols, architecture boundaries and ordinary project lifecycle. ruff check src tests tools and staged diff checks passed.
  • New regressions reproduce claim-to-launch expiry for both draft and verification, recover through a reopened store, preserve the qualified-root fallback, and distinguish uncertain launch intent. Known-overrun/unknown-cleanup coverage checks second-domain denial, retained domain ownership, monotonic usage holds and idempotent reconciliation.
  • Added review coverage for cross-candidate prerequisites with one completion seed; starting-evidence identity versus receiving IDs; explicit rejection of legacy records without initial evidence; injected role/JSON checkpoint round trips; invalid deadlines; and one total Engine deadline across arguments/execution/assessment.
  • Full regression for that earlier revision: Python 3.10 and Python 3.11 each report 5,288 passed, 37 skipped, 672 subtests passed, 1 warning. Both raw logs identify merge checkout e371b3b7fd663b8e84bb3f8ce0ebe982a38dc857, merging that revision into the stated base, and show Ruff passing. Installed-package checks for both Python versions, Windows/macOS compatibility, lint/smoke and reference checks also succeeded; dashboard-browser is skipped for the draft.
  • Repository qwen21-denoise-npu benchmark was previously run once in CPU smoke / check-only mode with a mathematically equivalent actual edit. Baseline and candidate correctness and weight-update checks passed; speedup is null. This is correctness evidence, not NPU performance or full-task qualification. Subsequent NPU diagnostic measurements are reported separately below.

The process-backed numeric grader and Skill report fixtures are software controls, not production pilot evidence. No live provider/model API experiment or production pilot is claimed; the NPU diagnostic runs below have a narrower scope. Earlier Harness-to-Model imports, eager Benchmark loading and the changed main-branch protocol API were fixed before this revision; their original failing runs are not counted as passing validation.

NPU diagnostic results (2026-11-19)

Scope under #363: one earlier serial reference and three additional VATS searches, with up to three candidates per VERIFY group. These are descriptive measurements; general search efficacy remains INCONCLUSIVE. The earlier single VATS pilot is excluded from the three-run statistics.

  • Benchmark: qwen21-denoise-npu, a scaled Qwen-Image 2.1 denoiser with random BF16 weights and full prefill plus eight denoising steps in the measured scenarios. All five cases run: compact/portrait/landscape contribute timing; padded-batch/smoke provide additional validation. This is not pretrained image-quality or production service evidence.
  • Environment and sampling: one Ascend 910B1, HF32 disabled, PyTorch 2.10.0+cpu and torch_npu 2.10.0.post7; pairs=2, samples=2, warmups=1. Speedup is the geometric mean of the three per-case baseline-median/candidate-median ratios, relative to the original denoiser, not a PR-versus-base speedup.
  • Revisions and controller: serial uses base cf671a880ac8037e01922e1be0b701771fb9922f; VATS uses PR head b84989494dfa69628aa918ed85653afc936a5493 plus an experiment-local TargetSearch wrapper. The per-candidate target-stop wrapper is outside the PR source, so these results are not a measurement of the unmodified PR head alone.
  • Search budget: the same fixed catalog of ten cumulative optimization candidates; ten is a maximum, excluding the root and final evaluations. Serial submits candidates in catalog order. VATS generates ten candidates and plans VERIFY groups of up to three, executing candidates synchronously on one worker. After each candidate completes and its receipt is settled, reaching 1.35× skips the group's unstarted jobs and triggers one independent final evaluation. A final target miss is retained without restarting search. Roles are hand-authored; no GVO training or LLM candidate generation was run.
Run Candidates verified First search hit ≥1.35× (s) Total including final (s) Selected optimization Search speedup Independent final speedup Final ≥1.35×
Serial reference (earlier run) 7 — 286.01 07_batched_conditioning 1.3513× 1.3705× Yes
VATS run-01 4 166.79 200.37 08_gate_up_fusion 1.3543× 1.3442× No
VATS run-02 2 99.83 133.41 07_batched_conditioning 1.3558× 1.3738× Yes
VATS run-03 1 63.62 96.26 07_batched_conditioning 1.3641× 1.3361× No

The serial first-hit time was not separately instrumented. VATS first-hit timing starts at controller initialization and includes root qualification and search; total timing additionally includes independent final and reporting. Candidate preparation, device queueing, container startup and environment preflight are excluded.

Search reached the target in 3/3 VATS repeats; independent final reproduced it in only 1/3. Mean first observed search-hit time was 110.08 s (median 99.83 s; range 63.62–166.79 s). Mean total time was 143.35 s. First observed search-hit time must not be described as time to a reliably confirmed target. All three outcomes, including both final target misses, are retained.

Actual VATS verification order and early stopping:

  • run-01: 10_kv_buffers → 04_shared_modulation → 05_decode_broadcast → 08_gate_up_fusion; two remaining jobs in the active group skipped.
  • run-02: 04_shared_modulation → 07_batched_conditioning; one remaining job skipped.
  • run-03: 07_batched_conditioning; two remaining jobs skipped.

The serial reference completed nine full evaluations / 486 compute calls; the three VATS repeats completed thirteen / 702. All passed the task's numerical correctness checks, including per-step and weight-update checks. Frozen-file integrity checks passed; skipped jobs created no Attempt/Receipt. Numerical correctness does not imply meeting the 1.35× performance target.

Comparison limits: the serial reference was measured earlier, without a contemporaneous paired serial control for these repeats. The three VATS runs used fresh containers/state on the same exclusively allocated NPU, while the CPU host was shared. UUID-dependent tie ordering can vary the verification order despite a fixed seed. The lower observed wall times accompany fewer candidate validations; this small, fixed-catalog experiment does not establish a stable causal gain from the PR, parallel execution, trained search roles, or reproduction of the paper's results.

Source summaries retained outside the repository: pr488-qwen-cap10-ab-20261119/results/comparison.json and pr488-qwen-bundle3-repeat-20261119/summary.json. Raw experiment bundles are not publicly attached to this PR; the table is a summary of those retained results, not a publicly replayable acceptance package.

Remaining full-issue requirements

The supported profile is a trusted local host, one worker and small menus. Domain exclusion covers cooperating controllers sharing the same RunStore; a name does not allocate a device or provide OS confinement. Runtime adapters package setup into whole checks; arbitrary standalone shared-setup jobs are represented only by the offline planner.

Production resource qualification, hard preemption of in-process callbacks, preflight/setup lifecycle accounting, campaign-wide provider token/currency reservations and multi-worker comparisons remain outstanding. Existing per-comparison provider bounds do not establish a campaign-wide token/currency budget. The supplied operator model provides neither an LLM training adapter nor supervised warmup data.

The NPU runs above exercise Engine search and independent final with a fixed candidate catalog. Full Engine GVO and live Agent/Skill pilots, and the complete AC-10 matched serial/VATS, bundle, learning, Reset/Carry and worker comparisons remain outstanding. A controlled live Agent/provider configuration and budget have not been supplied. Promoting a learned checkpoint as a default improver also requires separate independent qualification.

No parent-issue closure, production acceptance, efficacy gain or merge approval is claimed.

Separate editable lineage from candidate-specific verification evidence. Retain
scoped attempts and dependency epochs, select complete development candidates,
and enumerate bounded completion plans with shared preparation and cost caps.

Cover failed-parent repair, stale evidence, deterministic selection and planner
feasibility with CPU controls; document the shared execution integration boundary.

Agent-assisted-by: Codex (implementation, tests and documentation)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent non-voting review: not approved; acceptance and execution evidence are incomplete.

Reviewed head 853c92f35f3b4d3df841958ba33568f690f86c5d against base 8aba657da79e8ed9f6e6fbb33437e9d9fdd8f1c7, on the other-author draft PR. I inspected both new executable modules, the synthetic fixture and both new test files, plus machine CI evidence. AGENTS and repository review guidance were checked. No other reviews, PR result narrative or mixed Issue discussion informed this assessment.

Code facts established by static inspection

  • Search evidence is bound to the complete registered Attempt, including candidate/artifact, context, check, execution key, reservation and lease identity. Receipt registration rejects changed bindings/conflicting duplicate receipts; evidence selects the latest attempt and excludes invalidated dependencies and INCONCLUSIVE outcomes. Acceptance requires every required check to pass, including the objective, and preserves the qualified root on a score tie.
  • Candidate edit ancestry is distinct from generation/evidence dependencies: a fully checked repaired child can be selected despite a failed or incomplete edit parent, while inherited dependency invalidation removes affected descendants from admissible selection. This is a substantive semantic distinction from treating every parent failure as automatic descendant failure. It should not be assumed to settle the older bounded-successor RFC's acceptance contract.
  • Completion planning constructs prerequisite closures, counts shared jobs once, applies cost/job/attempt bounds and conflicts, rejects outstanding unsettled work, and returns search identity/state revision without spending the supplied budget. A plan is not an execution reservation or proof that its costs were incurred.
  • The inspected tests cover wrong receipt bindings, late results after invalidation, same-byte distinct candidates, immutable identities, fresh retries after INCONCLUSIVE, bounded fallback, and a three-candidate exhaustive planning oracle. The engine/skill-labelled fixture uses retained files containing synthetic numbers; it is not two real adapter integrations, independent holdout qualification or a performance experiment.

I did not identify a confirmed actionable defect in this bounded static pass. One useful additional coverage case is cross-candidate prerequisite closure: selecting one completion seed can incidentally complete another candidate. The current oracle fixture shares setup but does not exercise that coupling. Keep the seed-enumeration bound distinct from a bound on all incidentally completed candidates; this is a coverage/contract clarification, not a demonstrated runtime defect.

Important limits: SearchState revision checks operate on the supplied in-memory state. These modules do not themselves implement durable compare-and-swap, crash recovery, actual reservation/lease authorization, cancellation, process cleanup, holdout access control or atomic budget accounting. Nonempty reservation/lease identifiers are identity bindings, not proof those resources were reserved. The PR's isolated scope is offline candidate trees and verification planning; absence of a runner here is not by itself a code defect, but these helpers must not be presented as completion of those wider operational guarantees.

Evidence actually inspected

  • lint-and-smoke 134586515319: exact head appears in the raw log; Ruff passes and 8 passed in 0.05s. Those smoke tests do not establish the new search-tree/planning tests.
  • installed-package 3.10 134586617451: raw successful sdist/wheel build at synthetic merge b92eff795921472b952a4e8e0401cffce4a866e0 of this head into the stated base.
  • At inspection, full-check 3.10/3.11 and Windows were still running, macOS queued; browser was skipped. Installed-package 3.11, references and declarations had success metadata. Completed full-check logs were not available to verify the new tests.

No repository tests or code were executed by me. The detailed current AC remains unavailable in a safely isolated form; the earlier #363 requirements are not automatically the full edited VATS/GVO contract. Changed documentation was not used as acceptance authority or as proof of implementation. This COMMENT records a limited independent code/evidence review, not whole-RFC acceptance, workflow efficacy, permission for pilots, code changes or merge.

@david6666666 david6666666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One numeric boundary to address in the offline search helpers.

def verified_gain(state):
selected = accept(state)
check = state.contract.objective_check
gain = (evidence(state, selected.candidate_id)[check].objective

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both receipts can have finite objectives while their difference overflows. With a root of -1e308 and a selected candidate of 1e308, verified_gain() returns inf, which downstream strict JSON cannot serialize. Could we reject a non-finite gain explicitly (or bound the objective domain) and add a regression for both objective directions?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid for the original implementation; addressed in abe72f6 and retained in the current PR head dc7292a. verified_gain() explicitly rejects a non-finite subtraction before the value can reach strict JSON. The regression covers finite -1e308/1e308 objectives in both maximize and minimize directions.

Rechecked on the pushed revision: test_speculative_search.py passes as part of the local affected-area suite (216 passed, 54 subtests passed); Ruff also passes. The current-head full CI is running; these local results are not a full-suite or campaign-acceptance claim.

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent non-voting CI follow-up to review 5951024572: completed regression evidence, but not whole-scope approval.

Head 853c92f35f3b4d3df841958ba33568f690f86c5d, base 8aba657da79e8ed9f6e6fbb33437e9d9fdd8f1c7. I independently retrieved the newly completed machine logs rather than treating success metadata or author statements as test evidence.

Both full-check logs in Ready run 44240402818 show checkout b92eff795921472b952a4e8e0401cffce4a866e0, explicitly merging this exact head into this base:

These invoke pytest -rs across the configured tests directory. I rechecked the current test_speculative_search.py and test_verification_plans.py: 19 and 10 test functions before parameter expansion, with no skip decorators found; neither appears in the complete skip summaries. This is aggregate execution evidence for the newly inspected offline tests, not an individually verbose PASS list or an independent local rerun. The 36 skips include optional dependencies, real Cannbot qualification and hardware/platform-specific cases; they are not 36 passes.

Raw platform logs at the same merge report Windows 8 passed + 25 passed, and macOS 8 passed. Their selected modules concern plugin/DSH and Windows portability/session behavior, not a full cross-platform execution of the new planner/tree tests. Dashboard-browser is skipped, not passed. Other Ready jobs are now successful by metadata. This resolves the pending CI evidence noted previously.

The isolated specification clarification confirms max_targets limits the union of completion seeds, not the number of candidates incidentally completed by prerequisite closure. Consequently I do not classify extra incidental completions as a cap violation; my prior cross-candidate prerequisite suggestion remains additional coverage only.

The offline code assessment is unchanged. Receipt identity/dependency bookkeeping and proposal generation do not establish durable CAS/recovery, authenticated reservations/leases, real cancellation, holdout isolation or actual Engine/Skill adapter efficacy. Detailed current RFC acceptance provenance remains incomplete; old bounded-successor requirements must not silently substitute for the edited VATS/GVO scope. No code/test execution, campaign restart, model/device/paid experiment, code change or merge was performed by me.

Source accounting: isolated metadata for inline4609449850@2026-11-18T05:09:44Z contains no direct owner mention or independent machine locator. Its inseparable technical narrative was not consumed or adjudicated. This follow-up is driven by the independently verified CI transition, not that source's opinion.

@Dong1017 Dong1017 added area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice labels Nov 18, 2026
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
@amy-why-3459 amy-why-3459 changed the title harness: add offline SpecRSI candidate trees and verification planning harness: add bounded VATS/GVO runtime and benchmark adapters Nov 18, 2026
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent incremental review — NOT APPROVED. COMMENT rather than a Request changes vote.

Range: previous reviewed head 853c92f → abe72f6 → current 5ee4181; base remains 8aba657. Previous own review: #488 (review) . This is a substantial expansion from offline primitives into persisted VATS execution, CPU GVO roles, continuation/final control and Engine/Skill adapters, not merely the earlier passing offline tests.

P2 — Persisted not-started expiry strands Search recovery. In harness/vats.py:391–421, _execute atomically writes search.pending (and registers a verification Attempt) with its claim, then calls WorkControl.launch. In work_control.py:171–188, expiry between claim and launch conclusively records settled/not_started and releases the domain, but leaves search.pending untouched. Since the adapter never ran, vats.py:356–364 cannot find its receipt and permanently reports Unknown execution; finish at 635–640 also rejects pending work. A scheduling pause or approaching absolute deadline can therefore prevent a search with a valid qualified root from completing normally even though execution is known not to have started. Reconcile the durable not_started terminal record with the pending Attempt/work-block state, without relaunching uncertain work. Add a Search-level claim→expiry→recover/limit-finish test; the existing WorkControl-only expiry test does not exercise this composition.

P2 — Known overspend is ignored while cleanup is unknown. work_control.py:213–217 takes the unknown branch whenever cleanup_verified is false, even when actual_ms is known and exceeds the reservation. available() at 103–105 subtracts only the old cap, and another domain can still claim. Static example: campaign cap 1000, final reserve 100, claimed cap 200, then settle(actual_ms=950, cleanup_verified=False). spent remains zero, blocked remains false and available reports 700, allowing another 700 commitment despite known spend already consuming the final reserve. Retain the domain/unknown-work restrictions, but conservatively account for the known expenditure lower bound or block on known overspend; reconcile later without double charging. Add the combined known-overrun + unknown-cleanup + second-domain claim regression. This is source-path reasoning, not a locally executed reproducer.

Other inspected changes / prior findings: the latest Engine post-execution frozen-resource recheck and expanded Skill controller identity are present; their new mutation test is a source assertion, not independently executed evidence here. GVO separates forced continuation scorer targets from actor groups, verifies retained training labels, and uses retained checkpoints for Reset/Carry recovery. I did not establish an additional concrete training-formula defect in this bounded pass. CPU update timing is measured after update returns; block-on-overrun is not preemptive termination. Native host/process cleanup is not filesystem isolation or a GPU scheduler reservation. The previous completion-seed clarification remains unchanged and is not reopened.

Evidence: exact-head lint-and-smoke job 134828062677 raw log identifies 5ee4181, Ruff success and 8 passed in 0.04s for test_smoke.py/test_claim.py. That does not execute the new VATS/GVO tests. Current full-check 3.10/3.11 and Windows are in progress, macOS queued, dashboard-browser skipped; package/reference/declaration checks have success metadata. Cancelled intermediate-head jobs and historical 5149-test results are not current-head passes.

Before approval, address the two control-state/accounting paths and demonstrate their regression tests plus applicable new-head full integration checks. Current Issue #363 normative sections were isolated from opinions: its proposed recovery/budget/final/dual-pilot requirements are not evidence of maintainer acceptance or authorization for real experiments. Synthetic CPU fixtures, finite-edit softmax roles and a comparator bridge do not establish both real Engine/Skill VATS/GVO/final pilots or performance gains. Whole-RFC acceptance provenance remains unresolved.

Performed: actual incremental code/test inspection across execution/accounting, adapters/CLI, roles/GVO/continuation/final; current AGENTS, isolated normative requirements, own-review deduplication and machine CI evidence. Not performed: repository-code or test execution, model/device/provider trials, paid experiments, edits, CI rerun or merge. No other reviewers' opinions were read or used. Optional follow-up: keep explicit tests distinguishing known not-started work, unknown launched work and known overspend with unknown cleanup, rather than testing these states only in isolation.

Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent incremental review: 5ee4181 → bd2bfc4; base remains 8aba657. Previous own review: #488 (review) . COMMENT, still not approved.

The four-file delta adds an initial_conditions digest to Search.start and requires equality across GVO examples. It includes candidate/provenance, dependency epochs, actual execution/reservation/lease identities and receipt evidence, excluding only receiving attempt_id/search_id. I cross-checked import_qualified_root: its receiving-scope rebinding changes those two excluded fields, so independent searches can still share the same qualified evidence without falsely differing only by their local IDs. The added assertion rejects a differing digest. The documentation clarifies use of Comparator.holdout_scope, not an executable final-gate change. No new confirmed source defect was identified in this narrow delta.

Neither prior P2 is resolved by this commit. work_control.py is the identical blob, and the relevant VATS claim/launch/recover paths are unchanged:

  1. A conclusively not-started expired launch still leaves search.pending (and, for verification, its Attempt) behind. Receipt-only recovery cannot obtain a receipt from an adapter that never ran; normal finish remains blocked. Coordinate the durable not_started outcome with search/work-block state and add the composed expiry/recovery regression.
  2. Known actual_ms above cap with unknown cleanup still takes the unknown branch without blocking or charging the known overrun; available() retains only the original reservation. Other domains can still claim against an overstated available budget. Preserve resource exclusion while conservatively accounting/blocking the known overspend, with an idempotent reconciliation regression.

These remain static code-path findings, not executed failures. The new initial-condition equality does not address either state/accounting path.

Current evidence: raw lint-and-smoke job 134831307114 identifies this revision, Ruff success and 8 passed in 0.07s for smoke/claim tests. At inspection the full-check, host and package jobs were still in progress/queued; dashboard-browser was skipped. No completed current-head new-runtime regression result was established. Prior-head aggregate passes are not relabeled as this head's results.

Before approval: resolve the two P2 paths, execute the targeted combined-state regressions and applicable full checks, and retain the previously stated distinction between CPU/synthetic controls, proposed Issue #363 acceptance and real Engine/Skill pilots. The earlier acceptance-provenance and real-pilot gaps are not cleared here.

Non-blocking coverage suggestions: construct same-root-byte groups with changed actual receipt evidence/epochs to test digest derivation, plus same evidence with different receiving IDs to test admissibility. For compatibility with previously persisted experimental searches, define an explicit version/migration policy: records predating initial_conditions cannot be consumed by direct dictionary indexing without that field; do not fabricate their historical starting evidence.

Performed: all four changed files, relevant root-import and unchanged failure paths, own-review/head checks and machine smoke evidence. Not performed: repository-code/test execution, real model/device/provider trials, edits, CI rerun or merge. No other reviewers' opinions were read or used.

Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent incremental review: bd2bfc4 → 96e3c50; unchanged base 8aba657. Previous own checkpoint: #488 (review) . COMMENT, not approval.

I inspected all seven changed files. The delta removes direct Model imports from the new Harness modules and dispatches through the injected snapshot operations. For the supplied CPU Snapshot, choose/predict wrap the same choose/dot functions; train reconstructs Choice with the same tuple conversions previously performed in gvo, then passes the same reference/actors/values/pairs/settings to update; restore constructs the checkpoint's Snapshot type. The Harness finite-return validation remains equivalent for recorded numeric observations. I found no new concrete parameter/serialization defect in this bounded refactor. The documented dependency direction is improved; this is not an executed integration result.

The two previous P2 findings remain unresolved. I rechecked the current failure paths rather than relying on discussion state:

  • work_control.py is still blob 3a0993c42845765146573e31d4bf1a1ce3e60628. Expired launch persists settled/not_started without reconciling VATS pending/Attempt state; recover still requires an adapter receipt that cannot exist for unstarted work, and finish still rejects pending work.
  • The unknown-cleanup branch still ignores a known actual_ms overrun for blocking/available-budget purposes, while available subtracts only the old reservation. The previously described 1000-cap/100-final-reserve/200-claim/950-known-use sequence can still expose 700 available to another domain. Resource exclusion alone does not make this budget view conservative.

Before approval, fix those two composed-state/accounting paths and supply their targeted regression execution plus the applicable current-head complete checks. This dependency refactor does not resolve them. Earlier initial-evidence consistency and frozen-resource improvements remain present; no formerly resolved item is reopened.

Machine evidence inspected: exact-head lint-and-smoke job 134833551072 raw log identifies 96e3c50, Ruff success and 8 passed in 0.06s for tests/test_smoke.py and tests/test_claim.py. At inspection full checks, Windows/package/reference jobs were running, macOS queued and dashboard-browser skipped. This does not establish completion of the new search/GVO runtime tests. Historical-head results are not current-head passes.

Optional coverage: add direct injected-role boundary tests for the round-trip Choice document, checkpoint restore and same-input predict/train behavior, so future non-CPU role implementations cannot silently diverge at this seam.

Performed: seven-file static delta review, current original failure-path reread, own-review/head deduplication and raw CI evidence. Not performed: repository-code/test execution, real model/device/provider pilots, edits, CI reruns or merge. Prior proposed Issue #363 acceptance provenance and real-pilot boundaries remain unchanged. No other reviewers' opinions were read or used.

Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>

@david6666666 david6666666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two integration failures on the current Ready merge with main.

Comment thread src/inferrsi/apps/speculative_search.py Outdated
from inferrsi.harness.work_control import WorkControl
from inferrsi.model.search_roles import Snapshot

from .search_benchmarks import EngineEvaluator

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this import move into the engine command path? Loading the ordinary project CLI now reaches Benchmark through search_benchmarks and improvement, even when search is never used. The existing lifecycle isolation test fails on the Ready merge.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct for the reviewed revision. Fixed in bb6ecbd: EngineEvaluator is imported inside the explicit engine-command path, after the status/export branches. Ordinary project CLI loading no longer imports Benchmark through this module.

Rechecked at pushed head dc7292a: test_project_lifecycle_never_imports_benchmark, the project lifecycle suite and module-boundary checks pass within the local affected-area run (216 passed, 54 subtests passed). The current-head full CI is running.

Comment thread src/inferrsi/apps/search_benchmarks.py Outdated
raise ValueError("Frozen benchmark resources changed")
output.mkdir(parents=True)
candidates.retain(candidate / "candidate.py", output / "candidate.py")
protocol = task_protocol.protocol(self.frozen)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Ready merge with current main no longer has task_protocol.protocol(), so this adapter raises before the resource-mutation guard is exercised. Could this use main’s packaged child-task API and rerun that regression on the merge tree?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct for the reviewed merge. Fixed in bb6ecbd after integrating main cf671a8: the adapter uses task_protocol.run() for arguments/assessment in isolated child processes, with the remaining campaign deadline.

Rechecked at pushed head dc7292a: both frozen-resource mutation regressions and the protocol-child tests pass. This revision also adds invalid-deadline rejection and an Engine arguments → execution → assessment test showing remaining time decreases under the original deadline. The affected-area run reports 216 passed, 54 subtests passed, with Ruff passing; full CI for this head is running. These are software-control checks, not an NPU performance qualification.

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent incremental review: 96e3c50 → bb6ecbd. Base advances from 8aba657 to cf671a8. Previous own review: #488 (review) . COMMENT, not approval.

I compared actual old/new contents of all 30 current PR paths and separately checked the base-to-base changes. The inherited R3 protocol-child and Apps/storage/dashboard changes are not being counted as entirely new search implementation. VATS, WorkControl, GVO/roles, adapters other than Engine, planning/tree and the existing VATS/work-control regression files are byte-identical to the previous reviewed head.

The substantive integration changes are appropriate in this bounded static pass: Engine now invokes arguments/assess through task_protocol.run rather than the removed in-process protocol API; both protocol calls and benchmark execution derive remaining time from the same supplied deadline. The optional keyword-only monotonic deadline in task_protocol.run is finite-validated, rejects already expired work, and is combined with the existing protocol timeout; _supervise also checks expiry before starting. The new tests cover a short deadline and already expired no-launch case. EngineEvaluator is imported only for the engine command, preserving the lightweight status/export path. I found no additional concrete source defect in these changes; source assertions are not test execution.

Both existing P2 blockers remain. WorkControl and the relevant VATS state transitions are unchanged:

  1. A conclusively not-started expired launch leaves search.pending/verification Attempt unreconciled; recover seeks a receipt from an adapter never executed, and finish refuses pending state. The new protocol-child timeout tests do not exercise this separate claim→launch→Search-recovery composition.
  2. Known actual_ms overspend with cleanup_verified=False still leaves only the old reservation deducted and does not block further different-domain claims. The shared budget can therefore remain overstated despite known expenditure. Deadline propagation does not correct that accounting branch.

Before approval, resolve these paths and demonstrate the combined-state regressions plus applicable rebased integration checks. The earlier qualified-root evidence and frozen-resource improvements remain present; no resolved item is reopened. Proposed Issue #363 acceptance provenance and real Engine/Skill pilot qualifications remain separate from this code integration.

Current machine evidence: raw lint-and-smoke job 134843558515 identifies bb6ecbd, Ruff success and 8 passed in 0.06s for smoke/claim tests. At inspection full-check 3.10/3.11 and Windows were running, macOS queued; installed-package 3.10/3.11, references and declarations had success metadata, dashboard-browser was skipped. I did not establish completed new-head full runtime verification or relabel historical-head test results.

Optional coverage: test invalid deadline inputs, omitted-deadline compatibility, and an Engine arguments→execution→assess scenario showing that the original total deadline is not reset.

Performed: old/new content and base separation, actual Engine/protocol/CLI/test inspection, unchanged P2 paths, AGENTS and own-review/current-head checks, raw smoke evidence. Not performed: repository-code/test execution, real model/device/provider trials, edits, CI reruns or merge. No other reviewers' opinions were read or used.

Recover conclusively unstarted work without adapter execution, keep unknown launches blocked, and reserve known usage until cleanup is reconciled. Reject legacy searches without retained initial evidence and cover the reviewed planner, role, identity and deadline boundaries.

Agent-assisted-by: Codex (implementation)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

增量独立检视:bb6ecbd5a6888f43345ba5376f71f2c6e55f0c25 → dc7292a。
上轮:#488 (review)

结论:本轮两个已报 P2 的源码修复成立,新增回归定义覆盖原失败路径;目前仍为 COMMENT,不把尚未完成的验证或 proposed RFC 验收计为通过。

已解决(源码层面):

  1. 过期未启动导致 Search pending 无法恢复:recover 现在只对持久 settled + not_started/deadline 的明确证据清理 pending/active_action;verify 对原绑定 Attempt 写 control-only INCONCLUSIVE,无 objective/PASS,不调用 adapter 或再次 dispatch。未知 launch intent 仍须真实 adapter receipt,不能走此例外。新增 draft/verify 两类恢复测试包含重开 RunStore、重复恢复、零额外收费、根 incumbent 和 domain 释放,并有未知 intent 的负向用例。
  2. 已知超额但 cleanup 未知时预算虚增:known_ms 保留已知用量下界,available 按 max(cap_ms, known_ms) 保留占用;超过可执行预算立即 blocked,未知 cleanup 继续保留 domain。最终结算不得低于已知用量,最终收费及重复回执保持幂等。新增 200 cap / 950 known usage 用例覆盖跨 domain 拒绝、重开库、后续未知观测不抹除下界、拒绝倒退、一次收费及最终 domain 释放。

其他增量:

  • legacy initial_conditions 缺失改为明确拒绝 Search 恢复及 GVO examples,仍允许 WorkControl 原始读取;不从后来的树猜测历史初始证据。初始证据测试区分 receiving IDs 与真实 execution/evidence/epoch。
  • 阅读共享 Engine 总 deadline、无效 protocol deadline、注入角色序列化及 checkpoint、completion prerequisite 的新增测试定义。本轮静态阅读未发现新增确定缺陷。

实际检查:

  • 独立阅读本轮全部 11 个变更文件的差异,并复核 WorkControl/Search/GVO/record 编解码控制路径和仓库 AGENTS;未读他人检视意见。
  • 已读取当前 head 的 lint-and-smoke 原始日志:Ruff 通过;tests/test_smoke.py tests/test_claim.py 为 8 passed in 0.07s。证据:https://github.com/InferMatrixLab/InferRSI/actions/runs/44319975680/job/134856020871
  • 最近检查快照:full-check 3.10/3.11、Windows 仍进行中,macOS 排队;installed-package 3.10/3.11、references、declarations 为成功状态,dashboard-browser skipped。状态成功不等于我已检查其全部日志。

批准前必做 / 未执行:

  • 取得当前 head 对上述新增 WorkControl/VATS 等回归的实际执行结果及必要完整检查结果;smoke 的 8 项不覆盖这些新增用例。本轮没有在本地执行仓库代码或测试。
  • 保留上轮对 #363 proposed 范围、owner 验收与真实 Engine/Skill pilot 证据的区分;本轮新增单元/替身测试不能替代真实集成或性能验收。

非阻塞建议:可另补仅 operation deadline 过期、campaign deadline 尚未到达时的恢复后继续搜索用例,区别于本轮将 campaign 时钟整体推进到期的回归。
未合并、未修改分支、未启用自动合并。

@amy-why-3459

amy-why-3459 commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator Author

Review follow-up at pushed head dc7292a (base cf671a8).

The two P2 findings in review 5957370567, reiterated in 5957447009, 5957503359 and 5957834456, were valid. Regression tests reproduced both before the fix. This reply consolidates the repeated findings without treating them as separate defects.

Feedback Disposition and verification
Expired, conclusively unstarted launch strands Search Fixed in dc7292a. Search.recover() uses the durable settled/not_started record, clears pending work and the unfinished block, and records verification attempts as control-only INCONCLUSIVE without objective/evaluator PASS. Reopened-store tests cover both draft and verification expiry, idempotent recovery, no adapter recovery/dispatch, no execution charge and qualified-root limit completion. Separate tests preserve rejection of uncertain launch-intent recovery.
Known overrun ignored while cleanup is unknown Fixed in dc7292a. WorkControl blocks new claims immediately on observed overrun, retains the domain and the maximum of the reservation/known usage, and preserves that lower bound across unknown observations. The 1000/100/200/950 example now reports -50 available and rejects another-domain claims. Later cleanup settles usage exactly once; downward revision below known usage is rejected. Tests reopen the store and verify both resource exclusion and idempotent reconciliation.
Cross-candidate prerequisite coverage (initial review, clarification) Added the suggested regression: one completion seed can complete two candidates via prerequisite closure while shared setup is charged once. max_targets still bounds seeds, not incidental completions; no demonstrated planner cap defect was claimed or silently redefined.
Starting-evidence identity and legacy records (5957447009) Added derivation tests: receiving IDs alone are admissible, while changed actual evidence, execution identity or dependency epoch is rejected despite identical root bytes. Records missing initial_conditions remain readable/exportable, but resume/training now fail explicitly with a fresh-search requirement; no historical evidence is fabricated.
Injected-role seam (5957503359) Added choose/train parity, JSON Choice round-trip, checkpoint restore and prediction tests for the supplied CPU role implementation. This does not qualify an arbitrary external role adapter.
Deadline coverage (5957834456) Added rejection of nonfinite/non-numeric deadlines before child launch, and an Engine arguments → execution → assessment case with decreasing remaining time under one original deadline. Existing omitted-deadline protocol cases remain in the executed suite.

The three inline findings were also valid for their reviewed revisions and had already been fixed; they were rechecked and answered in their original threads: finite gain, CLI import isolation, main's isolated protocol API. I found no substantive finding that needed rejection as incorrect.

The subsequent review of dc7292a confirms the two source fixes and identifies no new concrete defect. Its additional operation-only-expiry/continued-search suggestion is reasonable non-blocking follow-up coverage; this revision's expiry tests deliberately exercise campaign-limit fallback, so they are not presented as execution evidence for that additional scenario.

Validation on this revision: the local affected-area suite reports 216 passed, 54 subtests passed; Ruff and diff checks pass. Current-head full regression: Python 3.10 and Python 3.11 each report 5,288 passed, 37 skipped, 672 subtests passed, 1 warning. Both raw logs identify merge checkout e371b3b7fd663b8e84bb3f8ce0ebe982a38dc857, merging this exact head into the stated base, and show Ruff passing. Installed-package checks for both Python versions, Windows/macOS compatibility, lint/smoke and reference checks also succeeded; dashboard-browser is skipped for the draft.

The acceptance-scope cautions remain correct: repository tests and finite-edit CPU training do not establish real Engine/Skill VATS/GVO/final pilots, production resource isolation, complete provider accounting or performance gains. The PR description and support matrix retain those gaps, and the PR remains Draft. This follow-up records fixes and test evidence, not independent approval, whole-issue completion or a merge decision.

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同 head CI 证据增量检视,精确 head:dc7292a3cd3d624794ed280a0d677d20e03fff29。源码范围 dc7292a → dc7292a(无新提交);上轮源码检视:#488 (review)

评论 6981045390 提供 Ready run 定位。本轮不采纳评论中的修复/验收判断,直接核验 GitHub 当前对象及原始 job 日志:

  • full-check Python 3.11:Ruff 通过,pytest -rs 为 5288 passed、37 skipped、672 subtests passed、1 warning,498.38s。日志检出的执行对象是合成 merge e371b3b(当前 dc7292a 合入 cf671a8),不是另一个旧 head。https://github.com/InferMatrixLab/InferRSI/actions/runs/44319989516/job/134856070233
  • Windows host compatibility:8 passed + 25 passed;macOS:8 passed,均同一合成 merge。它们是选择性 host 测试,不能宣称覆盖全部 VATS/WorkControl 场景。
  • 最近状态:full-check 3.10 仍进行中,installed-package 双版本成功,dashboard-browser skipped。37 个 pytest skips 包含可选依赖/平台/opt-in 环境缺口,不能写成全能力验证。

已解决项仍沿上轮:两项 P2 的源码与回归定义已核验。本轮补到完整 3.11 suite 成功的服务端运行证据,不是本地重跑,也不以 host 8/25 项替代具体新回归。没有源码变化,不重复声称重新完成全部代码审查。

批准前仍需:当前 head 必要检查的完整结果,以及此前记录的范围/owner 验收与真实 Engine/Skill 试点证据边界保持明确。真实集成、硬件性能、生产资源隔离和完整 provider 成本没有在本轮得到验证;CI 不能替代这些证明。当前仍 COMMENT,不宣称总体验收通过。

实际执行:读取作者/head/本人既有 review、隔离评论请求和证据定位、读取上述原始 Actions 日志。未读取他人检视意见,未执行仓库代码,未修改分支或合并。非阻塞补测建议仍见上轮,无新增。

@david6666666 david6666666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One recovery case after a verified late cleanup.

body["spent_ms"] += actual_ms
if body["spent_ms"] > body["cap_ms"]:
body["blocked"] = True
if transition is not None and not body["blocked"]:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After a known overrun blocks the campaign, a later verified cleanup settles this operation but skips the transition, so search.pending still points at settled work. On reopen, Search.recover() only handles the already-settled deadline case and can ask the adapter for a receipt again. Could recovery archive and clear this settled pointer without adopting its output or unblocking dispatch? Please cover overrun → verified cleanup → reopen and retry.

Retain protocol timeout receipts with supervisor cleanup facts so known
usage settles and clean domains are released without relaxing overrun
blocking. Resume committed ACCEPT decisions without sampling again and
preserve consumed steps when restoring paid continuations.

Reject development Skill evaluators before final claims. Cover process
timeouts, uncertain cleanup, restart windows and final scope validation,
and document the differences from the SpecRSI reference implementation.

Agent-assisted-by: Codex (implementation, tests and reference analysis)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

增量独立检视:dc7292a3cd3d624794ed280a0d677d20e03fff29 → b849894(1 commit / 8 files);base 仍为 cf671a8。上次本人发表:#488 (review) 。本轮 COMMENT,尚不批准。

独立阅读全部八文件增量及相关调用链后,本轮未发现新增确定性源码缺陷;以下是源码层面的确认,不是运行结果:

  • Engine 把 TaskProtocolError(timeout) 转为保留的 INCONCLUSIVE 评估,并合取 task/protocol cleanup,arguments 尚未启动 task 时允许 execution digest 为 null;ArtifactAdapter.verify 随后保留实测 elapsed_ms 与绑定回执。非 timeout 的身份/协议异常仍不被这个分支放宽。新增测试既有真实睡眠子进程 timeout 控制,也有 arguments/assess × cleanup True/False/None 的替身用例,二者证据性质不同。
  • task_protocol 仅由 supervisor 的 domain_empty,或明确未启动,给出 timeout cleanup_verified。未知清理不能因捕获异常而变成已释放;实际超额的预算阻塞仍保留。此前已修复的 WorkControl 已知用量下界与未启动恢复问题未被本轮变更重新打开。
  • Search.run 在重采样前识别已持久化 ACCEPT;continue_from 的 forced ACCEPT 重启也走终止路径。decision_count 将被省略的自然前缀 step 加回 trace 长度,同时用于步数上限与随机流索引。已交叉检查 continuation 拷贝如何保留 draft_attempts、spent_before 与 forced trace,避免将只训练 suffix 理解成获得额外执行步数。新增测试覆盖重开 store、重复恢复及晚期 continuation 的两步上限。
  • Skill validate_final 在 claim/access 前拒绝 development、缺失或不匹配 holdout scope;已有 final_result 仍只返回旧结果。新增合法路径的二次调用确认不再次访问 holdout。

批准前尚需当前提交的执行证据。 当前关联 Ready run 44352212642:full-check 3.10/3.11 的 Ruff step 成功、测试进行中;Windows 进行中、macOS 排队;installed-package 双版本与 pr-references 成功,dashboard-browser skipped。这是连接器 job/step 状态,不是已读完整测试日志。PR 正文目前仍列 dc7292a 的 head/5288-pass 旧验证,不能作为本轮新增测试通过的依据;请在本轮检查结束后更新准确 head 与结果。

范围边界: #363 的 RFC specrsi-r2-2026-11-18 仍为 proposed,不能把本 PR 的局部软件控制当作 AC-01–AC-10 全部完成。新增 reference-comparison 文档明确有限编辑词表、线性特征、默认无 refinement 和缺少真实试点的限制;本轮没有独立核验外部论文/参考仓库的逐项一致性,因此不为“复现论文”背书。真实 Engine/Skill pilots、硬件资格、生产资源隔离、全生命周期/provider 成本仍未在本轮验证。

非阻塞补测建议:在 Engine timeout 回执之外,增加实际 task cleanup=False 与 protocol cleanup=True 的合取用例,确保不释放未知 task domain;此建议不将现有源码合取判断描述为已知缺陷。

实际执行:连接器读取作者/head、本人历史 review、八文件完整增量、AGENTS、关联 RFC 需求与 adapter/continuation/final/protocol 调用路径、当前 CI 元数据;未读取他人检视意见。未运行仓库代码/测试、真实模型或设备/provider 实验,未修改分支、合并或启用自动合并。

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

当前 head b84989494dfa69628aa918ed85653afc936a5493 的 CI 证据增量检视。源码范围 b849894 → b849894(无新提交);上次独立源码检视:#488 (review) 。本轮仍为 COMMENT,尚不批准。

上轮等待的常规 CI 执行证据已补齐。 我读取了 Ready run 44352212642 的原始日志,而非仅依据绿色状态:

  • Python 3.10 与 Python 3.11 均 Ruff 通过,5344 passed、37 skipped、672 subtests passed、1 warning,分别 530.86s / 575.75s。
  • Windows 为 8 项兼容性与 25 项 primitives 测试通过;macOS 为 8 项兼容性测试通过。
  • installed-package 双版本原始日志确认构建 wheel、在独立 venv / checkout 外以 isolated Python 检查,均列出 128 records、930 个资源文件及相同资源摘要。references、draft smoke、declarations 当前成功;dashboard-browser skipped,不能计作通过。

执行对象更正: 上述原始日志实际 checkout 是合成 merge e6f9d7b8d…,将当前 b849894 合入 965ab5cf83399a99530949b04bb934a892ee417c。PR API 的 base 字段仍返回 cf671a8,不能因此把此轮测试写成运行于该旧 base;PR 正文也仍保留 dc7292a / 5288 的旧说明。应按实际 H/B/T 更新验证段落。

本轮重新核对 AGENTS、协作契约及当前 speculative-search 支持/验收矩阵:上轮源码发现的修复仍适用,没有新源码缺陷结论。剩余批准前事项不是再次跑同一常规 suite: #363 当前仍为 proposed,并要求维护者冻结/接受交付契约;需要明确该实验性局部交付的接受范围及其未完成项处置,不能由全绿 CI 代替。完整 AC-01–AC-10 所需真实 Engine/Skill VATS/GVO/final pilots、资源/成本资格与对照实验没有在本轮得到新证据。若维护者明确接受有边界的软件切片,应据该接受范围判断 PR readiness,而不把整个 RFC 的完成当作本次源码检视已证实。

已执行:精确 head、本人历史发表、六个完整原始 job 日志、当前 checks/status、当前仓库契约及 #363 正文核对。未执行:本地测试/构建、仓库代码、模型或设备实验、原论文复现实验;未读取他人评审意见,未修改分支或合并。原非阻塞补测建议仍见上轮,不重复升级为缺陷。

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NO PASS

同 head 证据增量:b84989494dfa69628aa918ed85653afc936a5493 → b84989494dfa69628aa918ed85653afc936a5493,承接 上轮检视。新增单卡 910B1 诊断把证据推进到了 Engine 搜索与独立 final。作者报告搜索命中 1.35× 为 3/3,但独立 final 达标仅 1/3;110.08 秒是首次观察到搜索命中的平均耗时,不是可靠确认达标的耗时。速度比相对原始 denoiser,不是 PR 对 base 的收益。

批准前必做:明确由维护者接受的 #363 交付切片。若以这轮设备试验满足验收,请给出外置 TargetSearch 的源码 pin 和原始逐候选回执;当前 vats.py:635–693 按顺序执行 action,内置停止条件是预算或截止时间,不含试验的逐候选达标早停。若按完整 #363 验收,仍缺真实 Engine GVO、Agent/Skill 完整试点及 AC-10 的配对和消融对照。没有发现新增源码缺陷,也不把常规 CI 当成门槛。

本轮读了新增证据、实际控制路径和 #363 交付要求,未运行代码或 NPU,未复算尚未公开的原始数据。

@david6666666

Copy link
Copy Markdown
Collaborator

#516 has been merged into main. Please rebase this branch onto the latest main and adjust the changes as needed. Please verify that tools/speculative_skill.py can still read its selected skill through KnowledgeReader with the catalog introduced in #516.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants