Skip to content

Archive MiniMax-H3 Ref2VA Ascend 950PR historical evidence - #527

Open
JingjingWu6 wants to merge 1 commit into
InferMatrixLab:mainfrom
JingjingWu6:h3-ref2va-ascend-history-20261118
Open

JingjingWu6 wants to merge 1 commit into
InferMatrixLab:mainfrom
JingjingWu6:h3-ref2va-ascend-history-20261118

Conversation

@JingjingWu6

Copy link
Copy Markdown

Governing Issue and delivery slice

Refs #520

  • Accepted scope / delegation: Convert the privacy-reviewed historical attachment into the repository's compact evidence/archive format, following the Archive MiniCPM Seed-TTS details and retain verifiable core evidence #435 pattern.
  • What this PR delivers: A searchable eight-file evidence core, all 448 source-derived observation rows, v2 archive metadata, arithmetic recomputation, and evidence inventory registration.
  • What it does not deliver: A campaign PASS, model-quality equivalence, a reproducible optimization patch, or new NPU experiments.

Knowledge authorship declaration

Not applicable: this change does not modify protected knowledge paths.

Changed artifacts and evidence

Requirement Actual change Check / evidence and result Remaining gap
Preserve complete historical material outside Git Versioned archive metadata for h3-ref2va-ascend-history-20261118-v2.tar.gz Local bounded verification passed: 2,684 regular files; 40,740,727 expanded bytes; SHA-256 778f5c7c73cea6ee0f545a59fa91f9a40c79818545f86d49f001d2b36585a0cd Corporate upload policy blocked the 5.45 MB Release asset; maintainer publication at the declared URL remains pending
Keep compact searchable evidence in Git evidence/h3-ref2va-ascend-history-20261118/ 448 indexed observations; arithmetic recomputation passed Original runtime/source identities and media remain unavailable
Register the bundle tools/evidence_inventory.json tools/validate_evidence.py: no errors on the prepared snapshot External archive is intentionally unavailable to offline CI

Checks and dependencies

  • Checks actually run: archive metadata validation; bounded local archive verification (all 2,684 regular files and byte/hash checks); tools/validate_evidence.py; arithmetic recomputation; JSON syntax checks; targeted privacy scan; git diff --check after LF normalization.
  • Not run: pytest (the available local Python has no pytest); no NPU/GPU experiment; no model or VBench rerun.
  • Dependencies: The declared versioned Release asset is intended to be the large-evidence authority. Its publication is pending because the corporate proxy blocks uploads above 1 MB.

Remaining work and Issue closure

  • Remaining Issue requirements / follow-up: A maintainer should publish the verified archive at the declared upstream Release URL, then independently download and verify it. Any optimization reproduction belongs in a separately accepted campaign.
  • Goal-acceptance owner or accepted closure delegation: wujingjing6
  • Goal ready for acceptance? Not claimed. The historical evidence remains INCONCLUSIVE.

Preserve a compact searchable evidence core and register the versioned archive while retaining the INCONCLUSIVE verdict.

Agent-assisted-by: Codex (implementation and validation)
Signed-off-by: JingjingWu6 <JingjingWu6@users.noreply.github.com>
@Dong1017 Dong1017 added area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice labels Nov 19, 2026

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NO PASS

首轮静态检视 493416520f537590eb8f2a4e74f276f82e488307 → da8c9f49c6295484caf28ed42da92f91fb674fe1,对照 #520 与归档契约。

  1. 声明的 v2 replay 与比较器不兼容。 artifacts.json 把 results/trials.jsonl 指定为 replay 输出,但 448 条索引使用 trial_id/name/source;tools/evidence_archives.py:466–486 读取 id,还要求 conclusion/artifact_id/source_record/configuration。按现在的格式会得到 trial_mismatch,不是 manifest 通过就能验证。请生成符合共用契约的投影,或显式采用受支持的历史索引契约。
  2. recompute 不能独立重建输出。 validator/recompute.py:5–10 读取已有 summary 并核对两个算式,只打印两个标量,不从已认证的原始记录生成所声明的 trials/summary。请补可比对的重建输出;若只交付算术核对,应明确缩小范围和声明。
  3. 完整归档仍需可取回。 已核对 448 个索引 ID 和汇总计数,但未取得声明的 release 资产来独立核对完整覆盖和摘要。若保留完整归档交付,请发布后回取验证,或提供获授权的可核查存储路径。无需重跑 NPU 实验,历史结论仍是 INCONCLUSIVE。

非阻塞建议:将 7 条 FAILED 等计数标为源标签统计,避免把含非零退出码的 RECORDED 行误解为成功。已静态核对九个改动文件、完整索引及共用比较器;未运行验证器、下载完整资产或执行实验。

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR review verdict — DO NOT PASS

Reviewed base 493416520f537590eb8f2a4e74f276f82e488307 and head da8c9f49c6295484caf28ed42da92f91fb674fe1 (full diff, first review). pr/prior-reviews.md records no formal APPROVED review on this head and no outstanding CHANGES_REQUESTED, so this is the first formal pass. I also read Issue #520 and its two existing comments.

The framing is honest and the verdict is stated consistently: INCONCLUSIVE appears in README.md, plan/effective.json, results/summary.json, results/external_optimization_record.json and results/artifacts.json → availability.verdict; historical PASS strings are demoted to source observations; 4.2242× (50-step engine stack) and 7.3747× (10-step + Cache-DiT) are kept as separate, non-cumulative comparisons, matching both prior comments on #520. No gain-ranking or acceptance language anywhere in the bundle. The blockers below are pointer and classification integrity, plus a diff I could not finish reading.

Verified in the seeded diff

  • summary.json arithmetic holds: 701.954369 / 166.172687 = 4.2242463648…, matching external_optimization_record.json aggregate.
  • Three-seed Cache-DiT mean holds: (7.71175565987307 + 7.2540855539449725 + 7.158391805120384) / 3 = 7.374744339646142, and deepseek_range is the correct min/max of the same three values.
  • status_counts in results/summary.json sums to 448 (7 FAILED + 404 RECORDED + 4 RECORDED_VALUE + 27 COMPLETED + 6 INCONCLUSIVE), matching trial_rows: 448 and the results/trials.jsonl hunk header @@ -0,0 +1,448 @@.
  • File count and uncompressed bytes in artifacts.json (2,684 files / 40,740,727 bytes) match Issue #520 exactly.

Blockers

  1. Incomplete diff — diffComplete: false. The seed truncates inside results/trials.jsonl (hunk header claims 448 rows; I read well under half). Two files the bundle depends on are not visible: validator/recompute.py (named in README.md "Retained core", artifacts.json → key_files.validator and verification.command) and results/redactions.json (named as key_files.redaction_inventory). results/redactions.json should have sorted between external_optimization_record.json and summary.json and does not appear, so it is either absent from the change or dropped by the seed. Either way I cannot check the arithmetic-replay claim or the privacy-review claim the archive rests on. Please confirm both files are in the head.

  2. Archive identity changed from v1 to v2 without explanation. Issue #520 pins h3-ref2va-ascend-history-20261118-v1.tar.gz, SHA-256 06b95a85…a2627d, 5,537,276 bytes. results/artifacts.json now declares -v2.tar.gz, SHA-256 778f5c7c…85a0cd, 5,454,990 bytes, against a release-download URL. Compressed bytes and digest changed while file count and uncompressed bytes did not, which reads as a re-compression of the same tree rather than a re-collection. State which it is: if v2 is a re-gzip of the same 2,684 files, say so and retire the v1 digest in #520 so two competing hashes are not circulating; if the tree changed, say what changed. I cannot confirm the v2 release asset exists or is published, so the "results/artifacts.json is the authority for the external archive" sentence currently points at an unverified URL, and verify_evidence_archive.py --download is unproven.

  3. key_files.record points at a path the bundle does not contain. results/artifacts.json names h3-ref2va-ascend-history-20261118/records/external_optimization_record.json, but the file added by this PR is results/external_optimization_record.json, and README.md "Retained core" also locates it under results/. Every other key_files entry uses the same relative layout as the repo bundle. Fix the pointer or relocate the file so the key-file inventory can be resolved by tools/verify_evidence_archive.py.

  4. duplicates: 0 is unsupported as written. Many distinctly named attempts share byte-identical retained source files, e.g. sha256 cef20a21…6286 on hccl_a, mix8s_a, mix8_a and tp4_03; a7634062…e3dc on b0_tp4_s50_d5, rsi_base_trial-001, sweep_b0sub, vbench_b0, vbench_b0sub_rep, vbench_fs8_s6, vbench_s10, vbench_s8, vbench_s9; bf4a9131…83ed on res_probe, sci_fi_dense, vbench_b0_s2102/2103, vbench_s10_s2102/2103 and all vbench_s1*. Issue #520 already concedes that related documents describe the same named attempt, which is exactly what these shared digests show. duplicates: 0 either means something narrower than it reads (duplicate index rows) or is wrong. Define the field, or record the duplicate groups, so a downstream consumer does not read 448 rows as 448 independent observations.

  5. status conflates run outcome with historical gate verdict, so failed launches are counted as recorded. Visible rows with non-zero return codes carry "status": "RECORDED": b1_gate0p9_seed2202 and l0_tp2usp2_bf16 (rc: 52), u0_usp4_bf16 and phase0_c03_v032 (rc: 22), lossless_tp4_ref_c47 (rc: 7, wall_s: 0.0074). Separately, the local-summary row is source_status: "FAIL" / status: "FAILED" while carrying pooled speedups of 4.33× — a summary record classified as a failed run. And DETERMINISM_FINDING puts prose in the status field: source_status: "QUALITY_BAR.JSON INTENTIONALLY NOT FROZEN; RSI_VALIDATE_USP.SH KEEPS FAILING CLOSED." With FAILED: 7 and interrupted_or_incomplete: 0, the index under-reports failed launches. Enumerate the field (run outcome) and keep the historical gate label in a separate field, or state the classification rule explicitly and note that metrics.rc is the authoritative failure signal.

  6. plan/effective.json drops the environment identity that #520 retains. The evidence contract puts the RunSpec and environment under plan/. The 14-line file records backend, hardware, model, scenario, window, source revision and verdict — but not Python 3.12.16, torch-npu 2.10.0.post7, vLLM-Omni 0.32.0+npu, vLLM-Ascend 0.19.1rc2.dev3247+ga26cc45a2, or the missing CANN/driver/firmware identity, all of which #520 states. It also flattens "attempts used 2, 4, or 8 logical Ascend 950PR devices" to "4-8 logical Ascend 950PR devices". For a bundle whose purpose is preservation, dropping the recorded versions and device-layout spread loses the specific thing being preserved. Absence of frozen gates is fine and already explained; absence of the known versions is not.

Not verified here

  • No tests or validators run: validator/recompute.py, tools/verify_evidence_archive.py and tools/validate_evidence.py were not executed, and the latter two are not part of this diff, so I did not confirm they exist on base or accept this artifacts.json schema.
  • Archive bytes not downloaded or hashed; nothing in this review confirms 778f5c7c… against real content.
  • The visible trials.jsonl rows do not expose the 701.954369 / 166.172687 pair used for the 4.2242× claim, and external_optimization_record.json → source_refs digests the truncated file, so that pairing is unchecked beyond the division itself.
  • CI status on this head was not inspected.

Non-blocking

  • interrupted_or_incomplete: 0 alongside INCONCLUSIVE: 6 needs a one-line definition of what that field counts.
  • base_tp4_c03_v032_prof records foreign_requests: 3 with requests_served: 6 for requests_sent: 3; if that run feeds a retained ratio, add it to limitations as a contaminated-session observation.
  • Line endings are mixed: effective.json, summary.json and external_optimization_record.json are CRLF, artifacts.json and trials.jsonl are LF. Harmless, but inconsistent for a bundle whose integrity is digest-checked.

I have not approved anything and this review does not alter the scope of #520. Push the fixes and I will re-read the new head, including validator/recompute.py and results/redactions.json.


Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

@david6666666 david6666666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One measurement-boundary detail in the standalone evidence record.

"aggregation": "recorded_ratio"
},
{
"name": "10-step-cachedit-three-seed-recorded-speedup",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please label this as diffusion-stage speedup in the standalone record. The indexed timings observation contains both diffuse and e2e columns, and 7.3747× is the mean of the three diffusion ratios, not the end-to-end request ratios. The unit x and recorded-speedup name alone don't preserve that distinction for downstream readers.

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The three-seed Cache-DiT check has a reproducible false positive. With the indexed timings unchanged, I could replace the reported 7.3747× result with 1.0× and the checker still exited successfully. The inline comment gives the reproducer and a regression check.

p = summary['performance']
assert abs(p['opus_50_step_recorded_speedup'] - 701.954369 / 166.172687) < 1e-12
vals = p['deepseek_seed_group_speedups']
assert abs(p['deepseek_10step_cachedit_recorded_speedup'] - sum(vals) / len(vals)) < 1e-12

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a temporary copy of results/summary.json, I set deepseek_seed_group_speedups to [1.0] and deepseek_10step_cachedit_recorded_speedup to 1.0. Running python validator/recompute.py exited 0 and printed deepseek_speedup: 1.0. This assertion takes both sides from the edited summary, so dropping two seeds and changing the remaining value passes. Please derive each expected ratio from the timings row in results/trials.jsonl (b0.diffuse / cache.diffuse for seeds 2101, 2102, and 2103), then compare all three values and their mean with the summary. The single-seed edit above should exit nonzero.

@david6666666

Copy link
Copy Markdown
Collaborator

#516 has been merged into main. Please rebase this branch onto the latest main and adjust the changes as needed. Please check that the evidence archive still validates after the rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants