Repository navigation
Gate RSI search on measured bottleneck diagnosis - #519
Bounty-hunter wants to merge 4 commits into
Conversation
Agent-assisted-by: Codex (implementation and self-review) Signed-off-by: dengyunyang <584797741@qq.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
首次独立检视:9c7b7b05042f54ca71d13b2897a7505d818d3429 → f473060feb5ab51ea4a1d3ae3cab75840e9449b4,范围 #337 的诊断顺序和对照回收。
批准前必做:报告需确认引用的诊断仍含测量凭据。 optimization._load 会核对保留的评估/交付证据,但不检查新增的诊断记录;benchmark_compare 回收时只检查 diagnosis_id 是否存在。若归档中诊断 ID 保留、测量摘要缺失或为空,报告仍不会在此处增加缺证据项,可能把无法审计的对照算入有效结果。这与本 PR 要求保留测量诊断不一致。建议在回收时校验引用诊断的必填字段与摘要;增加摘要缺失/空白时报告降为不可纳入的回归。不是要求 Harness 判断测量本身是否科学有效。
启动、诊断登记、knowledge/round 门槛及 service-reuse epoch 分流已静态核对。未运行代码或测试,未查 CI。
david6666666
left a comment
There was a problem hiding this comment.
The comparison collector should verify the recorded activity-to-diagnosis binding, not only the existence of a diagnosis.
| if plan.get("diagnosis_required"): | ||
| diagnoses = {record["id"] for record in control.get("diagnoses", [])} | ||
| if any(record.get("diagnosis_id") not in diagnoses | ||
| for record in (*control["rounds"], *control["contexts"])): |
There was a problem hiding this comment.
This also accepts a valid diagnosis from the wrong point in the run. If a collected round's state.json reference is changed to a second, later diagnosis, the original observation.json hash still verifies and the membership check passes, even though the retained observation names the first diagnosis. Please check the round's reference against its retained observation (and the activity order), with a regression that swaps between two complete diagnoses.
Agent-assisted-by: Codex (self-review fixes and regression tests) Signed-off-by: dengyunyang <584797741@qq.com>
Agent-assisted-by: Codex (implementation and regression tests) Signed-off-by: dengyunyang <584797741@qq.com>
Agent-assisted-by: Codex (regression test) Signed-off-by: dengyunyang <584797741@qq.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
独立增量检视:f473060feb5ab51ea4a1d3ae3cab75840e9449b4 → 680827c3fae2c5c780542179e6a53e7e0b7f54a5,上轮意见。
上轮缺项已修复:回收读取会检查诊断 ID 唯一、瓶颈与测量摘要非空、诊断时间有效,与对应轮次的绑定及 epoch 一致。不再只检验 ID 存在,轮次的诊断引用与已固定的 observation 对照;service-reuse 恢复也复用该校验。回归覆盖摘要缺失/空白、重新绑定较晚诊断和记录时间失真时不计入有效对照。既有会话仍按冻结计划读取。未发现新增阻塞。
静态核对了代码和回归;这是对诊断记录完整性及顺序约束的检视,不验证摘要中的测量结论。未运行代码、测试或查 CI。
david6666666
left a comment
There was a problem hiding this comment.
Round binding is now checked against retained evidence. Knowledge capture still has a separate gap.
| if not isinstance(reference, str) or reference not in diagnoses: | ||
| raise ValueError("Campaign activity lacks a retained bottleneck diagnosis") | ||
| diagnosis = diagnoses[reference] | ||
| if round_record and ( |
There was a problem hiding this comment.
Both knowledge-capture read paths call this with round_record=False, and the diagnosis ID is stored only in the mutable context reference. As a result, rebinding a retained knowledge capture to a later or different-epoch diagnosis still passes as long as that ID exists. Please bind the diagnosis ID and the capture epoch/order to retained capture evidence and verify it on collection, with a regression for a knowledge capture rebound across epochs.
|
#516 has been merged into |
Refs #337
问题
#501 的 on/off 对照已经冻结任务、评测协议和预算。通用 RSI loop 目前允许直接查知识或提交候选,瓶颈分析和 profiling 的顺序由 Agent 自行决定。这个 PR 把先诊断再搜索的规则接入共享 Harness。
改动
campaign diagnose和benchmarks session diagnose复用同一控制器,记录当前瓶颈与 workload 对应的 UTF-8 测量摘要。摘要可来自当前 profiling 或已有的同条件测量。knowledge和候选round都要求先登记诊断,并记录关联的诊断 ID。service-reuse 建立新的参考条件后,需要登记当前 epoch 的诊断。benchmarks compare的 on/off 两组、独立 benchmark 和用户直接发起的普通项目优化都走同一规则。on 组在有具体实现疑问时按需检索,off 组完成诊断后直接提交候选。回收时核对测量摘要、原始轮次引用和诊断时间。start沿用现有参数。既有会话仍按各自的冻结计划执行。流程
diagnose保存瓶颈及对应观测。Harness 核查诊断记录和顺序。测量口径及瓶颈推断由执行者根据当前任务确认。
验证