Repository navigation
Claude/rea trace - #3
Merged
Merged
Conversation
…conciliation, obligations, MCP Reverse-engineer research folders by content instead of folder names: a read-only stdlib probe runs next to the data (local or SSH, resumable, NFS-hang tolerant), facts go into a disposable SQLite index, and coverage-aware tools recover experiments (configs, table columns, encoded run names, k=v lists, aliases, seeds vs. context), trace report numbers to raw values or recomputed mean±SD (sample vs. population, eval-seed-first, macro), follow provenance/xrefs, draft evidence-bound Cell/Run patches and audit evidence obligations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cro-per-seed reconcile Fixes from the first agent benchmark: one-call orientation (key documents with full paths), directory listing over the index (archive members grouped), validity flags for do_not_use/incomplete/quarantine studies with flagged-but-declares-complete contradictions, provenance ranking of same-study scripts that name the result (producers above aggregators) and declared host fields, the 'macro over task per training seed, then SD' aggregate in reconcile, configurable grep per-file cap. Also includes changes made in a parallel session (archive member indexing, text snapshots, --format text). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, reconcile filters From the second benchmark run: paired per-episode comparison of two result files aligned by episode id, read a single JSON branch instead of paging, provenance for directories (writers, scripts here/above, citers, host hints below), host names recovered from layout tokens (collected/h204, rp6.jsonl), reconcile filters by number/state/line, consistent --limit for read, and declared status/host facts in the probe (v6) for validity contradictions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One call answers several prose questions (phrase first, all-terms fallback), ranking the authors' key documents above vendored code and collapsing identical copies; no access to the source mount. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… grep line output pair accepts * to pair every seed and sums the totals (id arrays must be unique and distinct from the outcome array); provenance lists files that name the artifact (job queues, manifests) and declared hosts below a directory; grep accepts --path and prints one line per hit in text mode; filtered reconcile omits the report-wide summary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nce obligations From the INTACT-Lite map-building run: fetch splits comma lists and errors when no path is in the index; claim numbers that are differences of two cited values close as derived-difference-verified (notes, not open diagnostics); scientific notation parsed in claims. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A directory with >=5000 files whose sampled names are all non-text (dataset images, frames) becomes one bulk record (count + extension histogram) instead of per-file stats; sub-directories are still walked and coverage reports the summarized count. v5 parts remain resumable on huge trees. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…allback in find validity checks backticked paths in a flagged study's documents against the inventory (e.g. README requires results/cells/ that was never archived); reconcile mean±SD matches carry both sample and population SD; find falls back to structured facts (job queues, manifests) when no prose contains an id; grep auto-detects regex patterns instead of reporting a literal not_found; read --pattern; xrefs accepts --all-mentions; guide documents globbed pair. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…exit codes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…xed-SD flag, validity alternatives/divergent copies/cited-missing, local-first grep, fid-range queries (brief 15 min -> 7 s on 577k files), cell-stats disk cache, D-drive temp store Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tting outliers), probe v7 (launch args, empty dirs, git refs, archive hashes, big-tar listing), live archive-member reads, ancestor-relative citation resolution Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…D reports and missing members of file sets (absolute paths matched by suffix); 7/12 planted defects found in one 5 s call, no false positives Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hive listings when the probe capped or timed out; grep reports per-file match counts and only falls back to the host when unsnapshotted text files exist Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… SD printed from rounded per-seed means is matched (flagged sd_rounding); bisect cell lookup and float stats make reconcile ~20x faster Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… label identical content as one version Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tes-flagged-study, table-row-count-mismatch); planted benchmark: 11/12 defects in one audit call, no new findings on the real INTACT / INTACT-Lite trees; INVESTIGATE round-8 notes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…outliers (referenced checkpoint shape), duplicate checksums; identical-replicate ignores host mirrors and dataset manifests; scale outliers only in percentage columns (INTACT real tree: ~730 -> 7 findings); ls shows symlinks/skipped folders; probe v8 records symlink targets and snapshots small text files; grep states non-exhaustive results; CLI option help Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ort queries; number lookup via numeric lines Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… files + shared hashes); INVESTIGATE round-9 notes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s), keep aggregate metrics first when a file has thousands; reconcile --number traces only the lines printing it and bounds the sibling search (collapse: 20 min -> 16 s) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…/count/compare follow symlinked folders inside the source; .tar.zst members listed/read via zstd; checksum lists resolve names from the folder a listed file is found in Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ame finished (summary-predates-run); count includes files inside skipped cache/.git folders (probe tallies them); probe v11 keeps text of summary/status/coverage files and text files up to 64 KB Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ismatch (grouped per declaring file) and verbatim categorical outliers (version drift) across result/eval-wrapper JSON; referenced-path outliers compare only the tail of the path (relayed copies are not outliers); INVESTIGATE round-10 notes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…loads another epoch's weights; one eval seed records a different checkpoint hash than its siblings), stamp-after-write, identical replicates among eval-wrapper JSON (seed_N names), infra knobs ignored in setting outliers; probe v12 parses full launch command lines Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (Macro/Mean = mean of the task columns) and delta columns (difference or relative change learnt from the other rows); copies of one document merged into one finding Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lag/version differs from all others; fraction among percentages), identity-mismatch (checkpoint of another seed than the record's), version drift among 3+ sibling evaluations, findings per file merged, vendored doc copies skipped for SD conventions; INVESTIGATE audit procedure Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ck the base folder (list dir or a parent) that resolves most names and hash listed files live (index hashes go stale) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… rows, failed table arithmetic, missing cited files); table arithmetic checks Mean/Macro rows too (word-bounded skip columns); audit missing-in-replicate (result file present in all sibling folders but one) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ve only when it stands alone (fixes false mismatch on manifest JSON lines naming .pdf files) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d lists the raw files each report cites; identical-replicate covers per-run text outputs under results/eval; resolved-config files excluded; INVESTIGATE: split large audits by study Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es vs real sizes, headerless size<TAB>path listings too) and event-order (finalized before archived/started); probe v13 keeps text of index/sums/provenance/audit files Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eview and points to worklist Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…them (r2_mean vs r2_per_dim) and flags one eval among siblings that loads another seed's checkpoint Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s checkpoint (B_epoch5.json -> A_epoch5.ckpt) and a record whose start time disagrees with its own log by a lone offset Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…written map JSON (references/SEMANTIC_MAP.md) rendered as research summary, claims with status and numbers, experiments by research question with error-bar charts, timeline, superseded work, open questions and audit leads; trace report command; index-only HTML view; INVESTIGATE notes; scrub project-specific names Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… against the project (live files, symlinks, archive members, numbers recomputed from the cited table rows) and the report shows a verification badge per claim, result row and number Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…an ± SD pairs matched over any 2-5 rows of the cited range (interleaved replicates), CSV cells never merged Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…V rows), percent cells recomputed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…us and verification badges, experiments by research question, timeline, superseded work, open questions; evidence opens the hash-pinned snapshot and jumps to the Cell/Run that cites the same file; trace publish-map verifies a semantic map, imports cited files as snapshots and stores it in the workspace; hyphenated trace subcommands; #view= deep links Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.