Skip to content

refactor(metrics): delete the metrics journal and model registry (XWING-2440) - #114

Merged
smith-xyz merged 1 commit into
traust-security:developmentfrom
smith-xyz:refactor/retire-metrics-journal
Oct 27, 2026
Merged

smith-xyz merged 1 commit into
traust-security:developmentfrom
smith-xyz:refactor/retire-metrics-journal

Conversation

@smith-xyz

Copy link
Copy Markdown
Contributor

Plan: 17-shaun-traust-core-only.md (part)

Deletes the metrics journal and model registry from the engine. Merge after traust-security/traust#294, which retires metrics history / metrics spend and moves registry models spend onto traust-core's SpendService.record. That PR also freezes traust's parity tests, which used registry/models.py and metrics/{collect,attribute}_spend as oracles, so nothing in traust imports what this PR deletes.

What

  • Deleted src/traust_engine/metrics/ (history, spend, collect_spend, attribute_spend, ops) and src/traust_engine/registry/ (models, ops), along with the facade's .metrics and .models.
  • Nothing else needed them (git grep -nE "traust_engine\.(metrics|registry)|MetricsOps|ModelsOps|\.models\(|h\.metrics" -- src tests is empty). ops/ uses core's SpendService and the model registry through core. portfolio/ never referenced them. In sweep/, benchmark.py's scorecard ran python -m traust_engine.metrics.history append as a subprocess. That was already a no-op, because history.py has no __main__ (test_no_cli_entrypoints). The call and its docstring line are gone.
  • locations.METRICS_HISTORY_REL / metrics_history() (only metrics/ops.py used them) are deleted. The PENDING lists in the module docstrings drop "metrics journal". README, CODEOWNERS and the ruff per-file ignore for history.py are updated to match.
  • .ratchet.json shrinks: metrics/history.py (contracts 2), metrics/spend.py (1), registry/models.py (1) and registry/ops.py (1) are removed.

Deleted tests

Test Why
tests/test_metrics_history.py the hash-chained journal is retired
tests/test_build_spend_dashboard.py the legacy spend dashboard is retired
tests/test_attribute_session_spend.py transcript attribution lives in traust-core (SpendService.usage_summary / attribute_by); traust's frozen parity tests keep the engine's output
tests/test_model_pricing.py engine cost_usd / long-prompt pricing is deleted; core ModelCatalog.cost owns pricing
tests/test_model_registry.py engine resolve/stamp/validate/spend_row are deleted; core ModelCatalog owns them, and traust's test_registry_doors keeps the engine's outputs
tests/test_traust_engine.py::test_metrics_journal_follows_configured_progress_tracker, ::test_metrics_spend_build_uses_configured_paths, ::test_models_registry_and_validate_from_context test .metrics / .models, which are deleted

Changed tests

Test Old → new Why
tests/test_traust_engine.py::test_two_engines_are_isolated_by_their_context isolation shown by h1.metrics.append rows → each engine's sweep._progress_tracker() follows its own context .metrics is deleted; the property (one context per engine) is unchanged
tests/test_traust_engine.py::test_namespace_ops_are_cached_per_engine drops the .metrics / .models lines those namespaces are deleted
  • A person reviewed each changed test and the behaviour change is intended

Behaviour changes

  • HarnessEngine.metrics and HarnessEngine.models no longer exist (traust#294 removed the last callers).
  • sweep benchmark score no longer starts a subprocess that did nothing. Output and exit codes are unchanged.

Validation

  • make lint-fix && make test: 451 passed, 4 skipped (tree-sitter not installed), 5 subtests passed.
  • Importing every module (pkgutil.walk_packages) works.
  • Consumer ratchet as CI runs it (RATCHET pin from ci.yml, --reference = development's .ratchet.json in a file): rc 0 after the shrink (it asked for exactly these four).
  • traust#294's suite with this branch installed in place of the pinned engine: 2657 passed, 2 skipped.

What traust still uses from the engine

load_engine → HarnessEngine with .sweep (sweep stages), .portfolio (portfolio graph) and .adapters (the rule lane's opengrep config). Tests also use locations, sweep.engine, sweep.benchmark and adapters.opengrep. All of these wait on PENDING(decision: sweep stages | portfolio graph layers).

…NG-2440)

traust retired `metrics history`, `metrics spend` and the journal row behind
`registry models spend` (now a traust-core spend event). Nothing in ops/,
sweep/ or portfolio/ reads metrics/ or registry/, so both go with the facade's
.metrics and .models; the validation benchmark's journal append (a no-op since
history.py lost its __main__) goes too.
@smith-xyz
smith-xyz requested a review from a team as a code owner October 27, 2026 20:13
@openshift-ci

openshift-ci Bot commented Oct 27, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smith-xyz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 27, 2026
@smith-xyz
smith-xyz merged commit 0bafe00 into traust-security:development Oct 27, 2026
5 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant