Skip to content

feat(use): admit training-inference paths under a declared bound - #91

Merged
FeathBow merged 7 commits into
FeathBow:mainfrom
sysuTobo:feat/training-inference-paths
Nov 2, 2026
Merged

FeathBow merged 7 commits into
FeathBow:mainfrom
sysuTobo:feat/training-inference-paths

Conversation

@sysuTobo

@sysuTobo sysuTobo commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Why

A use contract can require a candidate implementation to reproduce the reference, but the training-inference gap — the learner and the engine scoring the same rollout tokens — was only a diagnostic summary in a history report. This PR turns it into a declared, replayable claim: a contract can require that the two sides agree on those exact paths, either bit for bit, per token, or as a whole-path log ratio.

What

  • New Invar.Paths module. A path observation carries the token path and prompt split, the distribution rule, the declared input (prompt, seed, token limit and temperature), the forward-weight identity (frozen base + trainable adapter), the implementation, the FP32 words and their provenance. Comparisons are exact on Rational words.
  • Execution identity and coverage. Each observation also records the admitted call, attempt and instance and the SHA-256 of its source log; the observation scope hashes them. Learner words use the admitted update report, engine words use the admitted rollout, and the rollout is checked against the sample's behavior words. Every declared input must have an attached observation before a path requirement can decide; a missing input leaves it undetermined even when another input is refuted. Policy differences are excluded only after that coverage check, and the declared learner/engine implementations are checked on every attached observation.
  • One comparability rule. Observations are compared only when the token path and distribution rule agree and the forward-weight identities are equal. A current step that changes learner state without changing the weights is comparable. Pairs with different weights are reported as policy differences and left out of the decision; a requirement is undetermined only when no comparable pair remains.
  • Three contract relations. training-path-words, training-path-token-log-gap and training-path-log-ratio, each naming the learner side. The contract's learner field declares the learner implementation; use prepare reads it from artifacts.learner_policy. The learner and engine implementations are compared by model, revision, tokenizer, base and assembly; the adapter is the forward identity.
  • Input and cohort binding. Path observations attach to the update/cycle cohort (History.paths and rollouts use the update index), and use checks each observation's prompt, seed, token limit and temperature against the contract input. Samples outside the contract's domain are refused.
  • One comparison for history and use. History admission derives observations from each generation's samples; History.paths exposes the proximal, current and reference pairs. use inspect and use admit attach them through the --history-* options (including --history-run/--history-log) and report policy differences in finding.path_policy_differences.
  • Core-derived current identities. Probability artifacts stay v5. Each current report's state is the adapter content digest the core checks; history derives current identities from those reports/applied steps. The path rule keeps the declared distribution rule. Retained v5 runs remain readable.
  • No correspondence in this PR. Correspondence and its premise are deferred to policy: publish a generation across artifact formats with a declared conversion #84.
  • Docs. Updated for cohort mapping, input binding, the learner implementation, policy differences, aggregate path_log_ratio, domain refusal and rule scope.

Validation

  • cabal test: PASS.
  • HF worker suite: 225 tests OK (2 skipped), including a path contract over two optimizer steps with learner-scored references and a stale run whose observations cover both cohorts while its policy differences are reported separately.
  • Shared worker suite: 20 tests OK.
  • test/types.sh ghc, fourmolu --mode check, hlint src test app, cabal check: clean.
  • Real use admit --history-run against the HF fixture returns admitted_under_declared_reliance.

Limits

A pair with different forward weights stays a reported policy difference; relating genuinely different weights is #84. A contract with a training-path relation must declare its learner implementation. The path check binds each observation to the declared input and to the learner/engine implementations, not to a second execution of the model.

The use contract can now require that the learner's and the engine's log
probabilities agree on the tokens a rollout took, per token or as a path
log ratio. A new Invar.Paths module measures both sides exactly from FP32
words, refuses pairs whose token path, distribution rule or forward
weights differ, and lets a declared correspondence relate genuinely
different weights. History admission derives the same observations from
each generation's samples and probability artifact, and use inspect/admit
attach them through the history options, so a history report and a use
decision share one comparison.

Probability artifacts move to v6 and record the adapter identity each
optimizer step actually used; current words are therefore bound to the
forward weights that produced them. The three relations are exposed as
training-path-words, training-path-token-log-gap and
training-path-log-ratio, with the learner side named explicitly.

Signed-off-by: bohaotu <bohaotu@pony.ai>
History.paths paired each workload name with a whole role list, so a
one-step two-input run only passed because the first step's current words
equal the proximal words, and reference observations were dropped. It now
pairs each name with its own proximal and reference observation, maps the
cohort by the request's behavior version rather than the generation index,
and leaves per-step current observations out of the claim because their
weights differ from the behavior policy by construction. rollouts uses the
same behavior version.

The path implementation check no longer compares the adapter digest, which
is policy content rather than implementation identity, and honors declared
transfers. Reference observations now carry the request's reference adapter
on both sides. The probability artifact's per-step policy is required to
equal the applied step's before digest in both runtime and history
admission, the path rule records truncation, negative path budgets are
rejected, unknown path inputs are refused, and use accepts --history-log.

The path contract fixture now runs two optimizer steps with learner-scored
references, and the new tests pin reference identities, per-step policy
binding, adapter-independent path implementation, and unknown inputs.

Signed-off-by: bohaotu <bohaotu@pony.ai>
@sysuTobo

Copy link
Copy Markdown
Collaborator Author
  • Added e7e102a after the code review.
  • Fixed History.paths role/sample pairing and switched the cohort to the request’s behavior version; per-step current observations are no longer claimed because their weights differ from the behavior policy after the first step.
  • Path implementation checks now ignore the adapter digest (policy content) and honor declared transfers; reference observations use the reference adapter on both sides.
  • The probability artifact’s per-step policy is now required to equal the applied step’s before digest in both runtime and history admission.
  • The path contract E2E now runs two optimizer steps with learner-scored references.
  • Re-ran: cabal test PASS; HF suite 224 tests OK (2 skipped); shared suite 20 OK; types.sh, fourmolu, hlint, cabal check clean.

@FeathBow FeathBow left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on. The exact rational comparison, the per-token relation with its cancellation test, and deriving learner_engine from the same observations all look right. A few things need to change before this can merge.

Needs changes

  1. Cohort lookup under staleness. History.paths and History.rollouts now select the workload cycle by the request's behavior version (src/Invar/History.hs:259, :267), while the runtime selects it by update index (src/Invar/Runtime.hs:443). With staleness 1, update 1 has version 0, so its samples get cycle 0's names and its observations attach to update 0's inputs. The previous positional lookup in rollouts was correct, so invar score over a history regresses as well.
  2. The learner's implementation is not bound by the contract. samePathImplementation (src/Invar/Use/Admission.hs:208) compares model, revision, tokenizer and base. On the learner side, model and revision are taken from the engine's report and the tokenizer is shared, so in effect only the base is checked. The assembly is never compared, and ImplementationId is not used outside Invar.Paths, so a contract admitted for one learner implementation also admits another without a transfer. Ignoring the adapter makes sense; the assembly should be compared.
  3. Path observations are not bound to the declared inputs. Generated pairs pass the domain and context checks, but path observations attach by cohort and sample name only (src/Invar/Use.hs:88). Their prompt, seed, temperature and token limit should be checked against the input they attach to.
  4. Incomparable pairs. Unknown takes precedence over a refutation in the evidence graph (src/Invar/Spec/Evidence.hs:163), so one pair with different forward weights makes the whole requirement undetermined and can hide a violation on a comparable pair. A run with staleness above zero and more than one update will almost always end undetermined. #79 asks for such pairs to be reported as policy differences and left out of the decision, with the requirement undetermined only when no comparable pair remains. Current observations are also excluded on the assumption that their weights differ after the first step (src/Invar/History.hs:258), while #79 decides comparability from the forward-weight identities alone. The design note says they are reported, but Paths.currents is not used outside Invar.Paths.

Smaller points

  1. Every learner observation records step 0 (src/Invar/Paths.hs:267), whichever optimizer step it comes from.
  2. The per-step policy field repeats what the core already has: each current report carries a state the core checks, and history has every applied step's before. Deriving the identity from those would avoid the worker change and the new probability format, which makes retained runs in the old format unreadable by invar inspect history.
  3. Correspondence and the forward-weight-correspondence premise have no producer, and invar use prepare does not predict that premise as the description says. They may fit better with #84.
  4. Rule.truncation takes the response-length truncated flag rather than distribution truncation, and both sides' rules come from the same sample record, so RuleDiffer cannot occur for history pairs.
  5. learner_engine.path_log_ratio sums over every sample of the generation rather than one path, and withPaths refuses a history with samples outside the contract's domain (src/Invar/Use.hs:92). Both are worth stating in the guide.

Tests #79 still needs

  • the first update of a run with staleness above zero, whose behavior and proximal words are compared;
  • an optimizer step that changes the learner state but not the forward weights, whose current words are compared;
  • policy-difference pairs reported separately while comparable pairs are still decided.

Happy to discuss any of these.

History.paths and rollouts select the workload cycle by the update index
again; the behavior version remains provenance only, so a stale update's
observations stay with the cycle they were rolled out on. Path observations
record prompt, seed, token limit and temperature; use admission checks them
against the declared input.

The contract gains an optional learner implementation. Path admission
compares the learner observation with it and the engine observation with
the declared side, including the assembly; a contract with a training-path
relation without a learner implementation is invalid. use prepare reads the
learner policy from artifacts.learner_policy.

Comparability follows forward weights alone. Current observations are
included; pairs with different weights are reported as policy differences
and left out of the decision, while comparable pairs still decide the
requirement. The learner observation records its real optimizer step.

Probability artifacts return to v5. Current identities come from the core's
current reports, which the core already checks, instead of a worker-written
per-step policy; retained v5 runs remain readable. The path rule keeps the
declared distribution rule and no longer treats response truncation as rule
truncation. The correspondence type and premise are left to FeathBow#84.

Tests add a stale run whose path observations span two cohorts, a policy
difference beside comparable evidence, a current step with unchanged forward
weights, learner assembly mismatch, input mismatch and v5 observations.

Signed-off-by: bohaotu <bohaotu@pony.ai>
@sysuTobo

sysuTobo commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author
  • Added f0fe5e4 addressing the review.
  • Cohort: History.paths/rollouts use the update index again; behavior version stays provenance. New E2E test_a_stale_training_path_uses_the_update_cohort asserts observations cover both cohorts and that a policy difference is reported separately.
  • Learner implementation: the contract now has an optional learner policy (read by use prepare from artifacts.learner_policy); path admission compares learner against it and the engine against the declared side, including assembly. A path contract without learner is invalid.
  • Inputs: each path observation records prompt, seed, token limit and temperature; withPaths checks them against the declared domain input.
  • Comparability: currents are included; forward-different pairs are reported as policy differences and excluded from the decision, comparable pairs still decide. A current step that changes state but not weights remains comparable.
  • Step identity: current observations record their real optimizer step.
  • v5: reverted the worker/format change; current identities come from the core’s current reports/applied state. Retained v5 runs remain readable.
  • Correspondence: removed from this PR; left to policy: publish a generation across artifact formats with a declared conversion #84.
  • Rule: no longer maps response truncation; the history shares one declared rule.
  • Docs: updated for cohort/policy differences, learner implementation, v5, path_log_ratio aggregation, domain refusal and rule scope.
  • Validation: cabal test PASS; HF suite 225 tests OK (2 skipped); shared suite 20 OK; types.sh, fourmolu, hlint, cabal check clean.

@FeathBow FeathBow left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough update. I checked the new commit against each point of the previous review, and all nine are addressed: the cohort uses the update index again, the learner implementation is declared and compared including the assembly, path observations are bound to their declared inputs, policy differences are reported and left out of the decision, current observations carry their own step and the state the core checks, the probability format stays v5, and the correspondence is gone.

A few small things before this leaves draft:

  1. The PR description still describes the earlier revision: the correspondence and its premise in use prepare, probability artifacts v6, the test list, and the note that differing forward weights stay unknown. Please bring it in line with the code.
  2. In docs/design/loop.md:107, "the probability artifact remains v5" reads as a note about this change rather than a description of the system, so it can go. The same paragraph says the learner-state digest is provenance, while the state of a current report is used as the adapter identity; saying that this field is the adapter content digest would avoid the confusion.
  3. Optional test tightening: test_a_stale_training_path_uses_the_update_cohort could assert that cohort 0's pairs are compared and cohort 1's proximal pairs are reported as policy differences. A fromUpdate case where a step's state equals the starting adapter and the bases agree would show such a step being compared through the real derivation; currentComparable builds the identities by hand.

With the description updated, I'm happy to approve.

The stale path contract now asserts that cohort 0 has comparable evidence
while cohort 1's proximal pairs are reported as policy differences. A
fromUpdate case with equal bases and a current step whose state equals the
starting adapter checks that a state-only step is compared through the real
derivation. The loop guide names a current report's state as the adapter
content digest instead of describing the probability format change.

Signed-off-by: bohaotu <bohaotu@pony.ai>
@sysuTobo

sysuTobo commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator Author
  • Updated the PR description to the current code: learner implementation, input/cohort binding, policy differences, v5 artifacts, and the deferred correspondence (policy: publish a generation across artifact formats with a declared conversion #84). No v6 or correspondence claims remain.
  • 6962427 tightens test_a_stale_training_path_uses_the_update_cohort (cohort 0 compared, cohort 1 proximal pairs reported as policy differences) and adds a real fromUpdate case where an unchanged current step is compared with equal bases.
  • docs/design/loop.md now calls a current report’s state the adapter content digest, without the probability-format note.
  • Re-ran cabal test and the focused HF path tests; both green.

@sysuTobo
sysuTobo marked this pull request as ready for review November 2, 2026 09:00
@sysuTobo
sysuTobo marked this pull request as draft November 2, 2026 09:03
@FeathBow

FeathBow commented Nov 2, 2026

Copy link
Copy Markdown
Owner

Thanks for the update. The earlier fixes look good, but I found two more issues:

  1. Missing path evidence can still pass. With four inputs in the contract, supplying path evidence for just a1 still returns admitted. Only supplying none returns unknown. Please check coverage before excluding policy differences: missing evidence must leave the requirement undetermined.

  2. Path observations lose their call identity. Changing the learner’s call, attempt and instance produces exactly the same observation and scope. The provenance keeps the update, step and sample, but cannot identify the execution that produced the words. Please retain the admitted call and source log identity in the observation and its scope.

Both cases reproduce on the current branch; the existing Paths and UseAdmission tests pass.

Path claims now check that every declared input has at least one attached
observation before deciding anything. An input without evidence leaves the
requirement undetermined, even when another input would be refuted, and policy
differences are excluded only after coverage is complete.

Each path observation now records the admitted call, attempt and instance plus
the SHA-256 of the source log that produced its words. The learner side names
the admitted update and the engine side names the admitted rollout, so changing
either execution changes the observation and its scope.

Test coverage: a partial history with only a1 attached stays unknown for both a
satisfied and a violated budget, and a changed learner binding changes the
observation and scope while the engine execution remains stable.

Signed-off-by: bohaotu <bohaotu@pony.ai>
Signed-off-by: bohaotu <bohaotu@pony.ai>
Path implementation checks now cover every attached observation, including
pairs excluded from the decision as policy differences. A foreign learner or
engine implementation therefore leaves the requirement undetermined instead of
being admitted through another comparable pair.

Engine executions are keyed by the probability sample name and checked against
the admitted inference's behavior words, so a reordered or mismatched cohort
cannot silently attach the wrong call or source log. Path judgments also reject
observations whose declared inputs differ.

Test coverage: a policy-difference observation with a foreign implementation is
undetermined, a path contract without a learner implementation is invalid,
duplicate path keys and missing engine executions are refused, and the HF
history E2E asserts both sides carry call, attempt, instance and log digests.

Signed-off-by: bohaotu <bohaotu@pony.ai>
@sysuTobo
sysuTobo marked this pull request as ready for review November 2, 2026 13:52

@FeathBow FeathBow left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both issues are fixed. I re-ran the original repro cases and the focused tests; all passed. Looks good to me.

@FeathBow
FeathBow merged commit 1665f9f into FeathBow:main Nov 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants