Repository navigation
feat: Let each policy step on its own clock - #94
Merged
Merged
Conversation
ApGa
force-pushed
the
apga/train-lockstep
branch
from
October 28, 2026 23:06
8cf1fa1 to
09feb74
Compare
Several trained policies stepped together: each step trained every
policy on its rows of the same groups, and each checkpoint saved them
all, so a policy that trains on few rows, or saves fast, waited for the
slowest (RFC 0021's lockstep). train.stepping: independent lets each
trained policy step, publish and save on its own clock (RFC 0021,
section 2.11, and RFC 0003's milestone M2), with its own batch
(policies.<id>.batch_size, else train.batch_size), staleness bound and
checkpoint cadence in its own steps (policies.<id>.checkpoint: {every,
recovery_every}, else checkpoint.*). lockstep stays the default, and
with one trained policy both are the same. A trainer's needs say which
clocks the run builds (TrainerNeeds.stepping; the rl trainer answers
with train.stepping), and run.clocks holds each policy's.
Each group is credited once, over every agent, and then splits into a
share per trained policy with rows in it, and always one of the
leader's: that policy's rows of the group (none of the leader's in a
group without them, so that the leader's steps take every group and the
plan stays in draws, train.batch_size groups per leader step; a group
that failed is the leader's share alone). The run's share ledger
(dispatch/shares.py) takes the result queue's place: each policy's steps
take its shares in the order they settled, judge them against its own
version and bound, train them with its own learner, record them consumed
with their age and fate, and publish that policy alone, after each of
its steps. The rl trainer runs one loop per policy, each step the
lockstep step with the policy's one lane (step/independent.py); a
trainer of its own takes shares through platoon.experimental.ShareLedger
(take, which stops on run.shutdown; judge_version, the version a share
that a resume put back is judged at; consumed under its lock; done) and
ends each step with run.end_step(policy=...). The dispatcher keeps a
horizon per policy, (k + 1) x batch of its shares, and starts a group
only when every one has room (dispatch/horizons.py); the leader's
remaining steps bound the plan, as in lockstep.
Every full save of any policy is a commit (checkpoints/commits.py): it
captures, under a commit lock that each group's settle section and each
step's consumption take too, and once no step's end runs its step hooks
(so that it saves no hook's state part-way), the next commit_seq, the
run's draws (the groups not yet retired, with what each policy did with
its share) and every other policy's newest durable row and published
version, which stay pinned against the pruners; then writes them and the
policy's weights; then appends the policy's row to its own
checkpoints.jsonl, naming every policy's row by its state_path. Another
policy than the leader names its commits with its id
(recovery-<id>-<step>), apart from the barriers' rows in its file, and a
resume refuses a state path that two rows of a file hold
(trainer/clocks.py keeps each policy's clock and cadence). A group stays
kept until each policy that consumed its share has saved after that
step. A resume takes the commit with the highest commit_seq whose named
rows hold (a deleted one is skipped and recorded in lock.json), each
policy going on from the row it names, at one version more than the
commit records it published when that is past the row's; each share it
consumed after that row trains again first, judged at the age it had.
commit_seq goes on after the highest in any file, and the launch and
exit saves are barriers of every policy whose leader row is a commit.
Each step logs a row of its own without a W&B step (StatsLogger.log_row,
which declares each prefix's axis): main's with its names and run/step,
every other policy's under policies/<id>/ with policies/<id>/step;
policies/<id>/share_wait_s, retrained_shares and peer_lag/<q>,
run/retained_groups and train/dispatch/blocked_share/horizon:<id> are
new. Evaluations count main's steps and record versions_end when a
policy published while they ran. Step hooks run at the end of each
policy's own step (StepEnd.policy) and may update that policy only,
through their end (the run's learners send nothing from a hook); a
commit saves a hook's state as of its capture while the other policies
resume from older rows, so a hook keeps what moves at a policy's step
ends by that policy's step (docs/extending/step-hooks.md shows how). A
stop request ends each loop at its step's end; once a stopped policy's
horizon is full, a step that still waits for shares closes short, and
the exit save names every policy. A start that resumes with the leader
at the plan's end trains the other policies' last shares and saves the
final checkpoint.
Loading refuses a policy's batch_size or checkpoint in lockstep (the
refusal named this milestone; it now says to set train.stepping:
independent), on a frozen policy, a batch that the step's minibatches
and microbatches do not divide, a group admission rule under independent
stepping, and a checkpoint.keep_recovery that a policy's own
recovery_every (else the run's), in its own steps, does not span past
its bound, and warns of a trained policy that saves no full checkpoint
before the run's end; each start checks the same against its trainer's
needs. identity.json records stepping: independent, and a resume that
changes it, or whose rows were saved with the other stepping, stops with
exit code 78. examples/two-agent-independent steps the subagents
example's worker about four times for each of its delegator's steps,
with a CPU test that kills it once and resumes. With train.stepping
unset every run is unchanged: the golden trace is byte-identical.
ApGa
force-pushed
the
apga/train-independent
branch
from
October 29, 2026 01:24
d3b3cd9 to
c0598db
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #91 ("feat: Train several policies in lockstep"); review that first.
This is RFC 0021's independent stepping (section 2.11, scenario S7 (m)):
train.stepping: independentlets eachtrained policy step, publish and save on its own clock, with its own batch, staleness bound and checkpoint cadence,
and no policy waits for another's training or saves.
lockstepstays the default. Withtrain.steppingunset nothinga run does changes: the golden trace is byte-identical, every effect of its scenarios and their variants is the same,
in the same order, but for calls of pure functions, and the lockstep runs of two policies (the previous PR's tests and
its example) are unchanged too (below).
What it adds
platoon/train/:config_defs.py,policy/spec.py,identity.py,trainer/api.pytrain.stepping(locksteporindependent);policies.<id>.batch_sizeandpolicies.<id>.checkpoint: {every, recovery_every}(PolicyCheckpointConfig; another key is refused by name), which reach the policy's spec; the refusal the lockstep PR gave them now applies to policies that step together only, saying to settrain.stepping: independent; besides: a frozen policy's, and, stepping independently, a batch that the step's minibatches and microbatches do not divide and a group admission rule (stepping_problems, which loading asks oftrain.steppingand each start again of its trainer's needs); a warning for a trained policy that saves no full checkpoint before the run's end;steps_independently(config);identity.jsonrecordsstepping: independent, and a resume that changes it stops with exit code 78;TrainerNeeds.stepping, which the rl trainer answers withtrain.stepping; undertrainer: sft, which trains one policy,train.steppingis refused as the othertrain:keys only the rl trainer reads areplatoon/train/dispatch/shares.py(new)ShareLedger, whichplatoon.experimentalexports: what a trainer of its own takes each policy's shares from, a protocol of five names:take(policy, n, stop=...)(the shares as settled groups, as the source'stakegives them in lockstep;TrainInterruptedoncestopis set),judge_version(the version to judge a share's staleness at: for a share a resume put back, the one at which it has the age it had when first consumed),consumed,doneandlock.Shares, the run's ledger: each settled group, after credit, split into a share per trained policy with rows in it, and always one of the leader's (with none of its rows in a group without them, so that the plan counts every group; a failed group is the leader's share alone, so failures throttle dispatch as in lockstep); each policy's share queue for the rl trainer's steps (queue,PLAN_ENDonce no share will come);consumed(each share's step, age and fate; a group's fate goes to the per-task statistics once, when its last share is consumed);durableand retirement (a group leaves the kept groups and the outstanding draws once every consuming policy has a row after the consuming step);snapshot,restore,revert(a resume puts back each share a policy consumed after its row, first, with its recorded age);stopped(a loop a stop request ended); the commit lock. The module imports neither Tinker nor torchplatoon/train/dispatch/:horizons.py(new),dispatcher.py,setup.py,source.py,runner.py(k + 1) x batchof its shares, counting the running groups and its shares queued and taken; a group starts only when every one has room, the leader's remaining steps bound the plan, and each applies the dispatcher's held-slot rule with its ownkand batch; once a stop request ended a policy's loop and its horizon is full, no group starts again (held_for_good), and a step that still waits for shares closes short, as at the plan's end;train/dispatch/blocked_share/horizon:<id>(andhorizon, their sum); the dispatcher hands settled groups to the ledger (SettledSink); the source builds the horizons and the ledger when the run's policies step independently, captures its draws for a commit (capture), and puts back shares on resume; the runner's settle section (_settle_group) takes the commit lock under independent steppingplatoon/train/checkpoints/:commits.py(new),participants.py,barrier.py,resume.py,saving.py,kept_groups.py,dispatch_state.py,pruning.pyCommits: every full save of any policy a commit: capture under the commit lock, which the ledger takes once the draws open (attach), once no step's end runs its step hooks (hook_phase: no commit saves a hook's state part-way through its work at another policy's step end, rule 10), withcommit_seq, the participants'captureorcheckpoint_state, every other policy's newest durable row and published version, pinned; write (a writing that fails closes those after it, unrun); append to the policy's owncheckpoints.jsonl; the launch and exit barriers carrycommit_seqand every policy's entry (published_version,step);RunCheckpoints.commit;newest_commit(the highestcommit_seqwhose named rows hold: a row deleted, or whose state path another row of its file holds too, is missing; newer ones are skipped and recorded inlock.json; a run saved in lockstep is refused by name),resumed_version(the version rule),next_commit_seq;StartPoint.read(..., independent=True)andPolicyStart.step; kept groups recordsharesandrecorded; each pruner skips the pinned rows, checked again before each deleteplatoon/train/:step/independent.py(new),trainer/clocks.py(new),step/microbatch.py,step/staleness.py,policy/set.py,trainer/loops.py,trainer/rl.py,trainer/run.py,trainer/state.py,trainer/waiting.pyPolicyLoop), each step aPolicyStep: the lockstep step with the policy's one lane, its shares from the ledger in place of the result queue, its batch, its clock, reverted shares judged at their recorded age, its consumption recorded under the lock, its own cadence (a commit, else weights only), one row of metrics; a stop request ends each loop at its step's end, and a loop that stopped tells the ledger;OwnClocks(trainer/clocks.py): each policy's clock, its own checkpoint cadence and the names of its commits (another policy's carry its id,recovery-<id>-<step>, apart from the barriers' rows in its file, which one Tinker session saves too), what the exit save records of each;run.clocks,run.end_step(policy=...)andrun.publishpublish and commit one policy on its own cadence (the run's_save_duewith the policy whose step ends, also forend_step(sampled=False); the leader's recovery commit goes to the trainer's_recovery_saved, as a lockstep step's end hands its recovery save; the rl trainer's steps commit throughown_step_samplerand hand their saves to their own pruners); a start checks the stepping rules against its trainer's needs (_check_stepping) and that its run's rows were saved with the same stepping (_check_resumed_stepping: a commit resumed by policies that step together exits 78); a start that resumes with the leader at the plan's end (a stop or a crash after its last commit) trains the other policies' last shares and saves the final checkpoint (_save_at_exit,OwnClocks.others_stepped)platoon/train/trainer/hooks.py,platoon/train/evaluation/,platoon/utils/stats_logger.pyStepEnd.policy: hooks run at the end of each policy's own step and may update that policy only (end.learnerof another is refused, and the run's learners send nothing from a hook,run.learnerof any policy);StepHooks.running, held while a step's end runs its hooks (nothing in lockstep, the commits' hook phase when stepping independently); evaluations recordversions_endwhen a policy published while they ran, and log a row of their own;StatsLogger.log_row: a row without a W&B step of its own, each metric prefix declared once against its row's axis (define_metric), a row on the run's axis withtime/elapsed_secondsexamples/two-agent-independent(new)main, batch 8) and its workers (worker, batch 2, bound 7, a commit at every 4th of its steps), about four worker steps per main step; its CPU test validates the config, runs it on the fake service, kills it once at the worker's second commit, resumes, checks each policy's steps, rows and that no policy's weights trained a row twice, and checks that a resume in lockstep stops with exit code 78How a policy's step goes
Where the code differs from the design, and why
refill.max_groups_per_stepper policy; a rule's markers, caps and short steps are counted for steps of everypolicy at once in this release's dispatcher. Loading says so; lockstep keeps every rule.
group's report and record; a failed group, or one with no trained policy's rows, is the leader's share alone. The
design says a group without rows of
pis no share ofp's, and that the plan stays in draws,end_step x train.batch_sizegroups: both cannot hold for the leader, whose steps carry the plan. A group without its rows wouldcount toward no step of the plan, and the run would draw more groups than planned (the verification measured 25
draws for a plan of 20); with no share at all a failed group would leave every horizon at once, and an outage would
start groups as fast as they fail. As the leader's share, each takes a place of the leader's step, as a lockstep step
takes such a group. The group's fate in the per-task statistics goes by the shares that hold rows (stale when all of
theirs went stale, as a lockstep step judges it). A trainer of its own sees such shares: an empty
rollout(Nonefor a failed group).
main's metrics keep their names (the lockstep PR's choice); every other policy's row is today's names underpolicies/<id>/, and the new per-policy metrics arepolicies/<id>/...for every policy,mainincluded.(when it has any), just before its policy's own save and publication, rather than in the step's synchronous commit
stage (
PolicyStep.commitdoes nothing), since taking the lock awaits, and the lockstep step'scommitstayssynchronous, its calls pinned by the comparison below. A commit of another policy captured in between records the
shares as not consumed and names this policy's previous row, so a resume from it trains them again, once.
the design says: only shares consumed after a policy's row keep their recorded age. A resume that bumps the version
(published + 1) and retrains several shares first can make such a share stale under a tight bound; the tests check
every share trains once per lineage and every share put back trains again, and the docs ask for small
recovery_every.only that policy's; the rl trainer's steps are not atomic, so it saves every policy at exit.
policies.<id>.checkpoint.everycounts steps only, not epochs: a policy other than the leader has no epochs ofits own. Left out, it is the run's
checkpoint.everyin steps.step that applied an update and waits for more shares when another policy's loop has stopped would wait for good once
the stopped policy's horizon is full (its shares are never taken); starting groups beyond that horizon instead would
give the stopped policy shares older than its bound after the resume. So the step closes short with the shares it
has, as at the plan's end, and counts as a step of the plan, which then draws that many fewer groups.
with no await; a hook's state (
keys) is captured there too, so without the wait a commit could save it part-waythrough another policy's step end, which rule 10 forbids. A commit's own step end publishes between its hooks'
phases, so it never waits for itself; it waits for the others' hooks.
train.stepping; each start checks the same rules against its trainer's needs. The design lets atrainer of your own answer the stepping from its own options. Loading cannot know the trainer, so it reads
train.stepping(the rl trainer's answer, the only trainer a config selects in this release); each start checks therules again against
needs().steppingbefore any service opens (a policy's own batch or cadence under a trainerwhose policies step together, a group admission rule or an undividable batch under one whose policies step
independently), and refuses to resume a run whose rows were saved with the other stepping (exit 78). A trainer of
your own whose needs say independent sets
train.stepping: independentto give a policy its own batch or cadence;identity.json, whichplatoon trainwrites for the rl trainer, recordstrain.stepping.platoon.experimentalexports one name,ShareLedger, a protocol of what a trainer calls:take(which, asthe design's, returns the shares as settled groups),
judge_version,consumed,doneandlock. A settled groupcarries no age, so
judge_versiongives the version a trainer that drops stale shares judges each at: a share aresume put back has the age it had when first consumed, as the rl trainer's steps judge it. The ledger's other methods
(
put_nowaitfor the dispatcher,durable,revert,snapshot, ...) belong to the run's implementation,Shares, which is not exported;takegainsstop=, as the source'stakehas.state (
keys) as it is then, while each other policy resumes from the older row the commit names and runs its stepends since that row again: state that moves at a policy's step ends would be ahead of that policy's weights. The
step hooks' guide says so and shows the pattern, keeping such state by each policy's own step and taking it up at
start.policies[<id>].stepinresume(); a test runs that pattern across a kill. A per-policy hook state that therun keeps for each commit is left for later.
recovery-<id>-<step>,<id>-<step>): thebarriers' rows in its file carry the leader's step, one Tinker session saves both, and a name of its own step alone
could save over a barrier's row when it is behind the leader. A resume also refuses to name a row whose state path
another row of its file holds.
With
train.steppingunsettrain.stepping,TrainerNeeds.steppinglockstep: one clock (run.clocksmaps every policy to it)policies.<id>.batch_size,.checkpointtrain.stepping: independent)StepClock.policy,StepEnd.policyNoneRunCheckpoints.savecommit_seq, the same entriesDispatcherhorizonsargument), counters and checks;binditerates a copy of the kept groups_settle_group), no lockTrainStep._publishidentity.jsonsteppingkeylog, as beforestepping_problemswithindependent=Falseand the resumed row'scommit_seqlooked up, both pure, at phases 1 and 3StepHooks.runningiscontextlib.nullcontext, and the run's learners check the run's clock alone, as beforekeep_recoverycheck at loadingtrain.stepping: every policy's bound against the run'srecovery_every, as beforeTests
tests/train/policy/test_stepping.py: the key, a policy's own keys and every refusal (an unknown key of its cadenceby name, a
keep_recoverythat its ownrecovery_everydoes not span past its bound), the warning for a policy thatsaves no full checkpoint before the run's end, the rl trainer's answer,
identity.jsonand a resume that changestrain.stepping(exit 78 throughplatoon trainin the example's test).tests/train/dispatch/test_shares.py: the split (a group without the leader's rows a share of the leader's with none,counted toward the plan, its fate its rows'), the fate recorded once, retirement, revert with recorded ages, the
plan's end, a trainer's
takestopped by a stop request (the shares it took stay unconsumed),judge_version(theversion now for a fresh share, the recorded age's for one a resume put back), a queue that ends once a stopped
policy's full horizon holds every group back, the horizons and their blocked shares.
tests/train/checkpoints/test_commits.py: the highestcommit_seqwhatever the order of the appends, the next numberafter it, a deleted named row skipped and recorded, a commit naming a state path that two rows of a file hold
skipped, a run saved in lockstep refused, the version rule, pins while in flight and once newest, a commit whose
writing fails (the writings after it closed, unrun; nothing pinned), and the ledger taking the commit lock that a
commit waits on for the step hooks, also once a step's end ran its hooks before the draws opened.
tests/train/trainer/test_independent_run.py(two fake services,tests/train/fakes/independent.py): stepping 1:4,each policy's own rows and cadence (the worker's commits named with its id), each share trained once in each policy's
weights, groups without
main's rows counted toward the plan (the run draws its 12 groups, each row trained once intoits policy), the metrics' rows and axes, a resume from the newest commit, an interrupt with one policy's step open,
and with a step of two updates whose shares the stopped policy's horizon holds back (it ends, closing the step short;
the resume trains each share once, none stale), a stop in
main's last step with a slow worker (the resume,mainatthe plan's end, trains the worker's last shares and saves
final), step hooks on a policy's own step and the refusalof another, through its end or through
run.learner(under the rl trainer and a trainer of its own, whose own stepssend through
run.learnermeanwhile), no commit saving a hook's state part-way through another policy's step end, ahook whose state follows each policy's own step across a kill after a commit that captured it past the worker's row
(each of its examples trains once, in order; also keeping only the cursors a commit can name, at most its cadence's
and one in every row), a trainer of its own through the ledger and stopped through
take(stop=...), one that dropsstale shares at
judge_versionacross a resume whose commit names the worker's row four or more versions old (noshare put back is dropped), a lockstep trainer's refusal of a policy's own batch, a start that keeps its run's
stepping (a commit resumed by policies that step together, and a lockstep run by independent ones, exit 78), and a
group admission rule refused against a trainer's needs.
tests/train/checkpoints/test_commits_run.py(two fake services, whose clients remember the lineage of each savedstate): a kill at every point between consume, capture, write, save and append, of either policy (each share once per
lineage, every put-back share trained again), also of a worker slower than main after a resume (its commits never save
over its launch row); a commit that names the worker's row four of its steps old (every share put back trains again
first, none dropped as stale); a kill right after
main's last commit with a slow worker (the resume savesfinal,every share once, and a start after it trains nothing); a pruner that runs while a commit is in flight (the named row
survives; a resume skips a commit whose named row was deleted, and records it); commits appended out of
commit_seqorder (the resume takes the highest; the next commit takes the number after it); a stateful credit estimator killed
between a group's credit and its settling (each group settled once).
tests/examples/test_two_agent_independent_example.py,tests/train/evaluation/test_eval_versions.py(
versions_end),tests/utils/test_stats_logger_rows.py(the run's rows carry the elapsed time); the lockstep, stephook, lifecycle, layers and credit workflow tests follow the new fields (
StepEnd.policy,run.clocks, thesource's
ledger,trainer/clocks.py) and the lockstep refusal's new message, and the sft trainer's refusal of thetrain:keys only the rl trainer reads includestrain.stepping.Checks
tests/train/paritypasses, its 52 golden files byte-identical.earlier refactors (faults and stops included), run twice on the base: every eqtrace event (the trainer's calls, the
dispatcher's, the statistics', the trackers', each await chain), logger names included, the golden-style files and
the refactored code's external calls are the same, in the same order. What differs is calls of one pure function,
getattr: the steps and the run asking whether the run has a clock per policy (run.clocks, the source'sledgerand
horizons). The await chains leave out the frames of the modules this PR changes (the dispatcher,waiting.py,the step hooks) and of its new ones, as the earlier comparisons did for theirs: their line numbers moved.
worker's late datum, a worker without rows, two starts) and its example under both configs, run on the base and
here: their checkpoint rows, metrics, publications and what each policy trained are the same, but for timings and
the checksums of snapshot files.
extensions, the recipes, the docs (the generated references and the public API's snapshot up to date) and the
release build. The full suite passes (8,716) but for one test that needs a git checkout, which fails on the base
the same way in that runner.
tyreports nothing new, the dependency rule holds, every patch a test makes still reaches the code it patches(359), every name of the 0.4.0 module paths still imports through them, and a patch made through an old path right
after its import reaches the policies' loops too (
platoon.train.step.independentis imported with the trainer).Every function touched stays within 60 lines but
Dispatcher.__init__, 95 lines at the base and 100 here (itshorizonsargument).tests/train/,policy/test_stepping.py14,dispatch/test_shares.py10,checkpoints/test_commits.py8,checkpoints/test_commits_run.py20,trainer/test_independent_run.py19 andevaluation/test_eval_versions.py1;tests/utils/test_stats_logger_rows.py2; the example's 3 (and its config inthe examples' no-defaults check). The lockstep refusal's test takes its new name. Each test module this PR touches
also passes alone, from the repository's root and from
tests/. The whole-start modules (the commits', theindependent runs', the example's, the ledger's) passed 16 times in a row with 16 at once on one node, and 4 times
under
pytest -n 8.variants of the slow worker's kill, at its consumption and before its save, reach no collision and pass there too),
and the reviewers' own probes run again here: a worker slower than main whose own commit saved over its launch row
(12 shares trained twice after a resume) now names its commits apart; an interrupt in a step of two updates whose
shares the stopped policy's horizon held back no longer hangs (32 stop points, each resumed: none hangs, none trains
a share twice, none drops one as stale); no commit saves a step hook's state part-way through another policy's step
end; a trainer's needs that say independent with
train.steppingat lockstep are held to the same rules, and alockstep resume of its run stops with exit code 78; a trainer of its own that takes shares with
stop=run.shutdownstops on a stop request; the run's rows carry
time/elapsed_seconds; an unknown key of a policy's cadence is refusedby name.
there; the hook test passes there, since its fix is the guide's pattern, and fails with a hook that takes its state up
as the commit saved it), and the verification's probes run again here, on two and three fake services, without the
probe-only patch it used:
mainat the plan's end, after a kill, a stop or a failure that came aftermain's last commitwith another policy behind, now trains that policy's last shares and saves
final: 1,114 kills (32 leave nofinal row, each a resume with nothing left to train, as the base's run of one policy), 357 stops, 155 failures,
21 runs of a trainer of its own and the 3 leader-at-the-end cases each train every share exactly once.
judge_versiontrains every share a resume put back (7 of 7,each of which it dropped before; judged at the version now, the new test fails).
policies; one that takes its cursor up as saved loses examples in each, the hazard the guide now describes.
train/dispatch/blocked_share/horizon:<id>,train/workload/*), and the how-to's metric tables list the new ones.take's docstring (fewer shares after a stop too), the policy spec's comments onbatch_sizeandcheckpoint(the reference had none forcheckpoint), one commit lock that the ledger takes (Commits.attach;the condition a commit waits on was bound to the lock of the moment), and a commit whose writing fails closes the
writings after it, unrun. Left:
PolicyStep.commitdoes nothing (above), and the private namesstep/independent.pyuses from other modules, which the open cleanup PR renames or moves.there; the two cases of the hook-state test pass there, since that fix is the guide's example, which the new case's
hook follows), and the verification's probes that found them run again here:
checkpoint.keep_recoverywas checked against the run'srecovery_everyfor every policy, while each policy'spruner keeps its own bound at its own cadence: a worker saving every 3 of its steps with bound 7 and keep 3 was
refused, and one saving every step with bound 3 and keep 1 under the run's cadence of 4 was accepted, its pruner
then keeping more than
keep_recovery. Stepping independently, the check now counts a policy's ownrecovery_every, against its own bound (else the run's largest), and namespolicies.<id>.checkpoint.recovery_every.main's rows counted toward no step of the plan, so the run drew more groups than planned (17 for aplan of 12 in the new test, 25 for 20 in the probes). Every group is now a share of
main's (above): in the probes'runs whose every fifth group has no rows of
main, killed or not, the run draws its plan exactly (20 of 20, 16 of16) and every drawn group with a policy's rows trains into that policy once; a step of
mainwhose shares hold noneof its rows takes no update, as a lockstep step of such groups.
run.learner, whose guard looked atthat policy's clock only: the run's learners now send nothing from any step end's hooks (
RuntimeError), under therl trainer and a trainer of its own, whose own steps send through
run.learnermeanwhile as before.drops those no commit can name any more, older than the policy's full-checkpoint cadence.
judge_versionandShare.judged_atused it twice), andthe policy's view of the loops' state says why its leader is the policy whose steps run. Left:
PolicyStep.commitdoes nothing (above).
random time) of step hooks, trainers of their own, a change of stepping, groups without
main's rows, the bypass,and pruners that delete as commits run (
keep_recovery: 2): none trains a share twice or loses one, none judges ashare put back stale, no commit captures a hook part-way, and the final per-task statistics count each planned
group once. What the checks flag the base does too: launch rows written after
finalby a start planned after therun ended (35), states a pruner deleted after a start opened them (568), and runs killed after
main's last commitwith nothing left to train (5). The verification's prune probes with
keep_recovery: 1stay refused at loading, bythe run's own rule for
main._save_due(a trainerof its own commits there now), the verification's probes of the save and commit paths ran again: the commits'
crafted run directories, trainers of their own, out-of-order appends, a change of stepping, and samples of the kills
and the stops, 1,167 chains (2,257 starts, 823 kills at a point and 20 at a random time). None trains a share twice
or loses one, none judges a share put back stale, and the final per-task statistics count each planned group once;
what the checks flag the base does too (13 launch rows after
final, 262 states a pruner deleted after a startopened them, 7 runs killed after
main's last commit with nothing left to train).