Repository navigation
Draw batch-05 by CRF uncertainty; split CRF spans at line breaks - #34
Merged
Merged
Conversation
A tagged run that continued onto the next line produced a condition the validator rejects (span_crosses_line), which stopped predict for the whole run. The CRF now splits such a run into one condition per line, as the transformer decoder already does. Dev predictions are unchanged.
183 documents where crf-batches-01-02 is least certain plus a 78-document natural slice, at most 3 per leakage group. Uncertainty manifests now record the predictions that chose them (file, SHA-256, model ids).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
span_crosses_line). That stoppedpredicton the train pool. Such runs are now split into one condition per line, as the transformer decoder already does. New test; dev predictions inreports/agreement/are byte-identical.annotations/batches/batch-05.json): 261 unsampled train-pool documents, 183 wherecrf-batches-01-02is least certain and a 78-document natural slice, at most 3 per leakage group. This completes the planned ~500 train documents.reports/sampling/crf-batches-01-02_train_pool.jsonl(train-pool documents only), and re-drawing reproduces the manifest exactly.docs/annotation_plan.mdupdated.Checks
ruff check,ruff format --check,mypy src/,pytest -m "not slow"(235 passed).