[professor3333]

Summary

reports/neural/run-2/: the second GPU run (Kaggle background version, one T4). The distilbert-2 and deberta-2 grids were run with span weighting {sqrt, none} × learning rate {2e-5, 5e-5} × 3 seeds, 24 runs. Scores are against the 172 model-drafted dev records (agreement, not accuracy).

• DeBERTa-v3-small (sqrt, 5e-5):
  • typed F1 0.131 ± 0.001 across seeds; the best run scores 0.133 [0.108, 0.164];
  • against crf-batches-01-02: typed F1 −0.013 [−0.063, +0.043], not clear;
  • clearly worse on span precision (−0.224) and on the false-prerequisite rate (+0.254 [+0.188, +0.343]): about 8 conditions per document against 1.8.
• DistilBERT: best 0.075, clearly worse than the CRF (−0.070 [−0.113, −0.020]).
• Uniform span weights (none): 11 of 12 runs predicted nothing after epoch 1 and were stopped at epoch 4 by patience. This is an early-stopping artifact, not a test of the setting; run 3 needs a minimum epoch count.
• Hand-labelled slice (10 docs): no clear differences.
• Caveat: the transformer numbers are optimistic. Their epochs and runs were selected on the same 172 records, and the CRF was not tuned on them.
• Committed: per-run metadata, all reports, and the predictions of the two best runs (the paired comparison reproduces from them). README results table and model card updated.

The test split is not touched.

Checks

• ruff check, ruff format --check, pytest -m "not slow" (238 passed); the README reproduce command was re-run and matches.