professor3333 · Nov 6, 2026 · Nov 6, 2026
diff --git a/README.md b/README.md
@@ -156,7 +156,7 @@ item can be highlighted in the original advisory.
 | 1 · Feasibility | Pinned ingestion, corpus statistics, label schema, offset-preserving preprocessing, pilot sample, annotation tooling | ✅ built |
 | 1 · Pilot | 100-document pilot (`pilot-2`: 22 hand-labelled, 78 from model drafts), [decision gate](docs/pilot_gate.md), guidelines v0.2 frozen, [acceptance targets](docs/targets.md) declared | ✅ done |
 | 2 · Baselines | Rules baseline, CRF + logistic-regression attributes and groups, evaluator with bootstrap CIs, evidence viewer | ✅ built · preliminary pilot dev scores |
-| 3 · Neural models | Frozen leakage-safe splits, annotation batches (01–03, 06: model-drafted), DistilBERT and DeBERTa-v3 with span / role / group heads, experiment runner | 🟨 code ready · not yet trained |
+| 3 · Neural models | Frozen leakage-safe splits, annotation batches (01–03, 06: model-drafted), DistilBERT and DeBERTa-v3 with span / role / group heads, experiment runner | 🟨 trained on model drafts (run 2) · CRF still stronger on invented prerequisites |
 | 4 · Reliability | Calibration and review policy, formatting robustness (rules: 100%), challenge-set and error-catalogue tooling | 🟨 tooling ready · thresholds await labels |
 | 5 · Application | FastAPI API, review UI, SQLite (WAL) with append-only history, exports, Docker Compose stack, release manifests, tested rollback, CI image smoke test | ✅ built |
 | 6 · Production | Benchmark harness, ONNX/INT8 export (FP32 parity exact), local proxy benchmarks and load tests, monitoring; target-host numbers and hosted release pending | 🟨 harness and proxy results |
@@ -185,13 +185,23 @@ cannot be judged as declared.
 | Rules (`rules-2`) | 0.107 [0.058, 0.154] | 0.142 [0.085, 0.195] | 0.330 [0.165, 0.467] |
 | CRF (`crf-pilot-2-batch-01`) | 0.128 [0.095, 0.168] | 0.174 [0.134, 0.215] | 0.416 [0.294, 0.493] |
 | CRF (`crf-batches-01-02`, + 100 model-drafted `batch-02` records) | 0.145 [0.105, 0.185] | 0.221 [0.171, 0.268] | 0.353 [0.260, 0.416] |
+| DistilBERT (`distilbert-2`, best of 12 runs, same training data) | 0.075 [0.051, 0.105] | 0.101 [0.077, 0.131] | 0.541 [0.478, 0.601] |
+| DeBERTa-v3-small (`deberta-2`, best of 12 runs, same training data) | 0.133 [0.108, 0.164] | 0.165 [0.142, 0.194] | 0.607 [0.565, 0.640] |
 
 Adding `batch-02` clearly improves exact-span agreement (+0.047, 95% CI [+0.004, +0.086]), but not
 typed agreement. On the 10 hand-labelled documents, `crf-batches-01-02` scores 0.370 typed F1
 [0.000, 0.640], and the difference from `crf-pilot-2-batch-01` is not clear; see
 [`reports/agreement/`](reports/agreement/README.md). Its disagreements with the drafts, by
 error type, are in [`docs/error_catalogue.md`](docs/error_catalogue.md).
 
+The transformers were fine-tuned on one T4 GPU, with runs chosen on these model-drafted dev
+records only. DeBERTa is not clearly different from the CRF on typed agreement (−0.013, 95% CI
+[−0.063, +0.043]). It is clearly worse on the false-prerequisite rate (+0.254 [+0.188, +0.343])
+because it predicts about 8 conditions per document against 1.8. DistilBERT is clearly worse
+on typed agreement (−0.070 [−0.113, −0.020]). The transformer rows are optimistic: their
+epochs and runs were selected on these same records, and the CRF's were not. See [`reports/neural/run-2/`](reports/neural/run-2/README.md);
+the first run's failed configuration is recorded in [`reports/neural/run-1/`](reports/neural/run-1/README.md).
+
 95% project-group bootstrap intervals (1,000 resamples). Full reports, including scores against
 the 28 model-made dev records (which measure agreement with the drafting model, not accuracy),
 are in [`reports/pilot/`](reports/pilot/README.md). The paired difference from adding `batch-01` (+0.214 typed F1,

diff --git a/docs/model_card.md b/docs/model_card.md
@@ -28,8 +28,8 @@ reviewer confirms or corrects every item before export.
 |---|---|---|
 | `rules-2` | Deterministic cue rules over the shared preprocessing (no learning) | Runs; 100% formatting-consistent on 300 documents |
 | CRF | `python-crfsuite` BIO span tagger with logistic-regression category, role and group classifiers. Features `crf-features-1`, optional spaCy part-of-speech tags | Trained on pilot data only |
-| DistilBERT | `distilbert/distilbert-base-cased` (Apache-2.0), revision `6ea81172465e8b0ad3fddeed32b986cdcdcffcf0` | Run 1 failed (span over-prediction, [`reports/neural/run-1/`](../reports/neural/run-1/README.md)); run 2 pending |
-| DeBERTa-v3-small | `microsoft/deberta-v3-small` (MIT), revision `a36c739020e01763fe789b4b85e2df55d6180012` | Run 1 failed (span over-prediction, [`reports/neural/run-1/`](../reports/neural/run-1/README.md)); run 2 pending |
+| DistilBERT | `distilbert/distilbert-base-cased` (Apache-2.0), revision `6ea81172465e8b0ad3fddeed32b986cdcdcffcf0` | Run 2: best agreement typed F1 0.075 (CRF 0.145), [`reports/neural/run-2/`](../reports/neural/run-2/README.md) |
+| DeBERTa-v3-small | `microsoft/deberta-v3-small` (MIT), revision `a36c739020e01763fe789b4b85e2df55d6180012` | Run 2: best agreement typed F1 0.133, false-prerequisite rate 0.607 (CRF 0.353), [`reports/neural/run-2/`](../reports/neural/run-2/README.md) |
 
 The transformer models share one design:
 - the encoder is fine-tuned, never pretrained;

diff --git a/reports/neural/run-2/README.md b/reports/neural/run-2/README.md
@@ -0,0 +1,112 @@
+# Neural run 2 (2026-11-06)
+
+The second GPU run of `scripts/gpu/run_neural.sh`, a Kaggle background version on one Tesla T4.
+It covers the `distilbert-2` and `deberta-2` grids:
+- two span-tag weightings (`sqrt`, `none`) × two learning rates (2e-5, 5e-5) × three seeds, 24
+  runs;
+- up to 15 epochs, with patience 3.
+
+The other settings match run 1 ([`../run-1/`](../run-1/README.md)):
+- **Training:** the train split of `pilot-2`, `batch-01` and `batch-02` (239 documents: 7
+  `manual`, 232 model drafts).
+- **Early stopping and run selection:** only the model-drafted dev batches `batch-03` and
+  `batch-06`.
+- **Untouched:** the 10 hand-labelled dev documents had no part in selection, and the test
+  split was not touched.
+
+All scores against model drafts measure **agreement with the drafting model, not accuracy**.
+
+## All runs (agreement with 172 model-drafted dev records, at the selected epoch)
+
+| Run | Best / last epoch | Typed F1 | Span precision | Span recall | False-prerequisite rate | Conditions per document | Seconds |
+|---|---:|---:|---:|---:|---:|---:|---:|
+| `deberta-2-base-bio_weightingnone-learning_rate2e-05-seed0` | 1/4 | 0.000 | 0.000 | 0.000 | 0.955 | 0.9 | 69 |
+| `deberta-2-base-bio_weightingnone-learning_rate2e-05-seed1` | 1/4 | 0.000 | 0.000 | 0.000 | 0.937 | 8.6 | 69 |
+| `deberta-2-base-bio_weightingnone-learning_rate2e-05-seed2` | 1/4 | 0.000 | 0.000 | 0.000 | 0.959 | 4.7 | 68 |
+| `deberta-2-base-bio_weightingnone-learning_rate5e-05-seed0` | 1/4 | 0.000 | 0.000 | 0.000 | 0.000 | 0.0 | 68 |
+| `deberta-2-base-bio_weightingnone-learning_rate5e-05-seed1` | 1/4 | 0.000 | 0.000 | 0.000 | 0.000 | 0.0 | 68 |
+| `deberta-2-base-bio_weightingnone-learning_rate5e-05-seed2` | 1/4 | 0.000 | 0.000 | 0.000 | 0.000 | 0.0 | 68 |
+| `deberta-2-base-bio_weightingsqrt-learning_rate2e-05-seed0` | 15/15 | 0.072 | 0.055 | 0.238 | 0.660 | 11.5 | 267 |
+| `deberta-2-base-bio_weightingsqrt-learning_rate2e-05-seed1` | 15/15 | 0.077 | 0.062 | 0.244 | 0.658 | 10.9 | 265 |
+| `deberta-2-base-bio_weightingsqrt-learning_rate2e-05-seed2` | 11/14 | 0.070 | 0.054 | 0.269 | 0.701 | 13.4 | 246 |
+| `deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0` | 9/12 | **0.133** | 0.108 | 0.355 | 0.607 | 8.8 | 210 |
+| `deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed1` | 15/15 | 0.132 | 0.108 | 0.319 | 0.596 | 7.7 | 263 |
+| `deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed2` | 9/12 | 0.130 | 0.100 | 0.319 | 0.577 | 8.4 | 210 |
+| `distilbert-2-base-bio_weightingnone-learning_rate2e-05-seed0` | 1/4 | 0.000 | 0.000 | 0.000 | 0.000 | 0.0 | 39 |
+| `distilbert-2-base-bio_weightingnone-learning_rate2e-05-seed1` | 1/4 | 0.000 | 0.000 | 0.000 | 0.000 | 0.0 | 39 |
+| `distilbert-2-base-bio_weightingnone-learning_rate2e-05-seed2` | 1/4 | 0.000 | 0.000 | 0.000 | 0.000 | 0.0 | 39 |
+| `distilbert-2-base-bio_weightingnone-learning_rate5e-05-seed0` | 1/4 | 0.000 | 0.000 | 0.000 | 0.000 | 0.0 | 39 |
+| `distilbert-2-base-bio_weightingnone-learning_rate5e-05-seed1` | 14/15 | 0.064 | 0.073 | 0.104 | 0.510 | 4.1 | 146 |
+| `distilbert-2-base-bio_weightingnone-learning_rate5e-05-seed2` | 1/4 | 0.000 | 0.000 | 0.000 | 0.000 | 0.0 | 39 |
+| `distilbert-2-base-bio_weightingsqrt-learning_rate2e-05-seed0` | 8/11 | 0.039 | 0.034 | 0.168 | 0.724 | 13.2 | 101 |
+| `distilbert-2-base-bio_weightingsqrt-learning_rate2e-05-seed1` | 12/15 | 0.042 | 0.037 | 0.137 | 0.662 | 10.3 | 147 |
+| `distilbert-2-base-bio_weightingsqrt-learning_rate2e-05-seed2` | 14/15 | 0.033 | 0.027 | 0.111 | 0.667 | 11.1 | 147 |
+| `distilbert-2-base-bio_weightingsqrt-learning_rate5e-05-seed0` | 13/15 | 0.074 | 0.069 | 0.184 | 0.554 | 7.3 | 147 |
+| `distilbert-2-base-bio_weightingsqrt-learning_rate5e-05-seed1` | 4/7 | 0.041 | 0.030 | 0.194 | 0.770 | 17.7 | 69 |
+| `distilbert-2-base-bio_weightingsqrt-learning_rate5e-05-seed2` | 15/15 | **0.075** | 0.072 | 0.171 | 0.541 | 6.9 | 148 |
+
+Typed F1 across seeds (mean ± SD):
+
+| Grid cell | 2e-5 | 5e-5 |
+|---|---:|---:|
+| DeBERTa, `sqrt` | 0.073 ± 0.004 | **0.131 ± 0.001** |
+| DeBERTa, `none` | 0.000 ± 0.000 | 0.000 ± 0.000 |
+| DistilBERT, `sqrt` | 0.038 ± 0.004 | 0.063 ± 0.019 |
+| DistilBERT, `none` | 0.000 ± 0.000 | 0.021 ± 0.037 |
+
+For reference, `crf-batches-01-02` scores 0.145 typed F1 on the same records
+([`../../agreement/`](../../agreement/README.md)).
+
+## Best run of each grid against the CRF
+
+Paired project-group bootstrap, 1,000 resamples, B − A with A = `crf-batches-01-02`.
+
+| B | Slice | Typed F1 | Exact span precision | False-prerequisite rate (lower is better) |
+|---|---|---:|---:|---:|
+| `deberta-2-…sqrt…5e-05-seed0` | agreement (172) | −0.013 [−0.063, +0.043] | **−0.224 [−0.288, −0.163]** | **+0.254 [+0.188, +0.343]** |
+| `distilbert-2-…sqrt…5e-05-seed2` | agreement (172) | **−0.070 [−0.113, −0.020]** | **−0.260 [−0.320, −0.202]** | **+0.187 [+0.124, +0.252]** |
+| `deberta-2-…sqrt…5e-05-seed0` | hand-labelled (10) | −0.106 [−0.313, +0.194] | −0.304 [−0.525, +0.150] | +0.350 [−0.233, +0.690] |
+| `distilbert-2-…sqrt…5e-05-seed2` | hand-labelled (10) | −0.178 [−0.316, +0.040] | −0.357 [−0.583, +0.026] | +0.258 [−0.194, +0.500] |
+
+Bold marks a clear difference. Full reports: `compare_*.md`, and `<run>_dev_agreement.md` /
+`<run>_dev_manual.md` for every run.
+
+## Reading
+
+The transformer numbers are **optimistic**: early stopping and the choice of the best run used
+these same 172 records, while the CRF was not tuned on them. A difference in the CRF's favour
+is therefore conservative. A difference in a transformer's favour would need a separate
+held-out slice.
+
+- **The softer weighting fixed most of run 1's failure.** The best DeBERTa typed F1 rose from
+  0.023 to 0.133, and all three DeBERTa `sqrt` 5e-5 seeds agree within 0.003.
+- **DeBERTa is not clearly different from the CRF on typed agreement** (−0.013, interval
+  across zero). It is clearly worse on precision and on the false-prerequisite rate: it still
+  predicts about 8 conditions per document against 1.8 for the CRF, so it invents
+  prerequisites more often. On the error that matters most, the CRF remains the better
+  system.
+- **DistilBERT is clearly worse than the CRF** on typed agreement (−0.070).
+- **Uniform weights (`none`) were cut off by early stopping, not tested.** These runs predict
+  no spans after epoch 1, score 0.000, and patience 3 stops them at epoch 4 before they can
+  start tagging. The one that survived (DistilBERT, 5e-5, seed 1) climbed to 0.064 by epoch
+  14. Run 3 needs a minimum number of epochs before patience applies.
+- **Several runs were still improving at their last epoch** (best epoch 15 of 15).
+- **On the 10 hand-labelled documents nothing is clear**, as with every other comparison on
+  that slice.
+
+## Provenance
+
+- Results archive SHA-256 `7b760c8b52e7f5f69cd50a4f76ee9fee8b9ed540c78bc15a06f238941ba8aa85`.
+- Bundles archive (best run of each grid, weights not committed): 661,598,084 bytes.
+- Committed:
+  - per-run `metadata.json` in `metadata/` (configuration, base revision, training hash and
+    origins, span-tag weights, per-epoch losses and dev scores, peak GPU memory);
+  - the predictions of the two best runs in `predictions/`.
+
+To reproduce the comparisons from the committed predictions:
+
+```bash
+uv run triggermap compare annotations/batches/batch-03.jsonl reports/agreement/predictions/crf-batches-01-02.jsonl \
+    reports/neural/run-2/predictions/deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0.jsonl \
+    --add-gold annotations/batches/batch-06.jsonl --split dev
+```
diff --git a/.../compare_agreement_deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0.md b/.../compare_agreement_deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0.md
@@ -0,0 +1,25 @@
+# Agreement: crf-batches-01-02 -> deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0
+
+- Gold: `annotations/batches/batch-03.jsonl`, `annotations/batches/batch-06.jsonl` (dev split of `splits/pool-v1.json`; origin: all; 172 documents)
+- Baseline (A): `reports/agreement/predictions/crf-batches-01-02.jsonl` (crf-batches-01-02)
+- Candidate (B): `reports/neural/predictions/deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0.jsonl` (deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0)
+- Bootstrap: 1000 paired resamples, project clusters, seed 0
+- Documents: 172
+
+Differences are B minus A with 95% paired bootstrap intervals. *Clear* means the whole interval lies on one side of zero.
+
+| Metric | A (crf-batches-01-02) | B (deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0) | B - A | 95% CI | Verdict |
+|---|---:|---:|---:|:---:|---|
+| Typed condition micro-F1 (span + category + role) | 0.145 | 0.133 | -0.013 | [-0.063, +0.043] | no clear difference |
+| Exact span F1 | 0.221 | 0.165 | -0.056 | [-0.111, +0.007] | no clear difference |
+| Exact span precision | 0.332 | 0.108 | -0.224 | [-0.288, -0.163] | B clearly worse |
+| Exact span recall | 0.166 | 0.355 | +0.189 | [+0.127, +0.254] | B clearly better |
+| Role macro-F1 (roles with gold support) | 0.115 | 0.087 | -0.028 | [-0.078, +0.024] | no clear difference |
+| Category macro-F1 | 0.137 | 0.129 | -0.008 | [-0.060, +0.054] | no clear difference |
+| Group exact-match F1 | 0.000 | 0.000 | +0.000 | [+0.000, +0.000] | no clear difference |
+| Complete-document exact match, documents with conditions | 0.065 | 0.026 | -0.039 | [-0.076, +0.000] | no clear difference |
+| Complete-document exact match, documents without conditions | 0.882 | 0.176 | -0.706 | [-0.917, -0.462] | B clearly worse |
+| False prerequisite rate (lower is better) | 0.353 | 0.607 | +0.254 | [+0.188, +0.343] | B clearly worse |
+| Reversed meaning rate (lower is better) | 0.074 | 0.053 | -0.021 | [-0.089, +0.039] | no clear difference |
+| Category accuracy on exactly matched spans | 0.797 | 0.898 | +0.101 | [+0.010, +0.227] | B clearly better |
+| Role accuracy on exactly matched spans | 0.859 | 0.905 | +0.046 | [-0.043, +0.111] | no clear difference |
diff --git a/...mpare_agreement_distilbert-2-base-bio_weightingsqrt-learning_rate5e-05-seed2.md b/...mpare_agreement_distilbert-2-base-bio_weightingsqrt-learning_rate5e-05-seed2.md
@@ -0,0 +1,25 @@
+# Agreement: crf-batches-01-02 -> distilbert-2-base-bio_weightingsqrt-learning_rate5e-05-seed2
+
+- Gold: `annotations/batches/batch-03.jsonl`, `annotations/batches/batch-06.jsonl` (dev split of `splits/pool-v1.json`; origin: all; 172 documents)
+- Baseline (A): `reports/agreement/predictions/crf-batches-01-02.jsonl` (crf-batches-01-02)
+- Candidate (B): `reports/neural/predictions/distilbert-2-base-bio_weightingsqrt-learning_rate5e-05-seed2.jsonl` (distilbert-2-base-bio_weightingsqrt-learning_rate5e-05-seed2)
+- Bootstrap: 1000 paired resamples, project clusters, seed 0
+- Documents: 172
+
+Differences are B minus A with 95% paired bootstrap intervals. *Clear* means the whole interval lies on one side of zero.
+
+| Metric | A (crf-batches-01-02) | B (distilbert-2-base-bio_weightingsqrt-learning_rate5e-05-seed2) | B - A | 95% CI | Verdict |
+|---|---:|---:|---:|:---:|---|
+| Typed condition micro-F1 (span + category + role) | 0.145 | 0.075 | -0.070 | [-0.113, -0.020] | B clearly worse |
+| Exact span F1 | 0.221 | 0.101 | -0.120 | [-0.172, -0.062] | B clearly worse |
+| Exact span precision | 0.332 | 0.072 | -0.260 | [-0.320, -0.202] | B clearly worse |
+| Exact span recall | 0.166 | 0.171 | +0.005 | [-0.054, +0.061] | no clear difference |
+| Role macro-F1 (roles with gold support) | 0.115 | 0.060 | -0.055 | [-0.093, -0.009] | B clearly worse |
+| Category macro-F1 | 0.137 | 0.066 | -0.071 | [-0.108, -0.023] | B clearly worse |
+| Group exact-match F1 | 0.000 | 0.000 | +0.000 | [+0.000, +0.000] | no clear difference |
+| Complete-document exact match, documents with conditions | 0.065 | 0.013 | -0.052 | [-0.095, -0.007] | B clearly worse |
+| Complete-document exact match, documents without conditions | 0.882 | 0.882 | +0.000 | [-0.263, +0.214] | no clear difference |
+| False prerequisite rate (lower is better) | 0.353 | 0.541 | +0.187 | [+0.124, +0.252] | B clearly worse |
+| Reversed meaning rate (lower is better) | 0.074 | 0.046 | -0.029 | [-0.083, +0.013] | no clear difference |
+| Category accuracy on exactly matched spans | 0.797 | 0.848 | +0.052 | [-0.058, +0.192] | no clear difference |
+| Role accuracy on exactly matched spans | 0.859 | 0.879 | +0.019 | [-0.082, +0.108] | no clear difference |
diff --git a/...n-2/compare_manual_deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0.md b/...n-2/compare_manual_deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0.md
@@ -0,0 +1,25 @@
+# Hand-labelled dev: crf-batches-01-02 -> deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0
+
+- Gold: `annotations/pilot/pilot-2.jsonl` (dev split of `splits/pool-v1.json`; origin: manual; 10 documents)
+- Baseline (A): `reports/agreement/predictions/crf-batches-01-02.jsonl` (crf-batches-01-02)
+- Candidate (B): `reports/neural/predictions/deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0.jsonl` (deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0)
+- Bootstrap: 1000 paired resamples, project clusters, seed 0
+- Documents: 10
+
+Differences are B minus A with 95% paired bootstrap intervals. *Clear* means the whole interval lies on one side of zero.
+
+| Metric | A (crf-batches-01-02) | B (deberta-2-base-bio_weightingsqrt-learning_rate5e-05-seed0) | B - A | 95% CI | Verdict |
+|---|---:|---:|---:|:---:|---|
+| Typed condition micro-F1 (span + category + role) | 0.370 | 0.265 | -0.106 | [-0.313, +0.194] | no clear difference |
+| Exact span F1 | 0.370 | 0.294 | -0.076 | [-0.294, +0.239] | no clear difference |
+| Exact span precision | 0.500 | 0.196 | -0.304 | [-0.525, +0.150] | no clear difference |
+| Exact span recall | 0.294 | 0.588 | +0.294 | [+0.111, +0.545] | B clearly better |
+| Role macro-F1 (roles with gold support) | 0.200 | 0.158 | -0.042 | [-0.262, +0.143] | no clear difference |
+| Category macro-F1 | 0.237 | 0.285 | +0.048 | [-0.202, +0.324] | no clear difference |
+| Group exact-match F1 | 0.500 | 0.000 | -0.500 | [-1.000, +0.000] | no clear difference |
+| Complete-document exact match, documents with conditions | 0.222 | 0.000 | -0.222 | [-0.455, +0.000] | no clear difference |
+| Complete-document exact match, documents without conditions | 1.000 | 1.000 | +0.000 | [+0.000, +0.000] | no clear difference |
+| False prerequisite rate (lower is better) | 0.333 | 0.683 | +0.350 | [-0.233, +0.690] | no clear difference |
+| Reversed meaning rate (lower is better) | 0.000 | 0.000 | +0.000 | [+0.000, +0.000] | no clear difference |
+| Category accuracy on exactly matched spans | 1.000 | 1.000 | +0.000 | [+0.000, +1.000] | no clear difference |
+| Role accuracy on exactly matched spans | 1.000 | 0.900 | -0.100 | [-0.333, +0.778] | no clear difference |