From 72c48e4061c67f5a25a528514e94468c692934e2 Mon Sep 17 00:00:00 2001 From: Rafael Vuijk Date: Thu, 22 Oct 2026 12:16:00 +0000 Subject: [PATCH] Measure the 2.6.0 performance pair at the last kernel change, the 2662nd The pair recorded on 2026-10-18 measured v2.5.0 beside the 2549th, which was then the last change to the kernel; 113 commits came after it. Both columns again, v2.5.0 (the 1957th) and 784f69d5 (the 2662nd), from one run of benchmark_key_commits.sh on one machine with nothing else on it. The v2.5.0 column reproduces the earlier run's to within 0.02% on every gated row. Since the 2549th, SimplifyHard fell 10.4% more (-52.4% over the release) and every other gated row moved less than 1%; Derivate is +5.65% over the release, +0.1% of it after the 2549th. Part of #746. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura --- .../WhatsNew/version_performance_control.md | 96 ++++++++++--------- 1 file changed, 49 insertions(+), 47 deletions(-) diff --git a/Sources/AngouriMath/Docs/WhatsNew/version_performance_control.md b/Sources/AngouriMath/Docs/WhatsNew/version_performance_control.md index de2d05ed2..49e9c2093 100644 --- a/Sources/AngouriMath/Docs/WhatsNew/version_performance_control.md +++ b/Sources/AngouriMath/Docs/WhatsNew/version_performance_control.md @@ -180,48 +180,48 @@ honest reason -- a compiled delegate over `Complex` has nothing to put on the he --- -## The 1957th and the 2549th, measured together on one machine — the pair for 2.6.0 +## The 1957th and the 2662nd, measured together on one machine — the pair for 2.6.0 The pair the release checklist owes: `v2.5.0` ([`ab7a8bd6`](https://github.com/asc-community/AngouriMath/commit/ab7a8bd6b92d9ae99b3e23e51b10dca5c0e8fe11), the 1957th) re-measured beside -[`001b1caa`](https://github.com/asc-community/AngouriMath/commit/001b1caa694f1dd8476c3a54d93d10d7a7fcfe4d), the -2549th, which is the last change to the kernel before the release. Both columns come from one run of -`Sources/Utils/benchmark_key_commits.sh`, on 2026-10-18, on one machine with nothing else on it. The period is the -whole of 2.6.0, 592 commits. +[`784f69d5`](https://github.com/asc-community/AngouriMath/commit/784f69d5b07d2f0c153c67791468b3754f2e3250), the 2662nd, which is the last change to +the kernel before the release. Both columns come from one run of `Sources/Utils/benchmark_key_commits.sh`, on +2026-10-22, on one machine with nothing else on it. The period is the whole of 2.6.0, 705 commits. -The `v2.5.0` column agrees with the 1954th column of the 2.5.0 pair, measured on 2026-09-09 three commits earlier. -Every row the gate measures agrees to within 0.02%, and five agree to the byte, `SimplifyEasy` among them. -`CompileEasy` and `CompileHard` differ by 0.30% and 0.19%. The gate leaves those two ungated because their -allocation includes building a delegate. +It replaces the pair measured on 2026-10-18 against the 2549th, which was then the last change to the kernel; 113 +commits came after it. The `v2.5.0` column agrees with that run's on every row the gate measures, to within 0.02%, and +`Derivate` and `SimplifyEasy` agree to the byte. `CompileEasy` reads 11,003 here and 10,970 there, 0.30% apart. The +gate leaves it and `CompileHard` ungated because their allocation includes building a delegate. ### Allocation -| benchmark | v2.5.0 | 2549th | change | -|---|--:|--:|--:| -| `CompileEasy` | 10,970 | 11,003 | +0.30% | -| `CompileHard` | 20,434 | 20,434 | 0.00% | -| `Derivate` | 53,239 | 56,190 | **+5.54%** | -| `EvalPolynomialFresh` | 650,888 | 12,536 | **−98.07%** | -| `EvalPolynomialFresh15Digits` | 265,816 | 12,064 | **−95.46%** | -| `EvalTranscendentalFresh` | 5,668,395 | 34,760 | **−99.39%** | -| `EvalTrig` | 1,341,457 | 69,944 | **−94.79%** | -| `EvalTrigPrecise` | 12,742,285 | 336,552 | **−97.36%** | -| `ParseEasy` | 18,261 | 18,421 | +0.88% | -| `ParseHard` | 3,584,817 | 60,103 | **−98.32%** | -| `SimplifyEasy` | 79,282 | 78,978 | −0.38% | -| `SimplifyHard` | 331,710,136 | 176,205,784 | **−46.88%** | -| `SolveEasy` | 8,865,872 | 454,021 | **−94.88%** | -| `SolveEasyMedium` | 65,648 | 66,744 | **+1.67%** | -| `SolveHard` | 11,884,272 | 10,001,832 | **−15.84%** | -| `SolveMedium` | 455,893 | 459,134 | +0.71% | -| `SolveMediumHard` | 1,449,394 | 1,411,524 | −2.61% | +| benchmark | v2.5.0 | 2662nd | change | since the 2549th | +|---|--:|--:|--:|--:| +| `CompileEasy` | 11,003 | 11,003 | 0.00% | 0.00% | +| `CompileHard` | 20,434 | 20,737 | +1.48% | +1.48% | +| `Derivate` | 53,239 | 56,247 | **+5.65%** | +0.10% | +| `EvalPolynomialFresh` | 650,888 | 12,536 | **−98.07%** | 0.00% | +| `EvalPolynomialFresh15Digits` | 265,816 | 12,064 | **−95.46%** | 0.00% | +| `EvalTranscendentalFresh` | 5,668,395 | 34,760 | **−99.39%** | 0.00% | +| `EvalTrig` | 1,341,457 | 69,944 | **−94.79%** | 0.00% | +| `EvalTrigPrecise` | 12,742,285 | 336,552 | **−97.36%** | 0.00% | +| `ParseEasy` | 18,258 | 18,400 | +0.78% | −0.11% | +| `ParseHard` | 3,584,841 | 60,023 | **−98.33%** | −0.13% | +| `SimplifyEasy` | 79,282 | 79,682 | +0.50% | +0.89% | +| `SimplifyHard` | 331,710,008 | 157,898,664 | **−52.40%** | **−10.39%** | +| `SolveEasy` | 8,865,936 | 453,396 | **−94.89%** | −0.14% | +| `SolveEasyMedium` | 65,650 | 66,664 | **+1.54%** | −0.12% | +| `SolveHard` | 11,884,016 | 9,998,784 | **−15.86%** | −0.03% | +| `SolveMedium` | 455,925 | 458,910 | +0.65% | −0.05% | +| `SolveMediumHard` | 1,449,394 | 1,412,412 | −2.55% | +0.06% | Bytes allocated per call. `EvalEasy`, `RunEasy`, `RunMedium` and `RunHard` allocate nothing in both columns and are -left out. +left out. *Since the 2549th* compares with the 2549th column of the run of 2026-10-18. - **The evaluation rows** are the numerical-evaluation work of the 2050th to the 2077th, recorded below. - **`ParseHard`** is the 2203rd, the parser predicting in SLL first. +- **`SimplifyHard`** fell a further 10.4% after the 2549th. That is not attributed to any section of this file. ### The steps compose, and that was checked @@ -231,12 +231,12 @@ left out. - −41.4% and −20.2% at the 2054th to the 2075th; - −42.1% at the 2077th. -Composed, they predict −94.9%, and the end-to-end measurement is −94.88%. +Composed, they predict −94.9%, and the end-to-end measurement is −94.89%. -`SimplifyHard`'s recorded steps predict about −44.5%, and the measurement is −46.88%. The 2.4 points between them -are not attributed to any section of this file. +`SimplifyHard`'s recorded steps predict about −44.5%. It measured −46.88% at the 2549th and −52.40% here, and neither +gap is attributed. -### `Derivate` went up 5.5%, in steps none of which the gate could see +### `Derivate` went up 5.65%, in steps none of which the gate could see `Derivate` differentiates a parsed constant, so no parser change reaches it. Its gate baseline moved five times across the release, each move within the gate's 3%: @@ -249,7 +249,8 @@ across the release, each move within the gate's 3%: | 2117th | 54,567 | +1.9% | | 2130th | 55,095 | +1.0% | | 2203rd | 55,783 | +1.2% | -| 2549th, measured here | 56,190 | +0.7% | +| 2549th, the run of 2026-10-18 | 56,190 | +0.7% | +| 2662nd, measured here | 56,247 | +0.1% | A baseline is re-recorded by the pull request that moves it, so each step covers every commit since the step before it, not one change. The largest step, +1,039 bytes, is the 2117th, @@ -258,8 +259,8 @@ a cached list of their operands, and so makes each of those nodes larger. That i **no step is bisected**, so the increase is recorded here as unattributed. This file has one other case of steps each free by the gate summing to a real cost, the rule-set exchange's +13%. This is the same pattern at a smaller size, and it is why the pair is measured end to end. -`SolveEasyMedium` +1.67%, `ParseEasy` +0.88% and `SolveMedium` +0.71% are below the gate's resolution for a single step -and are not attributed either. +`SolveEasyMedium` +1.54%, `ParseEasy` +0.78%, `SolveMedium` +0.65% and `SimplifyEasy` +0.50% are below the gate's +resolution for a single step and are not attributed either. ### Timings @@ -267,18 +268,19 @@ These moved well outside this machine's run-to-run spread and agree in sign with | benchmark | change | |---|--:| -| `EvalTranscendentalFresh` | −99.50% | -| `EvalPolynomialFresh` | −98.81% | +| `EvalTranscendentalFresh` | −99.51% | +| `EvalPolynomialFresh` | −98.83% | +| `ParseHard` | −98.54% | | `EvalTrigPrecise` | −98.20% | -| `ParseHard` | −98.17% | -| `SolveEasy` | −95.81% | -| `EvalTrig` | −95.47% | -| `EvalPolynomialFresh15Digits` | −94.63% | -| `SimplifyHard` | −43.90% | -| `SolveHard` | −37.69% | - -`Derivate` was +5.15% (13.7 → 14.5 µs), in line with its allocation. The smaller rows from the same run are not worth -reading and are not quoted. +| `SolveEasy` | −96.62% | +| `EvalTrig` | −95.53% | +| `EvalPolynomialFresh15Digits` | −94.85% | +| `SimplifyHard` | −56.99% | +| `SolveHard` | −39.96% | + +`Derivate` was +1.66% (14.5 → 14.7 µs). `SimplifyEasy` was −14.9% and `ParseEasy` −25.9% with their allocation flat or +up. One run does not say whether that is the code or the machine, so they are not read as either. The smaller rows from the same run are not +worth reading and are not quoted. ## The 1845th and the 1954th, measured together on one machine — the pair for 2.5.0