Skip to content

Make reading, parsing and comparing an API fast on large assemblies - #103

Merged
Mpdreamz merged 1 commit into
mainfrom
perf
Nov 14, 2026
Merged

Mpdreamz merged 1 commit into
mainfrom
perf

Conversation

@Mpdreamz

Copy link
Copy Markdown
Contributor

Benchmarks of reading, writing, parsing and comparing, with allocations, on System.Collections.Immutable and CoreLib (35,751 .api lines), using BenchmarkDotNet's MemoryDiagnoser. The profiles come from dotnet-trace.

CoreLib Before After
Read 407 ms, 832 MB 108 ms, 109 MB
Write 19 ms, 25 MB 19 ms, 25 MB
Parse 160 ms, 120 MB 122 ms, 72 MB
Compare with its own .api file 39 s, 4.4 GB 20 ms, 23 MB

On System.Collections.Immutable everything takes 2 ms or less: read 1.8 ms, write 0.45 ms, parse 1.2 ms, compare 0.36 ms.

What was slow

  • Compare:

    • For every member and parameter it wrote both types out as C# to see whether they differed.
    • It built each member's report path from rendered parameter types, changed or not.
    • Writing a type looked at every member of the type it was in, so the cost grew with the square of the member count: AdvSimd, with 1,772 members, took 5.3 s alone.

    Now it compares the models structurally (Same), and writes C# and builds paths only for a change. It pairs members by a structural identity instead of a string key, checks attributes and constraints on the models first, and caches one ApiDeclarationWriter per type.

  • Read:

    • Decoding an attribute's enum argument, such as [EditorBrowsable(EditorBrowsableState.Never)], scanned every type in the assembly.
    • Every attribute check built the attribute type's name again.

    Both are looked up once now. Nullable attributes are read from their blob instead of decoded the general way, primitive types and empty lists are shared, and names are compared in place.

  • Parse:

    • Each line was checked for a type declaration twice.
    • Every word was allocated twice, once to peek and once to take.
    • Each member got a new scope.

What's left

GC pauses are 45–60% of the remaining time. Reading or parsing builds a model that survives, and background GC handles that poorly. With DOTNET_gcConcurrent=0, CoreLib parses in 61 ms and reads in 105 ms. The tool now sets <ConcurrentGarbageCollection>false</ConcurrentGarbageCollection>. A library can't choose its host's GC mode, so library callers keep the numbers in the table.

Correctness

Run the benchmarks with dotnet run -c Release --project benchmarks/Nullean.PublicApi.Benchmarks -- --filter '*' --inProcess. They run in-process because BenchmarkDotNet 0.15.8 doesn't recognise net11.0 when it launches a separate process.

🤖 Generated with Claude Code

@Mpdreamz Mpdreamz added the enhancement An existing capability improved (minor bump) label Nov 14, 2026
Mpdreamz added a commit that referenced this pull request Nov 14, 2026
The CsCheck mirror property failed on #103's CI with seed
`7EiLdrYJA8Z9`. That failure is a bug on main since #96, not a
regression in #103.

A parameter's changes were labelled with its new name. So a parameter
that was renamed and also made `params` read `parameter rest` in one
direction and `parameter p0` in the other. The rename itself was already
reported by position (`parameter 1 name`), and now every other part of a
parameter is too:

```
-A › M(int) › parameter a default  changed: 1 → 2
+A › M(int) › parameter 1 default  changed: 1 → 2
```

Checks: the seed passes with the fix. `mise run ci` passes: 81 library
tests and 630 tool tests.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Benchmarks (benchmarks/Nullean.PublicApi.Benchmarks) measured reading, writing, parsing and comparing
System.Collections.Immutable and CoreLib. On CoreLib, comparing an API with its own .api file took 39 seconds and
4.4 GB: for every member and parameter it wrote both types out as C#, and writing a type looked at every member of
the type it was in, so a type with 1,772 members took 5 seconds alone. It now compares the models and writes C# only
for a change, builds a member's path only for a change, pairs members by a structural identity instead of a string
key, and checks attributes and constraints on the models first: 20 ms and 23 MB.

Reading CoreLib took 407 ms and 832 MB: decoding an attribute's enum argument scanned every type in the assembly, and
every attribute check built its type's name again. Both are looked up once now, nullable attributes are read from
their blob, primitive types and empty lists are shared, and names are compared in place: 108 ms and 109 MB.

Parsing CoreLib's 35,751 lines took 160 ms and 120 MB: a line is checked for a type declaration once, the word under
the cursor is kept, a member without type parameters keeps its scope, and locations are indexed only when asked for:
122 ms and 72 MB. Most of what is left is collecting the model, which survives; the tool turns off concurrent GC,
which halves that.

ApiDeclarationWriter writes one type's declarations and references, so a caller writing many of them builds the
type's names once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Mpdreamz
Mpdreamz merged commit 1321fbe into main Nov 14, 2026
6 checks passed
@Mpdreamz
Mpdreamz deleted the perf branch November 14, 2026 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement An existing capability improved (minor bump)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant