[gmpassos]

Summary

Adds Brotli (br) response compression next to GZip, using brotli_fast ^1.0.0 (pure Dart: no native library, no FFI, no package dependencies). It also fixes how Accept-Encoding is handled.

• Negotiation. Accept-Encoding is now parsed properly: q-values, q=0 and *. Each response uses the best encoding from APIServerConfig.compressionEncodings (default [br, gzip]). On a tie, the server's order wins.
• Where it applies:
  • API responses, JSON included, are compressed once by compressResponse, using the headers of the final response. AutoCompressSink generalizes AutoGZipSink for direct resolveBody callers. AutoGZipSink is kept and works as before.
  • Static files (createCompressionMiddleware).
  • The static files cache, which keeps identity, gzip and br variants per file. A gzip file is converted to br when the client prefers it.
• Fallback: if the Brotli encoder throws, the next accepted encoding is used.
• Vary: Accept-Encoding is sent with responses that can be compressed.
• Request bodies: Content-Encoding: br payloads are decompressed (decompressPayload), and maxPayloadLength is enforced while decoding. If maxPayloadLength isn't set, decoding stops at 64 MB (APIServer.defaultMaxBrotliPayloadLength), because a few bytes of Brotli can decode to gigabytes. Decoding runs in the isolate pool (APIBrotli.decodeAsync), so it doesn't block the event loop. A corrupted body still throws a FormatException.
• Config, each available as an argument, a server.* config entry and a JSON field:
  • compressionEncodings: an empty list, false, none, off or no disables compression.
  • brotliQuality: 0..11, default 5. Cached static files use at least 9.

Fix: JSON was gzipped without Accept-Encoding

_jsonEncodePayload gzipped every JSON payload of 512 bytes or more, whatever the client sent in Accept-Encoding. It now compresses only with an encoding the client accepts. APIServer.resolveBody/resolveBodySync called without the new compression parameter keep the old behavior (always GZip), for direct callers.

Breaking: Dart SDK >=3.13.5

brotli_fast requires it. Raising the SDK constraint changes the language version, so this PR also has to:

• reformat the sources (the formatter style follows the language version);
• apply prefer_initializing_formals (private named parameters). Public parameter names are unchanged, so no call site changes;
• regenerate the reflection files (languageVersion goes from 3.10.0 to 3.13.0).

Those are in separate commits, so the feature commit (v1.28.0: Brotli ...) can be reviewed on its own.

Using brotli_fast

• Encoding is one-shot brotliEncode on the whole body, which is the library's fastest path. Its streaming encoder keeps all the input in memory anyway, and our bodies are already complete bytes. BrotliSink buffers the input, encodes on close, and falls back to GZip if the encoder fails.
• brotli_fast is pure Dart, so a large encode would block the server's event loop. Encodes of about 1.5 ms or more run in an isolate pool instead (see below).
• Decoding is brotliDecode(..., maxOutputSize: ...). The decoder buffer grows up to the limit, so no retry is needed.

Benchmark (AOT, 592 KB entity-list JSON, brotli_fast vs dart:io gzip)

	size	encode
gzip level 4	76.2 KB	1.2 ms
gzip level 9	64.4 KB	5.8 ms
br q5 (default, dynamic)	65.3 KB	2.6 ms
br q6	38.9 KB	11.4 ms
br q9 (cached static files)	34.3 KB	29 ms
br q11	34.3 KB	289 ms

Decoding (q5) takes 0.7 ms. Outputs are checked against the reference brotli CLI in the tests. --brotli-quality 6 is worth considering when bandwidth matters more than CPU.

Isolate pool for compression

APIBrotli.encodeAsync, APIGZip.encodeAsync and APIBrotli.decodeAsync run in compressionIsolatePool, one pool shared by Brotli and gzip. It's a new APIIsolatePool (lib/src/bones_api_utils_isolate_pool.dart):

• Bounded: at most maxIsolates workers. Extra encodes wait in a FIFO queue, so 30 simultaneous large files never means 30 isolates.
• Sized by the server workers: the default is half the cores divided by the number of APIServerWorkers, between 2 and 4 (APIIsolatePool.defaultMaxIsolatesFor). Each worker sizes its own pool at startup (configureCompressionIsolatePool), so 16 server workers don't each start 4 encoder isolates.
• Warm: workers are long-lived, so the encoder tables are built once per worker. A new isolate per encode (Isolate.run) rebuilds them every time, costing about +1.5 ms at q6 and +2.5 ms at q9.
• Idle disposal: a worker idle for idleTimeout (default 30 s) is shut down and spawned again on demand. The idle timer runs inside the worker, and an idle worker's port doesn't keep the isolate alive, so the pool never keeps a program from exiting.
• Failures: a computation that throws fails with RemoteError. A worker that dies is replaced, and close() stops all workers, including one that is still starting.

Benchmark (AOT, entity-list JSON, Apple Silicon with 10 cores, pool of 4).

One encode at a time, median:

size	q	inline	Isolate.run	pool
16 KB	9	0.81 ms	3.07 ms	0.78 ms
32 KB	9	1.85 ms	4.25 ms	2.05 ms
64 KB	6	1.32 ms	2.75 ms	1.53 ms
64 KB	9	3.38 ms	5.91 ms	3.35 ms
256 KB	5	1.19 ms	1.21 ms	1.22 ms
256 KB	9	13.3 ms	16.7 ms	13.5 ms
1 MB	9	66.1 ms	69.8 ms	66.4 ms

30 encodes at once, total time / longest event-loop stall in the main isolate:

payload	inline	Isolate.run (30 isolates)	pool (4 isolates)
64 KB q9	118 / 119 ms	48 / 21 ms	34 / 1.5 ms
256 KB q9	417 / 417 ms	103 / 59 ms	133 / 1.7 ms
1 MB q6	859 / 859 ms	177 / 112 ms	267 / 1.8 ms

A 1 ms periodic timer measures the stalls, so ~1.2 ms is the floor. Spawning and collecting 30 isolates blocks the main isolate for 21–112 ms. The pool keeps it under 2 ms, and its total time is a little longer only because it uses 4 of the 10 cores.

So an encode moves to the pool once it would take about 1.5 ms inline (APIBrotli.isolateMinLength): 32 KB at q9+, 64 KB at q6+, 256 KB below. Smaller encodes stay inline. Concurrent requests for the same cached file share one encode.

Gzip in the same pool. Gzip uses native zlib, but ZLibEncoder.convert runs synchronously on the calling isolate, so a large gzip encode blocks the event loop too. In the pool it costs the same as inline (+0.1 ms at most):

size	level	inline	pool
256 KB	9	2.60 ms	2.62 ms
1 MB	4	2.11 ms	2.23 ms
1 MB	7	4.28 ms	4.39 ms
4 MB	4	8.58 ms	8.83 ms
4 MB	9	40.7 ms	40.9 ms

30 gzip encodes at once, total time / longest event-loop stall:

payload	inline	pool (4 isolates)
256 KB level 4	16 / 17 ms	6 / 1.9 ms
1 MB level 4	66 / 67 ms	21 / 1.9 ms
1 MB level 7	139 / 139 ms	46 / 1.6 ms

The threshold uses the same ~1.5 ms rule (APIGZip.isolateMinLength): 192 KB at level 9, 384 KB at 6+, 768 KB below.

Tests

• New test/bones_api_utils_compression_test.dart covers Accept-Encoding parsing and negotiation, Brotli round trips at every quality, maxOutputSize, corrupted input, compressResponse, the middleware, and interop with the brotli CLI (skipped when the CLI isn't installed).
• AutoCompressSink/BrotliSink (sink tests), br cache variants, config args/JSON/apiConfig, br request payloads.
• HTTP end to end: br JSON/text/static, no compression without Accept-Encoding (regression), q=0, HEAD, Future and Stream payloads, and large files encoded in the pool with concurrent requests. Servers with gzip only, br only (no HttpServer.autoCompress) and no compression.
• New test/bones_api_utils_isolate_pool_test.dart: results, the concurrency limit and queueing, never more than maxIsolates workers, worker reuse, async computations, large data, idle disposal and respawn, the idle-timeout race, errors, unsendable computations and results, a dying worker, close (also while a worker starts), and the pool not keeping its isolate alive.
• Encoder failure fallbacks, using a simulated failing encoder (APIBrotli.encoder, @visibleForTesting): compressResponse, BrotliSink, cache variants, and the pool.
• br request payloads: the default 64 MB limit, a "decompression bomb", maxPayloadLength taking precedence, and decodeAsync keeping BrotliException.
• Cache: 304 and memory accounting with br/gzip variants, unsupported encodings (deflate), and variants not cached when their entry is removed during an encode.
• Local runs: dart format, dart analyze --fatal-infos, dependency_validator and the full suite, including the Docker DB suites, all pass, and so does ensure_build.
• Stacked on this PR: Fix: request hang on a failing Future payload #193 fixes requests hanging when a route's Future payload fails (a bug already on master).

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 98.55832% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.61%. Comparing base (ed9e707) to head (d357398).

Files with missing lines	Patch %	Lines
lib/src/bones_api_utils_isolate_pool.dart	95.12%	6 Missing ⚠️
lib/src/bones_api_entity_db_postgres.dart	73.33%	4 Missing ⚠️
lib/src/bones_api_server_cache.dart	99.20%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #192      +/-   ##
==========================================
+ Coverage   95.55%   95.61%   +0.05%     
==========================================
  Files          73       75       +2     
  Lines       25179    25747     +568     
==========================================
+ Hits        24061    24619     +558     
- Misses       1118     1128      +10     

Flag	Coverage Δ
unittests	95.61% <98.55%> (+0.05%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[gmpassos]

Review fixes (2a85474)

A code review found 10 issues. All of them are fixed, each with a regression test:

1. Brotli request decode limit. BrotliDecoderV2.decode stops at the last power-of-two buffer below the limit, so a 9 MB body under a 10 MB maxPayloadLength was rejected. It now retries once with a buffer at the limit.
2. ERROR path and apiResponse2 branch. A JSON body was compressed in resolveBody, but its Content-Encoding header was written after the response headers had been resolved, so it was lost. The client got a compressed body with no header (a GZip body that could be Brotli-compressed again). Both paths now leave compression to compressResponse.
3. Cache: failed conversion. A failed gzip→br conversion skipped the cache on every request. Failed target encodings are now remembered per cached entry.
4. Static files at q9+ when not cacheable. High quality is now used only when APIServerResponseCache.isCacheable is true, through a new resolveCompression option on the middleware.
5. Cached compressed 404. It was served without Content-Encoding, to every client. It now keeps the header (plus Vary) and is only served to clients that accept that encoding.
6. 206 / Content-Range. 1xx, 204, 206, 304 and any response with Content-Range are no longer compressed.
7. Unknown compression_encodings. These silently turned all compression off. Unknown names are now ignored with a log message, and if none is known the defaults are used. An empty value or none still disables compression.
8. x-gzip is accepted as gzip (RFC 9110).
9. Altitude (partially). Items 2 and 9 were the same ordering problem, and it's fixed in both branches. I kept the streaming GZip JSON sink, which avoids buffering the uncompressed JSON for gzip.
10. Accept-Encoding parsing now happens once per request in the cache.

Local runs: dart format, analyze --fatal-infos, the VM unit suite, ensure_build, publish --dry-run, dependency_validator and Chrome (707 tests) all pass.