Deterministic reproducers, synthetic counter-examples, and verification receipts for AI coding agents, LLM inference engines, and quantitative market microstructure.
Full interactive atlas & multilingual notes: vxi.me/invariants (English · 简体中文 · 日本語)
Each case pins the upstream version/commit it was verified against. Probes are stdlib-only, self-contained, and runnable with a single command — no API keys, no network, no real market data. Honest scope notes live in the linked articles: a probe models the mechanism, it does not re-run the upstream engine.
This README is generated by generate_readme.py from the canonical case index; do not edit by hand.
Rejected states, protocol errors, or incomplete reviews leaking across handoff, admission gates, or retry boundaries.
| System / Version | Failure Mode & Counter-Example | Standalone Probe | Evidence & Full Analysis |
|---|---|---|---|
| SGLang @ 8f3d694 / Cloudflare GLM-5.3 | Req.set_finish_with_abort() sets to_finish, but disaggregated prefill/decode admission checks finished_reason, allowing a 1-token placeholder handoff and cross-request KV cache exposure. | probes/glm53-kv-cache-cross-request-exposure.py |
📄 EN · 中文 · md · 🧾 JSON |
| OpenQodex 0.11.0 | checkPush() returns abstain when the matching review record is incomplete, so a Git pre-push hook lets the push through by default instead of failing closed. | probes/openqodex-prepush-review-fail-open.sh |
📄 EN · 中文 · md |
| MCP 2026 / Cakewalk | Client retry logic cannot cleanly separate transport HTTP 503, JSON-RPC protocol errors, and tool-level isError:true carried over SSE. | probes/mcp-client-http-jsonrpc-iserror.mjs |
📄 EN · 中文 · md |
When inclusion rules, dedup, or the effective sample count change, a metric can shift even if the formula looks the same.
| System / Version | Failure Mode & Counter-Example | Standalone Probe | Evidence & Full Analysis |
|---|---|---|---|
| vectorbt 1.1.1 → 1.1.2 | Tail CVaR counted the raw array length (n=21) before masking NaNs, giving -4.67% instead of -5.50%; the v1.1.2 release also fixed an isclose early-return bug in tests/utils.py (PR #878). | probes/vectorbt-nan-cvar-tail-sample-test-assertions.py |
📄 EN · 中文 · md |
| Polymarket Data API | Wash-trading volume traps: reported volume in the Data API vs. matched-trade logs diverge, distorting price-discovery reads. | probes/polymarket-election-volume-wash-trading-api-price-discovery.py |
📄 EN · 中文 · md |
| Strategy evaluation (synthetic) | The same stock selected repeatedly inflates win-rate; denominator science for repeated picks. | probes/quant-winrate-repeated-stock-picks.py |
📄 EN · 中文 · md |
| Multi-strategy scoring (synthetic) | Two strategies both scoring 80 can reverse rank after pooling — Simpson's paradox in probability calibration. | probes/stock-score-calibration-simpsons-paradox.py |
📄 EN · 中文 · md |
What a tool returns, a command guard, or a dependency authorization actually covers may be narrower than intuition.
| System / Version | Failure Mode & Counter-Example | Standalone Probe | Evidence & Full Analysis |
|---|---|---|---|
| TinyFish Fetch / AgentQL | Measured where a fetch-based AI browser returns hidden DOM content vs. AgentQL source-checked boundaries. | — | 📄 EN · 中文 · md |
| laya-guard @ be37a19 | Independent re-test of a regex + 322M local-model PreToolUse guard: the 3 author-reported bypass classes were confirmed fixed; real-model path shows a false-positive tradeoff (npm run deploy blocked at risk 0.857). | — | 📄 EN · 中文 · md |
| pnpm 12.11 | npm packages can ship Agent Skills; approving once at install time leaves upgrade-time scope checks open. | — | 📄 EN · 中文 · md |
| Meta Muse | Proactive-permission test: caught injected instructions but missed a 30-minute calendar conflict — source-integrity gap. | — | 📄 EN · 中文 · md |
Same-timestamp row order and truncated-batch end markers can be misread as real causality or batch completeness.
| System / Version | Failure Mode & Counter-Example | Standalone Probe | Evidence & Full Analysis |
|---|---|---|---|
| NASDAQ ITCH 5.0 / Hawkes process | 114 simultaneous execution messages (E/C) in one nanosecond (AAPL): row-by-row recursion inflates log-likelihood by +213.29 nats vs. permutation-invariant pre-batch scoring. | probes/hawkes-itch-nanosecond-ties-likelihood.py |
📄 EN · 中文 · md · 🧾 JSON |
| NautilusTrader 2.0.0rc7 | F_LAST groups detect a missing tail but not a missing head: a truncated batch can still close, and batch_deltas=True vs False changes which intermediate book states the matcher sees. | probes/nautilustrader-orderbook-batch-deltas-f-last.py |
📄 EN · 中文 · md |
See probes/README.md for the probe contract: stdlib-only,
one-command runnable, honest docstring stating what is not covered.