Repository navigation
perf(readers): read each module once per reader for diff --application's import-search checks (#1046) - #1137
Merged
Conversation
…, checked per carrier set (#1046)
…rison ahead of Packaging, with the full-corpus timings (#1046) The 1.3.0 section lists Application comparison before Packaging, and #1135 adds the same heading at that point; placed after Packaging, merging both would leave two Application comparison headings without a textual conflict. The timings now come from one development-corpus run of this head (load 2-6, every member under the cap, every answer identical to the pre-fix answers).
… into review-1137 # Conflicts: # CHANGELOG.md
pengfei-threemoonslab
added a commit
that referenced
this pull request
Nov 14, 2026
This was referenced Nov 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
diff --applicationbecame 7–13x slower with #964, and four development-corpus members exceedrun.py's 900 s cap. Most of the time went to re-reading the same modules for every import-search question #964's constructor-ownership checks ask. This PR makes each reader read a module once for those questions, and gives identical answers (answer_idand every row) on all 20 members sampled, including the four that exceeded the cap and now finish in 435–768 s. The full development corpus confirms it: all 49 answers are identical to main's, andrun.pyexits 0 with every member under the cap. The slowest member takes 575 s, and member time falls from 7,982 s to 3,705 s (see Corpus comparison).Refs #1046 (part of its run-scoped caching; not its 300-module budget or its
BuilderCallscounter, so it stays open).Symptom
Seconds per member from saved
run.pyruns (runs.json): 10-08 main before #964 (out-964-main) → #964's head (out-964-pr) → the 1.3.0 release build04fc9ced(out-cut-04fc9ced, load ~10; itssrc/is byte-identical to main). Summed over the 49 members: 1,374 s → 7,383 s → 7,982 s.VidulaWickramasinghe/O.R.I.O.N#126alliance-genome/agr_ai_curation#842/#858/#860Lling0000/OpenCMO#41liauto-siada/siada-cli#6Tiendat2703/MIS_TALENT#7SponsioLabs/Sponsio#173ITMO-NSS-team/FEDOT.MAS#36Profile and cause
cProfile of
SponsioLabs/Sponsio#173(diff --application, pinned base/head) on 10-08 main (c1c0691153, the parent of #964's merge), main (26bfa733b3) and this PR. The machine was shared, so the wall-clock columns are inflated by load (noted); the call counts are deterministic and are the comparison.ast.walkcalls_import_search_context: calls, cumulative s_type_checking_onlycalls_import_guardedcalls_imported_pathscalls_attribute_patchescallsCause, all introduced by #964 (5a22373, "read returned tool factories with constructor ownership checks";
_import_search_context,_import_search_issueandIMPORT_SEARCH_PATCHfirst appear there):_no_import_patch→_import_search_limit(runners). That answer is cached per runner list, but the lists differ from one resolution to the next (318 on Sponsio#173), and_import_search_contextrebuilt each list's closure from scratch: for every module it reached it re-ran_type_checking_onlyand_import_guarded(two whole-tree walks plus provider checks), walked the tree again for imports, and re-resolved every import (_imported_paths,_enclosing_packages). The closures of one read overlap almost entirely: about 750 module reads were repeated 28,014 times. This is the superlinear term (closures × closure size). It dominates the slow members too: timers onLling0000/OpenCMO#41on main put 2,342 s of 3,224 s in_import_search_contextand 3,122 s in_no_import_patch._import_search_issue(module, carriers)was cached per(tree, carriers), and the carrier set differs between closures, so the same module was read again for each set: 4,833 readings for 1,613 module trees onTiendat2703/MIS_TALENT#7, 42% of its CPU profile once (1) was fixed.ImportResolver, and each parses the closures it scans;_modulederives_attribute_patches(which feat(readers): read returned tool factories with constructor ownership checks #964 extended with_import_search_effects) for every parse. Timers onagr_ai_curation#860after (1): 19,270 module builds,_module444 s of 908 s.#1010 and #1016–#1019 add no comparable term in these profiles; the release build is 8% slower in total than #964's head, which these changes do not separately account for.
Fix
Three memos, each of a computation that does not depend on what varies between the calls that repeat it. Commits are separate for review.
ImportResolver._import_search_edges— what one module adds to any closure (whether it importssys/site, the caveat for each import no file provides, each module file an import runs) is read once per(path, parsed tree)per resolver and replayed in reading order by_import_search_context. The closure, the carrier set, the propagation and every verdict are still computed per invocation.ImportResolver._import_search_issue_uncomputed— the module's reading (_read_import_search_issue, the former body line for line) does not involve the carriers; they enter at two checks (a retained namespace, the files an unread import runs), which are now evaluated per call against a per-tree reading. Eachreturnbecame a terminal event, each carrier check an event with its data and message._module(..., parsed=True)— where the caller has just madetreewithast.parse(text)(ImportResolver.module,_patch_scan,_layout_module, and_RepositoryConstructorResolver.module, which wraps_layout_module's tree), the syntax-only_attribute_patches(tree)comes from a process-wide memo keyed by the text's SHA-256; every module gets its own copy.entry(), whose tree comes from the caller, is unchanged.Why none of this can change an answer:
_constructor_reference_modeor a read budget, and their own inputs (listings, the layout, provider proofs) were already cached per resolver (_runtime_attribute_patchesalready caches_type_checking_onlyper resolver and tree). (2) is the existing_import_search_issuescontract with the carriers factored out: the reading depends on the tree and, through_constructor_reference, on resolution state that only changes by parsing; so it is used only while_import_search_budget_spent()is false, as the existing cache's store rule requires, and every call past a read bound reads afresh and keeps nothing. (3) is a pure function of the syntax tree, andast.parseof the same text gives the same tree; the key is the exact text's digest.ImportResolverinstance, keyed by the tree they hold (as_import_search_limitsdoes). Base and head, each source, each_reachbinding and each_CapturedImportSearchResolver/_RepositoryConstructorResolverhave their own. (3) is content-addressed, so sharing it across readers, sides or runs cannot mix files; it starts over past 8,192 entries._LazyReading): replaying the events read so far makes no lookup, and reading resumes only when a caller needs the next event. So a closure cut short byMAX_PATCH_SCAN_MODULES, or a carrier set refused at the first check, has made exactly the lookups the uncached code made, in the same order, and no lookup (a parse that spendsMAX_MODULES, a directory listing an identity-bound snapshot records inscan/verify) happens earlier or more often than before._import_search_contextstill calls_patch_scanfor each module it dequeues andenqueuefor each edge, so BFS order,unreadorder and both module bounds fire where they did. A reading that ends in a bound refusal (exceeds) is dropped, so a later call meets the bound again, as_import_search_issuerequires._Stopraised while reading ends the reading and is re-raised at the same point for every caller that reaches it (stopped closures are still never stored). Any other exception is not kept: the record is marked broken and the next caller reads afresh. A reading is never re-entered: a nested call for a module whose reading is in progress reads afresh, uncached.inputs/builder_calls.pyis untouched. Theopenai_sdk_staticandgoogle_adkadapters (scan/verify) sharepython_imports; their tests and goldens pass unchanged.Regression guard
Five deterministic tests in
tests/test_import_search_ownership.py, no wall-clock budget. Each fails on main and on the commit before the change it guards (checked by running it against those sources).test_each_module_is_read_once_for_every_closure_that_reaches_it— seven runner lists over one read (a shared package, an optional import, a typing-only import, an import above the scope, a search-path change): each list is still its own closure and gets a fresh resolver's answer, while each module's guards are walked once. Main: 31 walks for 7 modules.test_a_stop_reading_a_module_is_raised_by_every_closure_that_reaches_it— an import of a symbolic link refuses every closure with a fresh resolver's reason and detail, each time; the module is read once.test_a_closure_cut_short_reads_no_import_past_its_bound— with the module bound at 3,_imported_pathsis asked about exactly the first three imports; with the bound restored, the next closure reads only the fourth.test_a_module_is_read_once_for_every_carrier_set— five carrier sets over a module with both carrier checks give a fresh resolver's answer each (line 3 namespace, line 4 unread import, none, line 3, none), and each reference is followed once in all, as for one empty set. Before change 2: 18 follows for 6 references.test_a_file_version_patch_map_is_computed_once_for_every_reader— four readers of one file compute its patch map once, each module gets an equal, equally ordered, separate copy, and an edited file is computed again.Timings and identical answers
Every sampled member was run with
benchmark/application-q2/run.py's exact per-member logic from detached pinned worktrees of715f23e0(this head'ssrc/; the commits after it touch onlyCHANGELOG.md), and compared with--require-identicalagainst the release build's answers (out-cut-04fc9ced) and, for the timeout members, main's uncapped answers (out-main-26bfa733-uncapped4). All 20 sampled answers are identical (answer_idand every row), includingO.R.I.O.N#126(453 rows), and all threeagr_ai_curationmembers (#858run directly, with function timers, rather than throughrun.py). The intermediate commits were checked too: change 1 alone (a3b3c74c) on all 18 members, changes 1–2 (1aeba50e) on 13; all identical.Seconds (wall;
run.pycap 900). "Release" isout-cut-04fc9cedat load ~10. This PR ran on a shared machine; its load is the 1-minute average at start, and CPU (user+sys) is given where load moved during the run.VidulaWickramasinghe/O.R.I.O.N#126alliance-genome/agr_ai_curation#842alliance-genome/agr_ai_curation#858alliance-genome/agr_ai_curation#860Lling0000/OpenCMO#41liauto-siada/siada-cli#6Tiendat2703/MIS_TALENT#7SponsioLabs/Sponsio#173mosnin/realestatecrm#612xju2/hepagent#74ITMO-NSS-team/FEDOT.MAS#36geroembser/livekit-agents#1mezgoodle/Vesta#53InseeFrLab/GRAAL#33krevetka-is-afk/llm-agent-for-ai-studio#20rustic-ai/rustic-ai#419jpka/attest#3hamzafarooq/multi-agent-course#64cuppibla/adk_tutorial#9JSebastianIEU/voice-freight-negotiator#10¹ Run directly with function timers on, not through
run.py.Load rose to ~57 during
realestatecrm#612,hepagent#74andrustic-ai#419, so their wall times are not comparable; main on the same machine at load ~8 used 124, 187 and 6 CPU-seconds on them (126.6, 194.0 and 6.2 s wall), against this PR's 79, 96 and 7. The small members do not reach the import-search path and are within noise. Per change, at load ~8–14: change 1 alone tookSponsio#173to 45 s,MIS_TALENT#7to 92 s,realestatecrm#612to 91 s,hepagent#74to 97 s andagr_ai_curation#842to 668 s; change 2 tookMIS_TALENT#7to 74 s; change 3 tookagr_ai_curation#842to 562 s.What remains slow:
alliance-genome/agr_ai_curation#858and#860.#858on this head took 768 s wall and 563 CPU-seconds at load 22–43 (with function timers on; answer identical to main's uncapped answer). Its time:_no_import_patch414 s, of which_import_search_limit266 s;_patch_scan199 s over 214,298 calls and 18,461 module builds (each reader parsing the modules it scans;_moduleitself is down to 85 s from 444 s on#860before change 3);_attribute_patches152 s over 6,069 calls (memo misses, and calls with a reader's own exclusion set from_runtime_attribute_patches, which the per-text memo does not cover);_type_checking_only50 s over 16,435 calls.#860took ~910 s with change 1 alone (timers on, load ~8) and 608 s on this head (load 23 falling to 3). Both are under the cap,#858with the least margin: it timed out at 900 s once while the machine's load was 56–85. More margin needs #1046's next step: one parse per file and side shared by every reader, rather than one per reader.Corpus comparison (development, 49 PRs)
Run.
benchmark/application-q2/run.py --jobs 4from a detached worktree of79b56830bf, over~/.cache/shipgate-q2/reposwith every pin present (no--fetch), 900 s cap. This head (6d728a9bfc) has the samesrc/; its last commit only moves the CHANGELOG entry. The run started at 08:08:23Z, at a one-minute load of 2.1, and finished at 08:24:21Z (16 minutes) at load 5.8. Output:~/.cache/shipgate-q2/out-pr2-79b56830.run.pyexited 0: all 49 members answeredokwithin the cap.Current head (2026-11-10). The corpus was measured on
79b56830bf. The head is now170995867f, which merges main's #1130 in; that merge's only conflict wasCHANGELOG.md. #1130 changescore.trust_roots,core.agent_boundaryandschemas.verification, whichdiff --applicationimports. So I re-ran 8 members on170995867f:jpka/attest#3,surrealdb/agent-memory-openai-agents-sdk#1,cuppibla/adk_tutorial#9,ITMO-NSS-team/FEDOT.MAS#36,Tiendat2703/MIS_TALENT#7,kkmiecik-coder/CRM#5,cianfhoghlaim/tuatha#1andmosnin/realestatecrm#612. All 8 answers are identical to the measured run (answer_idand every row;~/.cache/shipgate-q2/out-pr2-17099586-sample).Pre-fix answers (main):
out-cut-04fc9ced). Itssrc/is byte-identical to this PR's base, and main26bfa733b3reproduces itsanswer_idforjpka/attest#3, with the same dependency set.26bfa733b3re-run withrun.py's exact per-member logic and a 10,800 s cap (out-main-26bfa733-uncapped4). They took O.R.I.O.N#126 2,579 s, agr_ai_curation#842 3,053 s, An unrelated added allow rule splits a widened replacement into 'added' plus a misleading 'removed' #858 3,913 s and Route an enabled in-repository plugin's hook at a non-registry path to review (#809) #860 4,571 s, while the machine's load rose from 21 to about 310.Result: every answer is identical.
answer_idissummarize.py's digest of an answer without the engine block and thesummaryreading. Rows are keyed by (scope, agent, agent source, tool).partial, 6not_establishednot_establishedanswer_idchangemoved / rows on one side only / rows with other content changesSeconds for the 13 slowest members. The ledger run was at load 11→8; this run at load 2→6. Every other member takes at most 56 s.
alliance-genome/agr_ai_curation#860alliance-genome/agr_ai_curation#858alliance-genome/agr_ai_curation#842VidulaWickramasinghe/O.R.I.O.N#126Lling0000/OpenCMO#41carlospolop/chack-agent#25liauto-siada/siada-cli#6VRSEN/agency-swarm#804VRSEN/agency-swarm#796xju2/hepagent#74mosnin/realestatecrm#612Tiendat2703/MIS_TALENT#7SponsioLabs/Sponsio#173What this does not restore. The 13 members above still take two to five times as long as before #964 (Sponsio#173 2.2×, agr_ai_curation#860 2.8×, O.R.I.O.N#126 3.2×, OpenCMO#41 4.6×). The slowest, #860, finishes 325 s under the cap at this load. Under heavy load, #858 timed out once in the implementer's runs, at load 56–85. More margin needs #1046's next step, one parse per file and side shared by every reader, which this PR leaves open.
Reproduce:
The comparison exits 0. It reads
runs.jsonand<slug>.jsonfrom each directory; a later directory in a comma list overlays an earlier one. The per-member table is saved as~/.cache/shipgate-q2/pr2-vs-prefix.md.Both PRs together. I merged #1135 and this PR onto main locally (
63e7616938, not pushed, CHANGELOG folded into one group) and ran the full corpus at 08:28–08:50Z.run.pyexits 0: all 49 members answer.: Nonedetails and the 40 MIS_TALENTagents/uvicorntexts. Nothing else changes, and no row changes status.agr_ai_curation#860took 871 s, 29 s under the cap, so under heavy load the threeagr_ai_curationmembers can still reach it.CHANGELOG (main-session review fix,
6d728a9bfc). The entry now sits under### Application comparisonahead of### Packaging, in 1.3.0's group order. #1135 adds the same heading at the same place. In the earlier position, merging both PRs would have left two### Application comparisonheadings without a textual conflict. Whichever PR merges second now gets a CHANGELOG conflict, to be resolved as one group with two entries. The entry's timings now come from this corpus run.Type
Authorship
Verification
CI is authoritative for
python -m ruff check .,python -m compileall -q src tests, andpython -m pytest.Additional local checks run:
ruff check .— passed.-n 4(test_builder_calls.py,test_application_diff*.py,test_constructor_*.py,test_*constructor*.py,test_openai*.py,test_google_adk*.py,test_import_search_ownership.py,test_python_import_resolution.py,test_imported_tool_*.py,test_adk_tool_factories.py, and every test file that namespython_imports,ImportResolver, the SDK/ADK loaders or a golden) — exit 0 on this head; plustest_distribution_surface_parity.py,test_release_pipeline.py,test_release_advisory_pipeline.pyandtest_unread_changed_inputs.pyfor the CHANGELOG.Release-readiness notes
docs/checks.md(none changed)STABILITY.md(none)🤖 Generated with Claude Code