gh-157757: Fix to make lazy import a.b as c import the module a.b - #158092
brittanyrey wants to merge 5 commits into
Conversation
`import a.b as c` compiles to `IMPORT_NAME a.b` followed by `IMPORT_FROM b`. Lazily, IMPORT_NAME leaves a placeholder holding "a.b", and IMPORT_FROM rewrote it into the placeholder `lazy from a import b` produces. Reification then imported `a` alone and read `b` off it, so the module `a.b` was never imported under its own name: an attribute of the package shadowing it answered instead, and `math.pi`, which no module backs, bound the float where the eager statement raises ModuleNotFoundError. Mark the dotted import on the placeholder and keep the whole name on it. Reification imports that name and then walks its components with IMPORT_FROM, which is what the eager statement does. The test pinning `lazy import math.pi as pi` as working is inverted, since the eager statement raises.
It passes now that a lazy `import a.b as c` imports the module: the KeyError on 'test.tracedmodules.testmod' came from the submodule never being imported under its own name.
hugovk
added
interpreter-core
The flag means "bind the whole dotted name, not the root", which the old name did not say, and import.c already has unrelated lazy_pending_submodules machinery to be confused with.
brittanyrey
changed the title
import a.b as c namesimport a.b as c names
brittanyrey
changed the title
import a.b as c nameslazy import a.b as c import the module a.b
|
Hmm, maybe we can have each deferred For Perhaps we can use the existing fields for this: I prototyped this and it seems to work. We would keep the intermediate placeholders around until resolution, so it uses more memory. I think it’s worth considering, though. It follows what the bytecode does and avoids needing the extra flag. |
Each deferred IMPORT_FROM off a placeholder without a fromlist now keeps the previous placeholder in lz_from and the attribute name in lz_attr. Reification walks back to the placeholder IMPORT_NAME left, runs that import, and replays the lookups in order with _PyEval_ImportFrom, which is what the eager bytecode does. This drops the lz_dotted_as flag and also follows a custom __lazy_import__ that returns a placeholder for a different module name.
|
Thanks @pablogsal! That was very useful context + direction. |
`lazy from a import b` now records its lookup the same way as `import a.b as c`, so a placeholder holds either the module name and fromlist or the previous placeholder and an attribute name, and reification has a single path. The import passes only the name being resolved as the fromlist, so accessing b still does not import the other names' submodules.
3 participants
Bug:
import a.b as ccompiles toIMPORT_NAME a.bfollowed byIMPORT_FROM b. Lazily, IMPORT_NAME leaves a placeholder holding "a.b", and IMPORT_FROM rewrote it into the placeholderlazy from a import bproduces. Reification then importedaalone and readboff it, so the modulea.bwas never imported under its own name: an attribute of the package shadowing it answered instead, andmath.pi, which no module backs, bound the float where the eager statement raises ModuleNotFoundError.Fix: Follow @pablogsal's suggestion to have each deferred
IMPORT_FROMkeep the previous placeholder and the attribute name. Reification runs the original import and then applies the recorded lookups in order with_PyEval_ImportFrom, as the eager statement does.For
lazy from a import b, c, the import gets only the name being resolved as thefromlist, so accessingbdoesn't importa.c.lazy import a.b as cresolvesbas an attribute ofainstead of importing the modulea.b#157757