Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E)

[anth-volk]

Fixes #798

Same-Repository Draft PR Check

• This PR is a draft.
• This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

Phase 1 step 1.8, group 7: structure and imports. Bottom of the stack since #785 merged.

• Test exemption (decided for this group): **/tests/** and **/authenticated_tests/** now also ignore SLF001, INP001 and PLC0415, alongside S101, D1, ANN and PLR2004. Tests deliberately reach into private members, pytest test directories aren't packages, and tests often import inside a test after patching the environment. Application code still gets every rule. Both test-exemption rules in the registry are updated to match.
• Rules: ruff.toml gains A, E, FAST, INP, N, PLC and SLF. All 339 findings left after the exemption are fixed or kept with an inline reason.
• Imports inside functions (PLC0415, 176):
  • Moved (16): only imports that load nothing new, create no cycle and aren't patched by tests. Each was checked by importing every affected module in a fresh interpreter.
  • Listed as lazily loaded (51 suppressions removed):
    • What's listed: 14 heavy or optional packages and modules that no application module imports at module level:
      • the policyengine bundle, countries, provenance and tax_benefit_models modules, and policyengine_us;
      • spm_calculator, h5py, google.cloud and pythonjsonlogger;
      • three OpenTelemetry exporters and instrumentations;
      • the executor's spm and baseline_artifacts modules.
    • Mechanism: they're in ruff's banned-module-level-imports, with TID253 selected. Importing one at module level now fails, though TYPE_CHECKING imports are fine. PLC0415 accepts their in-function imports without a suppression.
    • Tests may still import them at module level, because TID253 joins the test exemptions.
  • Kept (116), each with a reason: Modal and image code, test seams, packages that some application module does import at module level (pandas, huggingface_hub, policyengine.core and others), and import cycles. Roadmap items cover the Modal imports (image.imports()) and the test seams.
• Namespace packages (INP001, 13): .github/scripts, scripts, the executor's scripts and rules/fixtures aren't packages, so they're declared in ruff's top-level namespace-packages setting. Compared with the full ruff check output before the change, nothing else differs. The nine executor src/modal/utils programs take INP001 in their existing file-level entry (a program under src) instead of inline. Declaring that directory as a namespace package made ruff treat import modal (the Modal SDK) as first-party, and an __init__.py would change the source copied into the Modal images. A per-file ignore changes neither.
• Names (N, A, PLC0414, SLF001):
  • Renamed exceptions: the entry app's BackendUnavailableError, BackendTimeoutError and TemporaryStage12UnsupportedRequestError, and the executor's ParityMismatchError. Each is caught by name and never logged or stored by class name.
  • Kept: TemporaryStage12DispatchFailed, because the entry app logs its class name as error_type.
  • Published fields: the 14 camelCase budget-window fields in gateway_models.py keep their names.
  • Other renames: three OpenTelemetry locals, the exit import aliased as app_exit, and four gateway test names shortened for line length.
  • Kept with a reason: the contract's explicit re-exports, the exit object projects import, and two private accesses with no public equivalent.
• FastAPI (FAST001, 13): response_model arguments that repeat the return type are dropped. The gateway golden OpenAPI is unchanged, and the entry, executor and gateway OpenAPI dumps are identical before and after.
• Line length (E501): wrapped, with every split string's text identical, which a syntax-tree comparison confirms. Four multi-line fake shell scripts in tests keep a suppression, because their one-line printf JSON can't be split without changing the script.
• Modal images: many executor and gateway image files change, mostly in import placement and line wrapping; src/modal/app.py gains no __future__ import. CI's image smoke will run when this PR reaches the bottom of the stack.
• Registry: sim.quality.ruff-lint now lists the seven families and TID253. The rules tool is synced from the Public API's group 7 (Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E) policyengine-api#3920).

Testing

• Suites: at this branch's tip after today's restack onto main, all seven passed:

  Project	Passed
  executor	1,190 (22 skipped)
  entry	529
  gateway	243
  contract	192
  observability	31
  stage12	17
  fastapi	14

  The lazy-import list and the INP001 change touch project code only in noqa comments and configuration. The suites weren't re-run for them.

• At each tip from Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E) #799 to Register weighted-microdata rules, weight the builder tests, and check change weights and household counts #807:

  • ruff check and ruff format --check (0.15.22) are clean. From Enable ruff argument, idiom and pytest-style rules (ARG, B, C4, FBT, PERF, PLW, PT, PTH, RUF, SIM and others) #802 up, that includes RUF100, so no suppression is left unused.
  • The rules guard, the ci-scripts guard and the rules tool unittests pass.
  • Rendered rules are fresh at every commit.
  • Project Python differs from the previous version only in noqa comments.

• The lazy-import mechanism was checked with ruff 0.15.22 in a small probe:

  • listing a package exempts in-function imports of it and its submodules from PLC0415;
  • TID253 flags a module-level import of it;
  • an import under TYPE_CHECKING is still allowed.

• Golden OpenAPI: the gateway golden OpenAPI test passes, and the golden file is unchanged.

• Integration tests: the conftest change in the integration-test project was checked by parsing only, because its tests need live services.

• CI/CD: no workflow changed, and no environment variable was added or rewired.

• CI: this is now the lowest unmerged PR in the stack, so its full CI runs.

🤖 Generated with Claude Code