Use two-phase type checking in sequential mode

[ilevkivskyi]

Fixes #21348

This means that (unless local partial types are disabled) we should have identical semantics for all three: sequential (in-process) type-checking, parallel type-checking, and in the daemon.

When reading the relevant code I noticed a bug, when using -c in parallel mode we do not check implementations in the code passed with -c, fix is quite simple (just reorder writing cache and flushing the errors), so I simply add it here.

cc @JukkaL

[ilevkivskyi]

@JukkaL as expected, most of the fallout is caused by fine details of how the partial None types work with --local-partial-types. But IMO it looks good. We get a bunch of unused type ignore and redundant cast, and in few situations where we do get new errors, they look correct/consistent (it worked before by accident). The only thing I decided to change is to allow:

class C:
    x = None

    @classmethod
    def get_x(cls) -> int:
        if cls.x is None:
            cls.x = int()  # This is OK, always allow partial types here
        return cls.x

for consistency with the same exception we have for regular methods. So I don't think there is any need for a new flag.

Btw, I also fix an off-by-one error to make multi-pass logic consistent between single-phase and two-phase type checking.

[JukkaL]

Looks good overall, but how does this interactive with per-file config overrides like # mypy: no-local-partial-types?

[ilevkivskyi]

@JukkaL Effectively, per-file overrides to disable local partial types will stop working. Since although formally the flag will still be set, it is not very useful if we check interface separately. Note this affects only non-None partial types (like x = []). If you are still worried about this, another option is to disable two-phase checking if at least one file in the SCC has local partial types disabled. What do you think?

[ilevkivskyi]

Btw, if local partial types are disabled per-file, the parallel type-checking will generate an error, so maybe what I propose above actually makes sense.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

steam.py (https://github.com/Gobot1234/steam.py)
- steam/chat.py:248: error: "fetch_message_reactors" of "ConnectionState" gets multiple values for keyword argument "server_timestamp"  [misc]
- steam/chat.py:248: error: "fetch_message_reactors" of "ConnectionState" gets multiple values for keyword argument "ordinal"  [misc]
- steam/chat.py:248: error: "fetch_message_reactors" of "ConnectionState" gets multiple values for keyword argument "reaction_name"  [misc]
- steam/chat.py:248: error: "fetch_message_reactors" of "ConnectionState" gets multiple values for keyword argument "reaction_type"  [misc]
- steam/chat.py:290: error: Too many arguments for "react_to_chat_message" of "ConnectionState"  [call-arg]
- steam/chat.py:290: error: "react_to_chat_message" of "ConnectionState" gets multiple values for keyword argument "reaction_type"  [misc]
- steam/chat.py:290: error: "react_to_chat_message" of "ConnectionState" gets multiple values for keyword argument "is_add"  [misc]
- steam/chat.py:318: error: Too many arguments for "ack_chat_message" of "ConnectionState"  [call-arg]

spark (https://github.com/apache/spark)
+ python/pyspark/pandas/frame.py:666: error: Unused "type: ignore" comment  [unused-ignore]
+ python/pyspark/pandas/frame.py:666: error: Redundant cast to "InternalFrame"  [redundant-cast]
+ python/pyspark/pandas/frame.py:666: note: Error code "redundant-cast" not covered by "type: ignore[has-type]" comment
+ python/pyspark/streaming/context.py:226: error: Incompatible types in assignment (expression has type "StreamingContext", variable has type "None")  [assignment]

prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/runner/runner.py:329: error: Incompatible return value type (got "EventsClient | None", expected "EventsClient")  [return-value]
- src/prefect/runner/runner.py:306: error: Cannot determine type of "_scheduled_run_poller"  [has-type]
- src/prefect/runner/runner.py:312: error: Cannot determine type of "_scheduled_run_poller"  [has-type]
- src/prefect/runner/runner.py:329: error: Cannot determine type of "_event_emitter"  [has-type]
- src/prefect/runner/runner.py:670: error: Cannot determine type of "_scheduled_run_poller"  [has-type]
- src/prefect/runner/runner.py:672: error: Cannot determine type of "_scheduled_run_poller"  [has-type]
- src/prefect/runner/runner.py:686: error: Cannot determine type of "_cancellation_manager"  [has-type]
- src/prefect/runner/runner.py:698: error: Cannot determine type of "_scheduled_run_poller"  [has-type]
- src/prefect/runner/runner.py:699: error: Cannot determine type of "_scheduled_run_poller"  [has-type]
- src/prefect/runner/runner.py:796: error: Cannot determine type of "_runs_task_group"  [has-type]
- src/prefect/runner/runner.py:1243: error: Cannot determine type of "_cancellation_manager"  [has-type]
- src/prefect/runner/runner.py:1360: error: Cannot determine type of "_event_emitter"  [has-type]
- src/prefect/runner/runner.py:1368: error: Cannot determine type of "_event_emitter"  [has-type]

operator (https://github.com/canonical/operator)
- ops/lib/__init__.py:86: error: "None" has no attribute "get"  [attr-defined]
+ ops/lib/__init__.py:86: error: Item "None" of "Any | None" has no attribute "get"  [union-attr]

altair (https://github.com/vega/altair)
+ altair/vegalite/v6/api.py:3997: error: Argument "spec" to "FacetChart" has incompatible type "_EncodingMixin"; expected "SchemaBase | Mapping[str, Any] | UndefinedType"  [arg-type]

egglog-python (https://github.com/egraphs-good/egglog-python)
+ python/egglog/builtins.py:271: error: Unused "type: ignore" comment  [unused-ignore]
+ python/egglog/builtins.py:278: error: Unused "type: ignore" comment  [unused-ignore]
+ python/egglog/builtins.py:394: error: Unused "type: ignore" comment  [unused-ignore]
+ python/egglog/builtins.py:401: error: Unused "type: ignore" comment  [unused-ignore]
+ python/egglog/builtins.py:873: error: Unused "type: ignore" comment  [unused-ignore]
+ python/egglog/builtins.py:879: error: Unused "type: ignore" comment  [unused-ignore]
+ python/egglog/builtins.py:974: error: Unused "type: ignore" comment  [unused-ignore]
+ python/egglog/builtins.py:981: error: Unused "type: ignore" comment  [unused-ignore]
+ python/egglog/builtins.py:1104: error: Unused "type: ignore" comment  [unused-ignore]
+ python/egglog/builtins.py:1110: error: Unused "type: ignore" comment  [unused-ignore]

pip (https://github.com/pypa/pip)
+ src/pip/_internal/resolution/resolvelib/candidates.py:410: error: Incompatible types in assignment (expression has type "Version", variable has type "None")  [assignment]
+ src/pip/_internal/resolution/resolvelib/candidates.py:411: error: Incompatible return value type (got "None", expected "Version")  [return-value]

pandera (https://github.com/pandera-dev/pandera)
- pandera/engines/polars_engine.py:270: error: "type[object]" has no attribute "to_schema"  [attr-defined]
+ pandera/api/pandas/array.py:38: error: Unused "type: ignore" comment  [unused-ignore]

sympy (https://github.com/sympy/sympy)
+ sympy/matrices/matrixbase.py:3319: error: Unused "type: ignore" comment  [unused-ignore]

trio (https://github.com/python-trio/trio)
+ src/trio/_dtls.py:739: error: Incompatible types in assignment (expression has type "tuple[bytes, bytes]", variable has type "None")  [assignment]

jax (https://github.com/google/jax)
- jax/_src/array.py:409: error: No overload variant of "asarray" matches argument types "ndarray[tuple[Any, ...], dtype[Any]]", "dtype[Any] | None", "dict[str, bool]"  [call-overload]
+ jax/_src/array.py:409: error: No overload variant of "asarray" matches argument types "Any", "dtype[Any] | None", "dict[str, bool]"  [call-overload]

git-revise (https://github.com/mystor/git-revise)
+ gitrevise/odb.py:413: error: Redundant cast to "bytes"  [redundant-cast]

[ilevkivskyi]

@JukkaL It turns out we didn't actually give an error in parallel mode when --local-partial-types is disabled per-file. This is definitely wrong, so I fix this. Now it is consistent with how --local-partial-types works with --allow-redefinition: if problem appears in global config we give a blocker, if it is per-file we give a non-blocking error.

This caused a bunch of tests to fail in parallel mode. But we can safely remove --no-local-partial-types from them, since they actually worked before, because they use None partial types. After removing this option, few tests became duplicates, so I deleted them. Instead, I added couple more tests to test non-None partial types, since those actually behave differently with and without --local-partial-types.

I also implemented my idea of falling back to single phase checking if at least one file in an SCC has --local-partial-types disabled. I think this will give the best UX: in sequential mode there now should be ~0 negative consequences, while in parallel mode we will (gently) push people to enable --local-partial-types (if it was disabled somewhere).