Skip to content

1.3.0 regression: gradual overload call inferred as Unknown #4910

Open

Description

@jorenham

Describe the Bug

I noticed in scipy-stubs that after upgrading Pyrefly to 1.3.0, many assert_type tests started failing (scipy/scipy-stubs#2371) because Pyrefly now infers Unknown when part of the input (shape-types, usually) is gradual.

Here's a minimal-ish reproducer:

from typing import Any, overload, reveal_type

@overload
def f(a: tuple[int]) -> tuple[str, int]: ...
@overload
def f(a: tuple[int, int]) -> tuple[str, list[int]]: ...
@overload
def f(a: tuple[int, ...]) -> tuple[str, int | list[int]]: ...

x: tuple[Any, ...]
reveal_type(f(x))
# 1.2.0: tuple[str, int | list[int]]
# 1.3.0: Unknown

The entire return type is now inferred as Unknown, which is a lot broader than what the overload typing spec currently requires (after python/typing#2350).

The strange thing is that when the returned int | list[int] is replaced with Any, then Pyrefly 1.3.0 suddenly behaves as expected and infers tuple[str, Any], same as 1.2.0. So by making the return type more gradual, Pyrefly infers a less gradual return type 🤔. To me, this sounds a lot like a regression.

I also thought that such "gradual in the face of ambiguity" overload behavior was intended to stay opt-in through the --spec-compliant-overloads flag?

Sandbox Link

https://pyrefly.org/sandbox/?project=v2.w1VkkuQ7oGKE5kw8BS7OQssBJgPNAhqJVgolpcBcGg2M0lhwMoJwgWlUBxjvJbFWCnp6egQ0QlSi6wYXCSAJIs0AqsHmAmCGxDApJq8CphXsX5DWmDwkv2qkaVRoauKJkUylWgA

(Only applicable for extension issues) IDE Information

No response

Activity

  1. jorenham commented on Sep 11, 2026

    @jorenham
    CollaboratorAuthor
  2. rchen152 commented on Sep 12, 2026

    @rchen152
    Contributor

    This looks correct to me. The overload spec doesn't differentiate between a fully gradual or partly gradual argument type. The relevant part says:

    For all arguments, determine whether all possible materializations of the argument’s type are assignable to the corresponding parameter type for each of the remaining overloads. If so, eliminate all of the subsequent remaining overloads.

    tuple[Any, ...] can materialize to a lot of things that are not assignable to each of the parameter types, so we have to keep all of the candidate overloads. Previously, pyrefly would then select tuple[str, int | list[int]], but that's not assignable to the other return types, so we now fall all the way back to Any. That's the update I made to be compliant to the spec.

    The reason pyrefly will still select tuple[str, Any] is that it is assignable to the other return types. I agree that that's a sharp edge. Ideally, we'd fix it by also inferring something more precise than Any in the tuple[str, int | list[int]] case. Given the history of overload evaluation (lots of complexity, lots of divergence between type checkers), I've been reluctant to introduce a rule that isn't standardizable. I would be open to experimenting with something like AnyOf with an eye towards working out the semantics and standardizing it, but I doubt I'd have time to drive it. (That's why I was excited about the proposal I made a few months back - it used only concepts already in the type system :) )

    I also thought that such "gradual in the face of ambiguity" overload behavior was intended to stay opt-in through the --spec-compliant-overloads flag?

    I tried to be clear in the Discuss thread that I didn't intend to keep pyrefly's spec-incompliant behavior. I don't want to create a situation where pyrefly is blocking progress because it's doing something that the other checkers can't or won't copy, but the behavior is too entrenched for us to change.

  3. added
    needs-discussionAn issue where it's not clear whether there is a bug or we are behaving as expected.
    on Sep 12, 2026
  4. jorenham commented on Sep 14, 2026

    @jorenham
    CollaboratorAuthor

    With my typing council hat on, I agree this is correct. But it would also be correct to infer tuple[int, Unknown] instead of Unknown here, which would make this more consistent with the tuple[int, Any] case and would make this less type-unsafe. Ideally, AnyOf would be better, or alternatively treating the ambiguous return type as a "free" type variable, i.e. def f[T: int | list[int]](a: tuple[int, ...]) -> tuple[str, T]: ..., which would be similar to AnyOf + constraint solving.

    With my NumPy/scipy-stubs maintainer hat on, I feel obligated to push back on this. Because the consequence of this new Pyrefly behavior is that many calls to numpy and scipy-stubs functions will now result in Unknown: the incarnation of type-unsafety. That's because it's very common for users to annotate their array types as np.typing.NDArray[T], a gradual alias of np.ndarray[tuple[Any, ...], np.dtype[T]]. And if you combine that with the fact that many, if not most, of NumPy's and scipy-stubs' functions are overloaded for shape-types, I foresee that very many calls to these functions will now be Unknown when using Pyrefly. As a maintainer there is no way for me to control this behavior either, like I can for mypy and basedpyright using the "Never-trick" (prepending an overload with tuple[Never, Never, Never] as shape-type, because mypy/pyright will then eagerly select this overload in case of tuple[Any, ...]).
    My worry is that users won't switch to Pyrefly, or even switch to another type-checker, because of this. And if they do, I wouldn't blame them tbh...

  5. rchen152 commented on Sep 15, 2026

    @rchen152
    Contributor

    I realize this is a painful change, and I'm fully aware that users have other options for type checking. I didn't think it was reasonable to put it off longer once it became clear that what pyrefly was doing wasn't going to make it into the spec, and Any is the best honest alternative we have right now, as bad as it is.

    Another option that just occurred to me is that we may be able to use Type::Overloaded from this proposal. The semantics are roughly what we want (if rather circular as currently described - "calls follow overload semantics" isn't very helpful!) My progress there has been slow so far because it keeps getting preempted by more urgent work, but I should have some time this week to hack on it.

  6. changed the title [-]1.3.0 regression: gradual overload call inferred as `Unknown` when overload returns `T | Any`[/-] [+]1.3.0 regression: gradual overload call inferred as `Unknown`[/+] on Sep 15, 2026
  7. jorenham commented on Sep 15, 2026

    @jorenham
    CollaboratorAuthor

    Ideally, we'd fix it by also inferring something more precise than Any in the tuple[str, int | list[int]] case.

    The typing spec now also allows this to be inferred as tuple[str, Any], which you could call the "gradual meet" of tuple[str, int] and tuple[str, list[int]], i.e. the most specific gradual type assignable to both. So for example for overloads returning ndarray[tuple[int]] and ndarray[tuple[int, int]] that'd be ndarray[tuple[Any, ...]]. After AnyOf this seems like the next best solution.

    TBH I haven't read your Type::Overloaded proposal, so maybe I'm reinventing the wheel here, but do you think something like that could work? Because I think that this would solve this issue for at least NumPy and scipy-stubs.

  8. rchen152 commented on Sep 24, 2026

    @rchen152
    Contributor

    Type::Overloaded is closer to an ordered AnyOf than a gradual meet. Anyway, I did manage to get Type::Overloaded in last week, so let me check if it can be dropped in here to give us a better return type than Any for ambiguous evaluation.

  9. self-assigned this
    on Sep 24, 2026
  10. added this to the v1.4 milestone on Oct 7, 2026
  11. rchen152 commented on Oct 13, 2026

    @rchen152
    Contributor

    Unfortunately, this isn't going to make it into 1.4 :( Making overload evaluation use Overloaded causes it to show up in a lot more places than before, and all those places need new Overloaded handling.

  12. removed this from the v1.4 milestone on Oct 13, 2026
  13. jorenham commented on Oct 13, 2026

    @jorenham
    CollaboratorAuthor

    Ah ok too bad, but makes sense. Anyway, thanks a lot for digging into this :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

needs-discussionAn issue where it's not clear whether there is a bug or we are behaving as expected.overloadsquansight

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions