You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
1.3.0 regression: gradual overload call inferred as Unknown #4910
I noticed in scipy-stubs that after upgrading Pyrefly to 1.3.0, many assert_type tests started failing (scipy/scipy-stubs#2371) because Pyrefly now infers Unknown when part of the input (shape-types, usually) is gradual.
The entire return type is now inferred as Unknown, which is a lot broader than what the overload typing spec currently requires (after python/typing#2350).
The strange thing is that when the returned int | list[int] is replaced with Any, then Pyrefly 1.3.0 suddenly behaves as expected and infers tuple[str, Any], same as 1.2.0. So by making the return type more gradual, Pyrefly infers a less gradual return type 🤔. To me, this sounds a lot like a regression.
I also thought that such "gradual in the face of ambiguity" overload behavior was intended to stay opt-in through the --spec-compliant-overloads flag?
This looks correct to me. The overload spec doesn't differentiate between a fully gradual or partly gradual argument type. The relevant part says:
For all arguments, determine whether all possible materializations of the argument’s type are assignable to the corresponding parameter type for each of the remaining overloads. If so, eliminate all of the subsequent remaining overloads.
tuple[Any, ...] can materialize to a lot of things that are not assignable to each of the parameter types, so we have to keep all of the candidate overloads. Previously, pyrefly would then select tuple[str, int | list[int]], but that's not assignable to the other return types, so we now fall all the way back to Any. That's the update I made to be compliant to the spec.
The reason pyrefly will still select tuple[str, Any] is that it is assignable to the other return types. I agree that that's a sharp edge. Ideally, we'd fix it by also inferring something more precise than Any in the tuple[str, int | list[int]] case. Given the history of overload evaluation (lots of complexity, lots of divergence between type checkers), I've been reluctant to introduce a rule that isn't standardizable. I would be open to experimenting with something like AnyOf with an eye towards working out the semantics and standardizing it, but I doubt I'd have time to drive it. (That's why I was excited about the proposal I made a few months back - it used only concepts already in the type system :) )
I also thought that such "gradual in the face of ambiguity" overload behavior was intended to stay opt-in through the --spec-compliant-overloads flag?
I tried to be clear in the Discuss thread that I didn't intend to keep pyrefly's spec-incompliant behavior. I don't want to create a situation where pyrefly is blocking progress because it's doing something that the other checkers can't or won't copy, but the behavior is too entrenched for us to change.
With my typing council hat on, I agree this is correct. But it would also be correct to infer tuple[int, Unknown] instead of Unknown here, which would make this more consistent with the tuple[int, Any] case and would make this less type-unsafe. Ideally, AnyOf would be better, or alternatively treating the ambiguous return type as a "free" type variable, i.e. def f[T: int | list[int]](a: tuple[int, ...]) -> tuple[str, T]: ..., which would be similar to AnyOf + constraint solving.
With my NumPy/scipy-stubs maintainer hat on, I feel obligated to push back on this. Because the consequence of this new Pyrefly behavior is that many calls to numpy and scipy-stubs functions will now result in Unknown: the incarnation of type-unsafety. That's because it's very common for users to annotate their array types as np.typing.NDArray[T], a gradual alias of np.ndarray[tuple[Any, ...], np.dtype[T]]. And if you combine that with the fact that many, if not most, of NumPy's and scipy-stubs' functions are overloaded for shape-types, I foresee that very many calls to these functions will now be Unknown when using Pyrefly. As a maintainer there is no way for me to control this behavior either, like I can for mypy and basedpyright using the "Never-trick" (prepending an overload with tuple[Never, Never, Never] as shape-type, because mypy/pyright will then eagerly select this overload in case of tuple[Any, ...]).
My worry is that users won't switch to Pyrefly, or even switch to another type-checker, because of this. And if they do, I wouldn't blame them tbh...
I realize this is a painful change, and I'm fully aware that users have other options for type checking. I didn't think it was reasonable to put it off longer once it became clear that what pyrefly was doing wasn't going to make it into the spec, and Any is the best honest alternative we have right now, as bad as it is.
Another option that just occurred to me is that we may be able to use Type::Overloaded from this proposal. The semantics are roughly what we want (if rather circular as currently described - "calls follow overload semantics" isn't very helpful!) My progress there has been slow so far because it keeps getting preempted by more urgent work, but I should have some time this week to hack on it.
Ideally, we'd fix it by also inferring something more precise than Any in the tuple[str, int | list[int]] case.
The typing spec now also allows this to be inferred as tuple[str, Any], which you could call the "gradual meet" of tuple[str, int] and tuple[str, list[int]], i.e. the most specific gradual type assignable to both. So for example for overloads returning ndarray[tuple[int]] and ndarray[tuple[int, int]] that'd be ndarray[tuple[Any, ...]]. After AnyOf this seems like the next best solution.
TBH I haven't read your Type::Overloaded proposal, so maybe I'm reinventing the wheel here, but do you think something like that could work? Because I think that this would solve this issue for at least NumPy and scipy-stubs.
Type::Overloaded is closer to an ordered AnyOf than a gradual meet. Anyway, I did manage to get Type::Overloaded in last week, so let me check if it can be dropped in here to give us a better return type than Any for ambiguous evaluation.
Unfortunately, this isn't going to make it into 1.4 :( Making overload evaluation use Overloaded causes it to show up in a lot more places than before, and all those places need new Overloaded handling.
Activity
jorenham commentedon Sep 11, 2026
cc @rchen152
rchen152 commentedon Sep 12, 2026
This looks correct to me. The overload spec doesn't differentiate between a fully gradual or partly gradual argument type. The relevant part says:
tuple[Any, ...]can materialize to a lot of things that are not assignable to each of the parameter types, so we have to keep all of the candidate overloads. Previously, pyrefly would then selecttuple[str, int | list[int]], but that's not assignable to the other return types, so we now fall all the way back toAny. That's the update I made to be compliant to the spec.The reason pyrefly will still select
tuple[str, Any]is that it is assignable to the other return types. I agree that that's a sharp edge. Ideally, we'd fix it by also inferring something more precise thanAnyin thetuple[str, int | list[int]]case. Given the history of overload evaluation (lots of complexity, lots of divergence between type checkers), I've been reluctant to introduce a rule that isn't standardizable. I would be open to experimenting with something likeAnyOfwith an eye towards working out the semantics and standardizing it, but I doubt I'd have time to drive it. (That's why I was excited about the proposal I made a few months back - it used only concepts already in the type system :) )I tried to be clear in the Discuss thread that I didn't intend to keep pyrefly's spec-incompliant behavior. I don't want to create a situation where pyrefly is blocking progress because it's doing something that the other checkers can't or won't copy, but the behavior is too entrenched for us to change.
jorenham commentedon Sep 14, 2026
With my typing council hat on, I agree this is correct. But it would also be correct to infer
tuple[int, Unknown]instead ofUnknownhere, which would make this more consistent with thetuple[int, Any]case and would make this less type-unsafe. Ideally,AnyOfwould be better, or alternatively treating the ambiguous return type as a "free" type variable, i.e.def f[T: int | list[int]](a: tuple[int, ...]) -> tuple[str, T]: ..., which would be similar toAnyOf+ constraint solving.With my NumPy/scipy-stubs maintainer hat on, I feel obligated to push back on this. Because the consequence of this new Pyrefly behavior is that many calls to numpy and scipy-stubs functions will now result in
Unknown: the incarnation of type-unsafety. That's because it's very common for users to annotate their array types asnp.typing.NDArray[T], a gradual alias ofnp.ndarray[tuple[Any, ...], np.dtype[T]]. And if you combine that with the fact that many, if not most, of NumPy's and scipy-stubs' functions are overloaded for shape-types, I foresee that very many calls to these functions will now beUnknownwhen using Pyrefly. As a maintainer there is no way for me to control this behavior either, like I can for mypy and basedpyright using the "Never-trick" (prepending an overload withtuple[Never, Never, Never]as shape-type, because mypy/pyright will then eagerly select this overload in case oftuple[Any, ...]).My worry is that users won't switch to Pyrefly, or even switch to another type-checker, because of this. And if they do, I wouldn't blame them tbh...
rchen152 commentedon Sep 15, 2026
I realize this is a painful change, and I'm fully aware that users have other options for type checking. I didn't think it was reasonable to put it off longer once it became clear that what pyrefly was doing wasn't going to make it into the spec, and
Anyis the best honest alternative we have right now, as bad as it is.Another option that just occurred to me is that we may be able to use
Type::Overloadedfrom this proposal. The semantics are roughly what we want (if rather circular as currently described - "calls follow overload semantics" isn't very helpful!) My progress there has been slow so far because it keeps getting preempted by more urgent work, but I should have some time this week to hack on it.str.joinresult incorrectly inferred asUnknownforlist[Any]#4845 as a duplicate of this issue[-]1.3.0 regression: gradual overload call inferred as `Unknown` when overload returns `T | Any`[/-][+]1.3.0 regression: gradual overload call inferred as `Unknown`[/+]jorenham commentedon Sep 15, 2026
The typing spec now also allows this to be inferred as
tuple[str, Any], which you could call the "gradual meet" oftuple[str, int]andtuple[str, list[int]], i.e. the most specific gradual type assignable to both. So for example for overloads returningndarray[tuple[int]]andndarray[tuple[int, int]]that'd bendarray[tuple[Any, ...]]. AfterAnyOfthis seems like the next best solution.TBH I haven't read your
Type::Overloadedproposal, so maybe I'm reinventing the wheel here, but do you think something like that could work? Because I think that this would solve this issue for at least NumPy and scipy-stubs.rchen152 commentedon Sep 24, 2026
Type::Overloadedis closer to an orderedAnyOfthan a gradual meet. Anyway, I did manage to getType::Overloadedin last week, so let me check if it can be dropped in here to give us a better return type thanAnyfor ambiguous evaluation.rchen152 commentedon Oct 13, 2026
Unfortunately, this isn't going to make it into 1.4 :( Making overload evaluation use
Overloadedcauses it to show up in a lot more places than before, and all those places need newOverloadedhandling.jorenham commentedon Oct 13, 2026
Ah ok too bad, but makes sense. Anyway, thanks a lot for digging into this :)