Repository navigation
Conversation
A runtime-checkable protocol declaring `__slots__ = ()` was classified as a data protocol, so `issubclass(bytes, BufferProtocol)` and `issubclass(int, SupportsInt)` reported `invalid-argument` even though CPython accepts both. The same happened for typeshed protocols such as `typing.SupportsInt` and `collections.abc.Buffer`, reported in pyrefly-org#5614. `ProtocolMetadata.members` answered two questions at once: what structural subtyping requires, and what the runtime protocol check inspects. `__slots__` is an ordinary class-body field, so it entered `members`; the protocol subsetting code already skipped it, which is why the consumers disagreed. Add `runtime_members`, built from the class-body fields minus the names CPython's `typing._get_protocol_attrs` excludes, plus the slot names that survive the same filter. `has_data_slots` records whether the declaration proves a data attribute, because a slot name can resolve to an inherited callable and the member type alone cannot decide it. `is_data_protocol` and the unsafe-overlap presence check now read `runtime_members`. The overlap check compares types only for names in `members`: a slot-only name is part of the runtime contract but has no protocol-side type to compare. `_abc_*` is skipped by prefix, and `__class_getitem__` (3.9+), `__match_args__` (3.13+), and the `__annotate__` family (3.14+) are gated to match the CPython versions that exclude them. Inheritance merges both sets and the flag from protocol bases. An unresolvable `__slots__` value is treated as non-data. The runtime may reject the `issubclass()` call, but the checker cannot prove it; this is a deliberate false negative. Tests cover empty, non-empty, excluded, and inherited slots; string and annotation-only declarations; typeshed protocols; the version boundaries; and the unsafe-overlap presence rule.
A subclass method that shadows an inherited `__slots__` name was reported as a data member, so `issubclass()` rejected the protocol even though CPython accepts the call. The `has_data_slots` flag was inherited by every protocol subclass, so it could not tell which class actually binds the name. CPython decides whether a member is a method with `callable(getattr(cls, name))`, so a data member is a name whose runtime value is non-callable. `is_data_protocol` now walks the MRO per runtime member: a class-body binding lets the declared read type decide, and a slot declaration that no nearer class body initializes is a data member. A body that only annotates the name creates no class attribute, so it does not shadow a slot. Members with no runtime binding are classified by their declared type, which keeps the check aligned with the type an implementation must provide; CPython would reject those calls. The flag is no longer needed and is removed. The test for a child method shadowing an inherited slot now expects no error. New tests cover a slot and a method on separate bases in both orders, an annotation-only member that still exposes the inherited slot, and the declared-type rule for a callable annotation-only member and a property whose getter returns a callable.
CPython decides whether a protocol member is a method with `callable(getattr(cls, name))`. A `property` object is not callable, so a runtime-checkable protocol whose member is a property always rejects `issubclass()`. `is_data_protocol` classified a class-body binding by its declared read type, which for a property is the getter's return type. That accepted the call and, for a property shadowing an inherited slot, removed an error the previous rule reported. A property binding is now a data member. Other bindings, and members with no runtime binding, still follow the declared read type. Tests cover a property shadowing an inherited slot, a protocol whose only member is a property, and decorated methods shadowing an inherited slot. Two shapes where the check trusts the declaration over the runtime value keep their divergence from CPython and now carry `bug` markers.
|
Hi @amas127! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
✅ This PR meets all requirements from our pull request checklist. Optional recommendations:
Note: this comment updates automatically after changes to the PR. Updates may take up to half an hour. |
|
This pull request has been imported. If you are a Meta employee, you can view this in D131788114. (Because this pull request was imported automatically, there will not be any future comments.) |
stroxler
left a comment
There was a problem hiding this comment.
Thanks! I think this overall approach looks good.
A round of AI-assisted review found two issues: one regression that I think is worth digging into, and an easy fix where there's another dunder name we might as well include in our exclusions.
A. unsafe-overlap regression
Head location: special_calls.rs:700–716.
from typing import Protocol, runtime_checkable
@runtime_checkable
class P(Protocol):
_abc_marker: int
def f(self) -> int: ...
class C:
_abc_marker: str = "wrong"
def f(self) -> int:
return 0
isinstance(C(), P)
p: P = C()In this example:
- The protocol requires _abc_marker: int.
- The implementation provides _abc_marker: str.
- Python ignores _abc_marker during isinstance(), so the check returns True.
- Pyrefly still rejects assigning that implementation to the protocol, because str is incompatible with int.
That combination is what the unsafe-overlap warning catches: the runtime check succeeds even though the object doesn't satisfy the protocol's types.
The PR correctly removes _abc_marker from the runtime check. But it uses that same shortened list for checking types, so it loses the warning.
The suggested fix: keep the shortened list for checking which attributes Python looks for; use Pyrefly's ordinary protocol-assignment logic to check whether their types satisfy the protocol. Don't make _abc_marker a runtime requirement again.
incomplete runtime exclusions (minor but easy to fix)
Head location: class_metadata.rs:99–135.
from typing import Protocol, runtime_checkable
@runtime_checkable
class P(Protocol):
__type_params__: tuple[object, ...]
def f(self) -> int: ...
class C:
def f(self) -> int:
return 0
issubclass(C, P)Targeting Python 3.14, both base and head emit the non-method-members diagnostic; CPython accepts the call. Thus this is an uncompleted case of the stated excluded-attribute problem, not a newly introduced regression. The same exact-head probe confirms the combined __protocol_attrs__ / __non_callable_proto_members__ annotation case is rejected.
The tested CPython 3.14 exclusion set also contains __static_attributes__ and __firstlineno__, absent from the new table. These entries are corroborated by pinned CPython v3.14.0 source. Use version-aware tests for the omitted names; do not infer a generated-name regression where Pyrefly never gathers the injected field.
|
According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
main moved the explicit_slots call after metaclass computation, while this branch needs the slots before final protocol metadata. Compute both after the metaclass is known and before parsed_results is consumed.
The unsafe-overlap check only iterated runtime members, so it skipped members that CPython excludes from its runtime check. Keep the runtime presence check over runtime_members, then compare the declared types of the protocol members the implementation provides. A missing runtime-excluded member stays optional, while an incompatible annotation or assignment is reported.
The unsafe-overlap check used runtime_members for both the presence gate and the type comparison, so a runtime-excluded member was skipped even when the implementation omitted it. Keep the presence gate on runtime_members, then check every static protocol member. A missing runtime-excluded member now reports an unsafe overlap, while a missing ordinary member still fails the presence gate first. Remove exclusions with no observed manual declaration: `_MutableMapping__marker` and the internal PEP 749 companions `__annotate_func__` and `__annotations_cache__`. Add tests for missing members on both sides of the gate and for the `__match_args__` 3.12/3.13 boundary.
Rename tests so the API in the name matches the call in the body: several `test_issubclass_*` tests actually call `isinstance`. Correct comments that gave the wrong reason for unsafe-overlap warnings or described runtime-excluded names as static members. Fix typos in the version-gated exclusion notes. Make the private-name test observable by adding a type mismatch: pyrefly now reports the false positive on the unmangled implementation and misses the warning on the mangled one, so the bug marker covers both directions.
Only `pyrefly/lib/alt/special_calls.rs` conflicted. Main's display and `TypeEnv` refactor changed `SubsetError::to_error_msg` to take the solver, while this branch rewrote the runtime protocol overlap check. Keep the new overlap logic and pass `self` to the error-message helper.
|
@stroxler Thanks for your review and helpful comments! I believe issue A is real, while issue B is a deliberate FP misalignment with CPython runtime behavior after careful consideration. A. unsafe-overlap regression: FIXEDThe unsafe-overlap check used
When the CPython-excluded names were removed from the runtime set, they also disappeared The two questions now use separate sets:
A testcase!(
test_isinstance_raise_unsafe_overlap_by_wrong_typing_with_abc_marker,
r#"
from typing import Protocol, runtime_checkable
@runtime_checkable
class P(Protocol):
_abc_marker: int
def m(self) -> None: ...
class AnnotatedWrong:
_abc_marker: str
def m(self) -> None: ...
class AssignedWrong:
_abc_marker = "wrong"
def m(self) -> None: ...
class Missing:
def m(self) -> None: ...
isinstance(AnnotatedWrong(), P) # E: Runtime checkable protocol `P` has an unsafe overlap with type `AnnotatedWrong`
isinstance(AssignedWrong(), P) # E: Runtime checkable protocol `P` has an unsafe overlap with type `AssignedWrong`
isinstance(Missing(), P) # E: Runtime checkable protocol `P` has an unsafe overlap with type `Missing`
"#,
);B. incomplete runtime exclusions: DEFERREDThis is deliberate. The exclusion table keeps a version gate only for names with a realistic manual-declaration surface: Pyrefly used to treat every explicitly declared name as a protocol member. It now excludes the names real code writes while reports on the compiler- and runtime-injected ones, keeping its strict styles for implementation details. Omitted attribute names are most likely to be compiler- or runtime-injected, and there is very few, if any, practical scenarios for them to be declared in a
Additional Changes
|
Summary
Fixes #5614
issubclass()rejected@runtime_checkableprotocols that declare__slots__in their body, reportinginvalid-argumenteven though CPython accepts the call. Protocols from typeshed are affected, so the false positive reaches ordinary library code:One list of names was serving two different questions:
The structural check skips
__slots__because it is class machinery, not an instance member. The runtime check asked the same question of the same list with no such exception, so__slots__, a non-callable class attribute, was enough to make the protocol a data protocol, and correct code was rejected.The runtime check now reads its own member list, built by the same rules CPython uses to decide what a protocol declares. It also decides whether a member is a method the way CPython does, from the value the name resolves to on the class rather than from the declared type. That fixes the reverse case too: a subclass method, or a decorated method,
that shadows an inherited slot is now accepted.
A few corners still diverge from CPython: an unresolvable
__slots__is treated as non-data, an annotation-only callable member is accepted even though CPython finds no class attribute, and private slot names are not mangled for the overlap check. Each one has abugtest marking the divergence so a follow-up can close it.Test Plan
cargo test -p pyrefly --lib test::protocol::: 126 passed, including 27 new tests for slot declarations in each form, the two typeshed protocols, version boundaries, unsafe overlap, and shadowing through the MRO in both directions.cargo test -p pyrefly --lib protocol: 240 passed, which adds the solver-side protocol tests. The neighbouring suites pass too:test::slots(49),test::attributes(310),test::class_subtyping(52).operators.rswarning.mypy_primer
Not run yet.