[1.16 regression] descriptor not working in protocol

@ilevkivskyi

After #18831 this example started generating an error, which looks like a regression:

import typing

A = typing.TypeVar('A')

class Attribute(typing.Generic[A]):
    @typing.overload
    def __get__(self, instance: None, owner: typing.Any) -> Attribute[A]: ...
    @typing.overload
    def __get__(self, instance: object, owner: typing.Any) -> A: ...
    def __get__(self, instance, owner): ...

class PathArg(typing.Protocol):
    path: Attribute[str]

def f(arg: PathArg) -> None:
    # No error expected
    s: str = arg.path  # error: Expression has type "Attribute[str]", variable has type "str'

If PathArg is a regular class, there is no error.

cc @ilevkivskyi as the author of the PR

This was an intentional change, to make descriptor access more consistent, see this PR comment for more motivation #18831 (comment). IMO the code in the example is a bug, such pattern should be reserved for when the actual attribute type is a descriptor, otherwise one should just use path: str in the protocol.

Ok, let me play around with this a little. The impact to the Dropbox internal codebase (where I found this) is very minor, so if using path: str works, we can close this issue.

I couldn't find a way to make a concrete class that uses the descriptor to be compatible with the protocol:

import typing

A = typing.TypeVar('A')

class Attribute(typing.Generic[A]):
    @typing.overload
    def __get__(self, instance: None, owner: typing.Any) -> Attribute[A]: ...
    @typing.overload
    def __get__(self, instance: object, owner: typing.Any) -> A: ...
    def __get__(self, instance, owner): ...

class PathArg(typing.Protocol):
    @property  # Attempt to use @property since the descriptor is read only
    def path(self) -> str: ...

class C:
    path = Attribute[str]()

def f(arg: PathArg) -> None:
    s: str = arg.path

def g(c: C) -> None:
    reveal_type(c.path)  # "str", as expected
    f(c)  # error: C is not compatible with PathArg <<-- this should be fine?

Is there a way to modify the old example so that it works using the new semantics?

Another use case where the new behavior may cause issues is when using type[PathArg] -- here path: Attribute[str] in the protocol makes a difference:

import typing

A = typing.TypeVar('A')

class Attribute(typing.Generic[A]):
    @typing.overload
    def __get__(self, instance: None, owner: typing.Any) -> Attribute[A]: ...
    @typing.overload
    def __get__(self, instance: object, owner: typing.Any) -> A: ...
    def __get__(self, instance, owner): ...

class PathArg(typing.Protocol):
    path: Attribute[str]   # no error with Attribute[str], but error when using str

class C:
    path = Attribute[str]()

def f(arg: type[PathArg]) -> None:
    reveal_type(arg.path)  # Attribute[str]

def g() -> None:
    f(C)  # error if using "path: str" in PathArg

OK, I understand what happened (i.e. why someone defined such weird protocol). The problem is that protocols never worked with descriptors, see #5481, so someone found this workaround, and now this workaround doesn't work anymore.

I think the best way forward is to just wait for the proper descriptor support added in #18943. I you are really worried about this, you can revert it in the release branch. But I wouldn't bother, because IMO this is really niche workaround.

This only impacts two files in the Dropbox codebase, so it's not a big deal. As you said, the code is kind of questionable anyway, so this shouldn't block the release.

This is a big issue for us. We have a system where we use dataclasses with a special descriptor field to define schemas. Since there's no common base class, we use a protocol to type "generic schema".

@eltoder

@eltoder I have two questions:

Did you try to play with current master?
Can you share some code for me to understand what exactly you are doing?

@ilevkivskyi

@ilevkivskyi

Same error on master according to mypy-play: https://mypy-play.net/?mypy=master&python=3.11&gist=e1f1bd3254690db213102de71f894854
Here is a more detailed example. The real code is much longer. Let me know if you need more:

from __future__ import annotations

# Framework code
from typing import Any, Protocol, overload

class DescInfo: ...

class Desc:
    @overload
    def __get__(self, instance: None, owner: Any) -> Desc: ...
    @overload
    def __get__(self, instance: object, owner: Any) -> Callable[[], DescInfo]: ...
    def __get__(self, instance, owner):
        if instance is None: return self
        return lambda: DescInfo()  # construct from self and instance

class HasDesc(Protocol):
    desc: Desc

def write_schema(schema: HasDesc) -> None:
    print(schema.desc())  # do something with DescInfo

# User code
import dataclasses

@dataclasses.dataclass
class MySchema:
    desc = Desc()
    name: str
    value: int

write_schema(MySchema("foo", 123))

@eltoder

@eltoder Then you should use this instead

class HasDesc(Protocol):
    desc: Callable[[], DescInfo]

to correctly specify the interface. This works on current master. This means however that you may need to wait for the next release for this to be available. Anyway, I don't think we need to keep this open, the old behavior was wrong.

@ilevkivskyi

@ilevkivskyi That's simply wrong. It does not work when using the type. For example:

ST = TypeVar("ST", bound=HasDesc)

def read_schema(cls: type[ST]) -> ST:
    print(cls.desc)  # use desc to read

TBH I don't see how the current behavior is defensible. It makes Protocols inconsistent with regular classes and it contradicts the runtime. Please fix.

@eltoder

@eltoder It is not for you to decide. Also please watch your tone.

Navigation Menu

Repository navigation

[1.16 regression] descriptor not working in protocol #19054

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[1.16 regression] descriptor not working in protocol #19054

Description

Activity

ilevkivskyi commented on May 8, 2025

JukkaL commented on May 8, 2025

JukkaL commented on May 9, 2025

JukkaL commented on May 9, 2025

ilevkivskyi commented on May 9, 2025

JukkaL commented on May 9, 2025

eltoder commented on Jun 9, 2025

ilevkivskyi commented on Jun 9, 2025

eltoder commented on Jun 10, 2025

ilevkivskyi commented on Jun 10, 2025

eltoder commented on Jun 10, 2025

ilevkivskyi commented on Jun 10, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions