Support recursive types

The following in particular would be useful:

Callback = Callable[[str], 'Callback']
Foo = Union[str, List['Foo']]

If (or when) mypy will have structural subtyping, recursive types would also be useful there.

My current plan is to postpone recursive types until simple structural subtyping is in and reconsider them later. After thinking about them more they are going to increase complexity a lot of I haven't seen much evidence for them being needed that often.

If I'm understanding this issue right, I'm running into it for classes that want a fluent interface. So for example, if I want callers to do something like this:

myfoo.add(bar).add(baz).finish()

Then the definition of the Foo class and the add method need to look something like this:

class Foo:
    def add(self, x) -> Foo:  # Python chokes on this line!
        # do stuff
        return self

Another place where Python commonly does return self is in the __enter__ method for context managers. Is mypy able to typecheck those right now?

@oconnor663

@oconnor663 Try:

class Foo:
    def add(self, x) -> 'Foo':
        # do stuff
        return self

Ah, thank you.

@kirbyfan64

@kirbyfan64, do you know if there are standard functions anywhere that understand this convention? Like, if I wanted to introspect a couple functions and compare the types of their arguments, should I handle the Foo == "Foo" case explicitly? That seems doable, but a string-aware version of say isinstance seems harder.

@oconnor663

@oconnor663 I don't think there's anything like that. If you're introspecting the functions via a decorator, you could try accessing the caller's globals and locals.

@oconnor663

You're aware of typing.get_type_hints(obj)right? It is similar to obj.annotations` but expands forward references.
https://docs.python.org/3/library/typing.html?highlight=typing#typing.get_type_hints

There used to be an instance() implementation in typing.py but Mark Shannon
made me take it out. It's being deleted in this rev:
python/typing@ac7494f

On Fri, Oct 16, 2015 at 9:37 AM, Ryan Gonzalez [email protected]
wrote:

@oconnor663 https://github.com/oconnor663 I don't think there's
anything like that. If you're introspecting the functions via a decorator,
you could try accessing the caller's globals and locals.

—
Reply to this email directly or view it on GitHub
#731 (comment).

--Guido van Rossum (python.org/~guido)

@gvanrossum

@gvanrossum that's exactly what I was looking for, thanks. Sorry for the n00b questions today, but awesome that all this is supported.

Mypy should detect missing string literal escapes (see #948).

Going back to the original point on the issue, I found a case in the stdlib where this would be needed; the type for isinstance() is currently:

def isinstance(o: object, t: Union[type, Tuple[type, ...]]) -> bool: ...

but it should actually be:

ClassInfo = Union[type, Tuple['ClassInfo', ...]]
def isinstance(o: object, t: ClassInfo) -> bool: ...

Because according to https://docs.python.org/3/library/functions.html#isinstance the tuples can be nested. I found an actual example of this while typechecking django.http.response.HttpResponse.content

I think this can now be closed by #13297 (and follow-up PRs). If there are remaining problems, they can be opened as separate issues.

Note this is only enabled in master with --enable-recursive-aliases (enables recursive type aliases, recursive TypedDicts, and recursive NamedTuples; as mentioned above proper classes and protocols can already be recursive). Note this is still considered an experimental feature, in particular type inference may sometimes fail, if this happens you can try switching to covariant collections, and/or simply give an explicit annotation.

@ilevkivskyi

@ilevkivskyi is there a way to set this config option in a pyproject.toml file? I checked the documentation, but couldn't find anything about the new cli flag (perhaps because it's still experimental, and will eventually be enabled by default?)

Yes, it is indeed a temporary flag (for few releases), and in 0.990 it will be on by default, you can try

$ cat pyproject.toml
[tool.mypy]
enable_recursive_aliases = true

This worked for me on Linux.

fix: use a correct recursive definition for DumperKey

It appears that v0.990/v0.991 may not have addressed the entirety of the challenge that is recursive/cyclic types: #14219

For those reading in posterity:

mypy==0.981 added --enable-recursive-alias: Enable recursive type aliases behind a flag #13297
mypy==0.990 started a deprecation cycle for --enable-recursive-alias: Temporarily put back --enable-recursive-aliases (as depreceated) #13852
mypy==1.7.0 completed the deprecation cycle, removing --enable-recursive-alias: Delete recursive aliases flags #16346

So with mypy>=1.7, recursive type support is built into mypy ✅

It doesn't work in this case:

from collections.abc import Sequence
from typing import TypeAlias

FloatVector: TypeAlias = Sequence[float]
FloatTensor: TypeAlias = FloatVector | Sequence["FloatTensor"]

rejected_float_vector: FloatVector = "ok"       # OK: rejected

accepted_float_tensor: FloatTensor = [[[3.14]]] # OK: accepted
rejected_float_vector1: FloatTensor = object()  # OK: rejected
rejected_float_vector2: FloatTensor = "fail"    # FAIL: accepted

https://mypy-play.net/?mypy=latest&python=3.12&flags=strict&gist=da0434e6062956cfd667a9e8c8bd3ff9

@jorenham

@jorenham Thanks for the report! Can you please open a new issue?

@jorenham

@jorenham Beware that this TypeAlias actually allows mixed lists, like [[1.2, 3.4], [[1.0, 0.0], [0.0, 1.0]]]. You might want to consider a non-recursive alias like S[float] | S[S[float]] | S[S[S[float]]] | S[S[S[S[float]]]] for a sufficient number of levels instead. The infinite union you probably want is currently not representable in the type system, I think.

EDIT: we would need inductive types. If typing.Map is introduced
(python/typing#1216, python/typing#1383, pre-pep discussion), then we should be able to express this as

Tensor = Sequence[float] | Map[Sequence, Tensor]

You might want to consider a non-recursive alias like S[float] | S[S[float]] | S[S[S[float]]] | S[S[S[S[float]]]] for a sufficient number of levels instead.

In numpy the current maximum number of dimensions is 64, so that would indeed be possible. But in order to keep error messages from getting to long, and the type-checking from becoming to slow, I decided to go for this concise yet "broader" approach instead.

@jorenham

@jorenham Thanks for the report! Can you please open a new issue?

🫡 #18184

@jorenham

@jorenham Beware that this TypeAlias actually allows mixed lists, like [[1.2, 3.4], [[1.0, 0.0], [0.0, 1.0]]].

This made me realize that some of the numpy array-like type aliases in optype (that are heavily used in scipy-stubs) could be improved. Specifically, this affected the optype.numpy.To{Bool,JustInt,Int,Float,Complex,Array}{2,3}D type aliases (docs), which also accepted the mixed lists that you mentioned, @randolf-scholz. So now this has been fixed in optype 0.7.2 👌🏻.

Navigation Menu

Repository navigation

Support recursive types #731

80 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Support recursive types #731

Description

Activity

JukkaL commented on Aug 4, 2015

JukkaL commented on Oct 13, 2015

oconnor663 commented on Oct 16, 2015

refi64 commented on Oct 16, 2015

oconnor663 commented on Oct 16, 2015

oconnor663 commented on Oct 16, 2015

refi64 commented on Oct 16, 2015

gvanrossum commented on Oct 16, 2015

oconnor663 commented on Oct 16, 2015

JukkaL commented on Oct 17, 2015

dmoisset commented on Aug 18, 2016

80 remaining items

ilevkivskyi commented on Aug 12, 2022

cj81499 commented on Sep 27, 2022

ilevkivskyi commented on Sep 28, 2022

bartfeenstra commented on Nov 29, 2022

jamesbraza commented on Jul 30, 2024

jorenham commented on Nov 24, 2024

brianschubert commented on Nov 24, 2024

randolf-scholz commented on Nov 24, 2024

jorenham commented on Nov 24, 2024

jorenham commented on Nov 24, 2024

jorenham commented on Nov 25, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions