Skip to content

[Feature Request] TypeVar variance definition alias #813

Closed
Closed
[Feature Request] TypeVar variance definition alias#813
Labels
topic: featureDiscussions about new features for Python's type annotations

Description

@jorenham
jorenham
opened on May 20, 2021

Defining the two variances of a TypeVar is rather verbose:

T_co = TypeVar("T_co", covariant=True)
T_contra = TypeVar("T_contra", contravariant=True)

So I propose an alternative syntax, similar to that of Scala's variances:

T_co = +TypeVar("T_co")
T_contra = -TypeVar("T_contra")

Co- and contra-variance describe increasing and decreasing specificities of the typevars, respectively. So using + and - to symbolize these "directions" makes intuitive sense.

Activity

  1. gvanrossum commented on May 20, 2021

    @gvanrossum
    gvanrossum
    on May 20, 2021
    Member

    This was proposed before. At the time I’ve didn’t care much for it, but now I think it’s actually a good idea.

    You probably shouldn’t use this on the TypeVar call though but rather in the use , e.g.

    def f(a: +T): ...

    A tricky thing will be introducing it, since TypeVar is imported from typing in the stdlib.

  2. jorenham commented on May 20, 2021

    @jorenham
    jorenham
    on May 20, 2021
    CollaboratorAuthor

    Yes, I like that better as well @gvanrossum .

    From what I can see in PEP 484, variance only applies to generic types, so I suppose it'll be used as e.g.

    class Spam(Generic[+T]): ...

    If I understand correctly, introducing this would require it to go through the PEP trajectory, right? I wouldn't mind writing up a draft PEP if you think that it has a chance of being accepted :)

  3. jorenham commented on May 20, 2021

    @jorenham
    jorenham
    on May 20, 2021
    CollaboratorAuthor

    I just realized that there is an incompatibility, which could make the implementation tricky, but not impossible:

    In the current api, the variance is a property of the TypeVar itself and has a unique name, e.g.

    T_co = TypeVar("T_co", covariant=True)

class Spam(Generic[T_co]):
    eggs: T_co

    But when using the corresponding + syntax, T and +T are both named "T". Furthermore, the (in my opinion desirable) way that generic variances work in Scala, is that the + or - prefixes are used only in the generic type definition, e.g.

    T = TypeVar("T")

class Spam(Generic[+T]):
    eggs: T

    From what I understand from the current typing.Generic and typing.TypeVar implementation, the variance of a generic class is implicitly determined from its TypeVars.
    So a "naïve" implementation where +T evaluates to e.g. TypeVar("T", covariant=True), will cause the typechecker to see T and +T as being distinctly different, and will probably result in the misinterpretation of a duplicate declaration.

    If this is in fact the case, it will require changing the implementation so that the generic class explicitly stores the variance of each of its TypeVars. Specifying the (co/contra)variance of a TypeVar with +/- should be only be done within the generic type arguments, e.g. class Spam(Generic[+T]).

    I see that the implementation will be tricky. But this "incompatibility" can also be seen as a strong argument in favor of this feature:

    In PEP 484 it says:

    Note: Covariance or contravariance is not a property of a type variable, but a property of a generic class defined using this variable. Variance is only applicable to generic types; generic functions do not have this property. The latter should be defined using only type variables without covariant or contravariant keyword arguments.

    Hence it's counterintuitive to have to mark the TypeVar itself as being co(ntra)variant.

  4. JelleZijlstra commented on May 20, 2021

    @JelleZijlstra
    JelleZijlstra
    on May 20, 2021
    Member

    I don't think that's a major concern, although it would require a bit of new coding in type checkers.

    Most type checkers are static; they never evaluate the code. So they don't care what +T does at runtime: they just need their parsers to understand this syntax. For those that do evaluate the code at runtime, you're right that we need some way to trace +T back to T. I'd suggest implementing it by making +T return something like TypeVarWrapper(T, covariant=True).

  5. jorenham commented on Jul 23, 2021

    @jorenham
    jorenham
    on Jul 23, 2021
    CollaboratorAuthor

    I'm new to contributing here, but I'd like to give this one a shot. Is my assumption correct that I should first draft a PEP, find a sponsor, and if the PEP gets accepted, implement the feature in typing_extensions and PR in this repo?

  6. JelleZijlstra commented on Jul 23, 2021

    @JelleZijlstra
    JelleZijlstra
    on Jul 23, 2021
    Member

    Pretty much! But you'd also want to implement it in cpython itself (more important than this repo, because this one is now just for backports) and in at least one major typechecker (e.g., mypy, pyright, pyre).

  7. uriyyo commented on Jul 30, 2021

    @uriyyo
    uriyyo
    on Jul 30, 2021
    Member

    @jorenham @JelleZijlstra Do you need help with this feature? If no one works on it I can help with it)

  8. jorenham commented on Jul 30, 2021

    @jorenham
    jorenham
    on Jul 30, 2021
    CollaboratorAuthor

    @uriyyo I'm currently working on drafting the PEP. From what I've seen, the reference implementations look feasible enough to write myself. But if anything changes, I'll be sure to let you know :)

  9. jorenham commented on Aug 2, 2021

    @jorenham
    jorenham
    on Aug 2, 2021
    CollaboratorAuthor

    python/peps#2045

  10. Gobot1234 commented on Aug 2, 2021

    @Gobot1234
    Gobot1234
    on Aug 2, 2021
    Contributor

    I'm curious as to why this syntax isn't valid in a function signature that takes a generic i.e.

    T = TypeVar("T")
def foo(bar: Sequence[+T]) -> Sequence[+T]: ...

    Surely this is still a common use case for variant type vars?

  11. JelleZijlstra commented on Aug 2, 2021

    @JelleZijlstra
    JelleZijlstra
    on Aug 2, 2021
    Member

    @Gobot1234 what would variance mean there?

  12. jorenham commented on Aug 2, 2021

    @jorenham
    jorenham
    on Aug 2, 2021
    CollaboratorAuthor

    @Gobot1234 Sequence is already covariant. Do you mean something like this instead?

    T = TypeVar("T", bound=Sequence)

def foo(bar: T) -> T: ...

  13. Gobot1234 commented on Aug 2, 2021

    @Gobot1234
    Gobot1234
    on Aug 2, 2021
    Contributor

    Ignore me, my comment doesn't make sense.

  14. jorenham commented on Aug 8, 2021

    @jorenham
    jorenham
    on Aug 8, 2021
    CollaboratorAuthor

    @JelleZijlstra Thank you for sponsoring the PEP! I'll be on holiday the coming week, but after that I'll work on expanding the specification so that e.g. class Spam(Sequence[+T], Container[-T]) will not be allowed, like we discussed in the PR. I'm not sure if this issue is the best place for discussing the PEP, so you can also reach me via email (you can find it in the draft PEP).

  15. JelleZijlstra commented on Aug 9, 2021

    @JelleZijlstra
    JelleZijlstra
    on Aug 9, 2021
    Member

    Sounds good! I'm moving, so I'll also be pretty busy for the next two weeks or so. I'd prefer to discuss the PEP in public. We could also use the issues on your fork of the peps repo (https://github.com/jorenham/peps) to discuss some details.

    You'll still need approval from the Steering Council to have me as a sponsor for the PEP. As far as I'm aware this will be the first time the procedure for a non-core dev sponsor will be exercised, so we may have to figure it out as we go along, but I think you should open an issue at https://github.com/python/steering-council/issues asking for permission.

  16. superbobry commented on Sep 7, 2021

    @superbobry
    superbobry
    on Sep 7, 2021
    Contributor

    @jorenham I've just started a thread about exactly this on typing-sig. Would would be the best way to provide feedback on the PEP draft?

  17. jorenham commented on Sep 7, 2021

    @jorenham
    jorenham
    on Sep 7, 2021
    CollaboratorAuthor

    It's good to see that you see the issue with the current implementation as well @superbobry :).
    I created an issue on my fork exactly for this a couple of days ago at jorenham/peps#1

  18. fylux commented on Sep 8, 2021

    @fylux
    fylux
    on Sep 8, 2021

    Would this syntax be only used in class definitions or it could appear somewhere else?

    It comes to my mind that the proposed syntax could be partially incompatible with adding in the future type arithmetic with nice syntax.
    In particular, if we would introduce the operators +,- at the type level, even if they are restricted to variables representing literal integers and int, it would be unclear what some expressions mean (e.g. A ++ B). And even if we limit variance syntax to class definitions it could be confusing that the + operator means different things.

  19. sobolevn commented on Sep 8, 2021

    @sobolevn
    sobolevn
    on Sep 8, 2021
    Member

    In particular, if we would introduce the operators +,- at the type level

    Later this can also be reused in type-level arithmetics. Like Literal[1] + Literal[2] would resolve into Literal[3]. Or -Literal[2] into Literal[-2]

    But, I won't go into this topic yet since it is not directly related.

  20. superbobry commented on Sep 8, 2021

    @superbobry
    superbobry
    on Sep 8, 2021
    Contributor

    @fylux The proposed syntax uses unary +/- operators, so it shouldn't interfere much with type level arithmetic, which I assume would mostly use the binary ones. A ++ B looks like a syntax error :)

  21. srittau
    added
    topic: featureDiscussions about new features for Python's type annotations
    on Nov 4, 2021

  22. JelleZijlstra commented on May 22, 2023

    @JelleZijlstra
    JelleZijlstra
    on May 22, 2023
    Member

    We instead ended up going with PEP 695, which eliminates the need for explicit variance markers by having type checkers infer variance from use.

  23. JelleZijlstra
    closed this as completedon May 22, 2023

  24. randolf-scholz commented on Oct 4, 2023

    @randolf-scholz
    randolf-scholz
    on Oct 4, 2023
    Contributor

    This would still be a useful feature, even with PEP695 in place. I outlined 2 main reasons here: https://discuss.python.org/t/proposal-optional-explicit-covariance-contravariance-for-pep-695/35204

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic: featureDiscussions about new features for Python's type annotations

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions