Parameter substitution in the union type does not work with typing.Union #88819

Activity

1. serhiy-storchaka commented on Jul 16, 2021

   

   serhiy-storchaka

   on Jul 16, 2021

   MemberAuthor

   More actions

   >>> import typing
   >>> T = typing.TypeVar('T')
   >>> (int | T)[typing.Union[str, list]]
   NotImplemented

   See also bpo-44633. But in this case the expected result is int | str | list or typing.Union[init, str, list].
2. serhiy-storchaka

   added

   3.10 (EOL)end of lifeend of life

   3.11only security fixesonly security fixes

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jul 16, 2021
3. Fidget-Spinner commented on Jul 19, 2021

   

   Fidget-Spinner

   on Jul 19, 2021

   Member

   More actions

   @serhiy, this doesn't just affect typing.Union, it seems that the rest of the typing types don't substitute:

   >>> (int | T)[typing.List[str]]
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   TypeError: Each union arg must be a type, got typing.List[str]

   We should probably loosen the check during the union_getitem substitution -- no need to raise the TypeError or check for is_union, just blindly replace the TypeVar. We already do this for types.GenericAlias:

   >>> list[T][1]
   list[1]

   Or if you want to continue checking, maybe checking for PyCallable_Check(obj) in substitution is enough - typing internally accepts callable(o) too:

   https://github.com/python/cpython/blob/3.10/Lib/typing.py#L146
4. serhiy-storchaka commented on Jul 19, 2021

   

   serhiy-storchaka

   on Jul 19, 2021

   MemberAuthor

   More actions

   There is also problem with other typing types:

   >>> (int | T)[typing.List]
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   TypeError: Each union argument must be a type, got typing.List
   >>> (int | T)[typing.List[int]]
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   TypeError: Each union argument must be a type, got typing.List[int]
   >>> (int | T)[typing.Hashable]
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   TypeError: Each union argument must be a type, got typing.Hashable
   >>> (int | T)[typing.Callable[[int], str]]
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   TypeError: Each union argument must be a type, got typing.Callable[[int], str]
   >>> (int | T)[typing.ParamSpec('P')]
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   TypeError: Each union argument must be a type, got ~P

   Despite the fact that they support the | operator.

   We can add one by one special support of different types supporting the | operator (ParamSpec, _GenericAlias, _CallableGenericAlias, _UnionGenericAlias, _LiteralGenericAlias, _ConcatenateGenericAlias, _AnnotatedAlias, _SpecialGenericAlias, _CallableType, _TupleType), but it is cumbersome and errorprone. We will need to synchronize code of unionobject.c with typing every time we add new kind of types.

   PR 27247 uses more general approach. It calls the | operator for arguments after substitution. So all types which support the | operator are now automatically supported. But the result of parameter substitution can now be typing.Union instead of types.Union.

   >>> import typing
   >>> import collections.abc
   >>> T = typing.TypeVar('T')
   >>> (int | T)[list]
   int | list
   >>> (int | T)[typing.List]
   typing.Union[int, typing.List]
   >>> (int | T)[list[int]]
   int | list[int]
   >>> (int | T)[typing.List[int]]
   typing.Union[int, typing.List[int]]
   >>> (int | T)[collections.abc.Hashable]
   int | collections.abc.Hashable
   >>> (int | T)[typing.Hashable]
   typing.Union[int, typing.Hashable]
   >>> (int | T)[collections.abc.Sequence[int]]
   int | collections.abc.Sequence[int]
   >>> (int | T)[typing.Sequence[int]]
   typing.Union[int, typing.Sequence[int]]
   >>> (int | T)[collections.abc.Callable[[int], str]]
   int | collections.abc.Callable[[int], str]
   >>> (int | T)[typing.Callable[[int], str]]
   typing.Union[int, typing.Callable[[int], str]]
   >>> (int | T)[typing.TypeVar('S')]
   int | ~S
   >>> (int | T)[typing.ParamSpec('P')]
   typing.Union[int, ~P]
   >>> (int | T)[str | list]
   int | str | list
   >>> (int | T)[typing.Union[str, list]]
   typing.Union[int, str, list]
5. gvanrossum commented on Jul 19, 2021

   

   gvanrossum

   on Jul 19, 2021

   Member

   More actions

   I need someone else to own this, sorry.
6. ambv commented on Jul 22, 2021

   

   ambv

   on Jul 22, 2021

   Contributor

   More actions

   New changeset 2e3744d by Serhiy Storchaka in branch 'main':
   bpo-44653: Support typing types in parameter substitution in the union type. (GH-27247)
   2e3744d
7. ambv commented on Jul 22, 2021

   

   ambv

   on Jul 22, 2021

   Contributor

   More actions

   New changeset 21db59f by Miss Islington (bot) in branch '3.10':
   bpo-44653: Support typing types in parameter substitution in the union type. (GH-27247) (bpo-27296)
   21db59f
8. ambv commented on Jul 22, 2021

   

   ambv

   on Jul 22, 2021

   Contributor

   More actions

   Thanks Serhiy for the report and the fix, and Ken for reviews.
9. ambv

   closed this as completedon Jul 22, 2021
10. ambv

    closed this as completedon Jul 22, 2021
11. gvanrossum commented on Jul 22, 2021

    

    gvanrossum

    on Jul 22, 2021

    Member

    More actions

    Thanks for taking over here, Ł!
12. ezio-melotti

    transferred this issue fromon Apr 10, 2022