Annotated and Optional get_type_hints buggy interaction

@gvanrossum

BPO	46195
Nosy	@gvanrossum, @JelleZijlstra, @sobolevn, @Fidget-Spinner
PRs	bpo-46195: Do not add `Optional` in `get_type_hints` #30304

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-03-02.05:31:58.542>
created_at = <Date 2021-12-29.18:38:47.765>
labels = ['type-bug', '3.9', '3.10', '3.11']
title = 'Annotated and Optional get_type_hints buggy interaction'
updated_at = <Date 2022-03-02.05:31:58.542>
user = 'https://bugs.python.org/med2277'

bugs.python.org fields:

activity = <Date 2022-03-02.05:31:58.542>
actor = 'JelleZijlstra'
assignee = 'none'
closed = True
closed_date = <Date 2022-03-02.05:31:58.542>
closer = 'JelleZijlstra'
components = []
creation = <Date 2021-12-29.18:38:47.765>
creator = 'med2277'
dependencies = []
files = []
hgrepos = []
issue_num = 46195
keywords = ['patch']
message_count = 8.0
messages = ['409319', '409325', '409340', '409341', '409357', '409375', '414328', '414329']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'JelleZijlstra', 'sobolevn', 'kj', 'med2277']
pr_nums = ['30304']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46195'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

This is two closely related issues with get_type_hints on an optional annotated member. I'm testing with Annotated/get_type_hints from typing extensions on 3.8 and assuming they're backport equivalent to current behavior.

The first issue is get_type_hints has inconsistent behavior depending on whether annotation comes from a function with a None default or an attribute with a None default.

class Foo:
    def __init__(
        self,
        x: 
            Annotated[Optional[str], "doc string"] = None,
    ):
        ...

class Foo2:
    x: Annotated[Optional[str], "doc string"] = None

get_type_hints(Foo.__init__) 
# {'x': typing.Union[typing_extensions.Annotated[typing.Union[str, NoneType], 'doc string'], NoneType]}

get_type_hints(Foo2)
# {'x': typing_extensions.Annotated[typing.Union[str, NoneType], 'doc string']}

Attributes with a None default are not wrapped by get_type_hints, but function parameters. Which of the two behaviors is correct I don't know, but I'd expect the two to be equivalent annotations.

The second issue is for function arguments with a None default the optional wrapper happens even if the type inside annotated already has optional. Example,

from typing_extensions import Annotated, get_type_hints

class Foo:
    def __init__(
        self,
        x: 
            Annotated[Optional[str], "doc string"] = None,
    ):
        ...

get_type_hints(Foo.__init__, include_extras=True)
# {'x': typing.Union[typing_extensions.Annotated[typing.Union[str, NoneType], 'doc string'], NoneType]}

For Annotated types I would expect any type rules like wrapping to apply only to the first argument and not the entire annotation. I mainly ran into this for a runtime type introspection library (similar in spirit to pydantic).

As a note include_extras being True or False while it changes type is an issue in either case. With include_extras as False the Annotated goes away, but the type still gets double wrapped as an Optional.

Could you try with 3.10 and the stdlib typing.Annotated please? There might
be changes (in the past a default of None automatically caused an Optional
to be added, but we changed our minds.--
--Guido (mobile)

I can verify that this happens on 3.10 and main branches:

from typing import Annotated, Optional, get_type_hints

class Foo:
    def __init__(self, x: Annotated[Optional[str], "d"] = None): ...

class Foo2:
    x: Annotated[Optional[str], "d"] = None

print(get_type_hints(Foo.__init__, include_extras=False))  # ok
# {'x': typing.Optional[str]}
print(get_type_hints(Foo2, include_extras=False))  # ok
# {'x': typing.Optional[str]}

print(get_type_hints(Foo.__init__, include_extras=True))  # not ok?
# {'x': typing.Optional[typing.Annotated[typing.Optional[str], 'd']]}
print(get_type_hints(Foo2, include_extras=True))  # ok
# {'x': typing.Annotated[typing.Optional[str], 'd']}

Notice that 3rd case does not look correct: {'x': typing.Optional[typing.Annotated[typing.Optional[str], 'd']]}

In my opinion it should be {'x': typing.Annotated[typing.Optional[str], 'd']}

I will look into it! :)

And on 3.9 as well.

api: Avoid programming errors due to nested Annotated types.

Navigation Menu

Repository navigation

Annotated and Optional get_type_hints buggy interaction #90353

26 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Annotated and Optional get_type_hints buggy interaction #90353

Description

Activity

med2277 commented on Dec 29, 2021

gvanrossum commented on Dec 29, 2021

sobolevn commented on Dec 30, 2021

sobolevn commented on Dec 30, 2021

26 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions