Narrowing str | dict[str, str] with isinstance and .get() #13351

Activity

1. Akuli

   added

   bugmypy got something wrongmypy got something wrong

   on Aug 7, 2022
2. AlexWaygood

   added

   topic-type-narrowingConditional type narrowing / binderConditional type narrowing / binder

   on Aug 7, 2022
3. RedKnite5 commented on Aug 19, 2022

   

   RedKnite5

   on Aug 19, 2022

   Last edited by RedKnite5

   More actions

   I've been looking at this and figured out so far that when mypy accepts the CallExpr node Union[builtins.str, builtins.dict[builtins.str, builtins.str]] is added to self.type_context in ExpressionChecker. This is propagated to apply_generic_arguments in applytype.py as context where it is used to deduce the return type of the call to get.

   The problem I'm having is that I'm not familiar with how apply_generic_arguments works so I'm not sure where in the function it messes up and what possible solutions would actually totally break its intended function.
4. Akuli commented on Aug 19, 2022

   

   Akuli

   on Aug 19, 2022

   Last edited by Akuli

   ContributorAuthor

   More actions

   I looked into this a bit too. Here's a more self-contained reproducer:

   from __future__ import annotations
   from typing import TypeVar
   
   T = TypeVar("T")
   
   class MyDict:
       def get_item(self) -> str: ...
       def get(self, x: T) -> str | T: ...
   
   def first() -> None:
       tag_name: str | MyDict
       if isinstance(tag_name, MyDict):
           tag_name = tag_name.get("asd")
       reveal_type(tag_name)
   
   def second() -> None:
       tag_name: str | MyDict
       if isinstance(tag_name, MyDict):
           tag_name = tag_name.get_item()
       reveal_type(tag_name)

   Next I tried:

   def third() -> None:
       tag_name: str | MyDict
       if isinstance(tag_name, MyDict):
           reveal_type(tag_name)               # MyDict
           reveal_type(tag_name.get("asd"))    # str
           tag_name = tag_name.get("asd")
           reveal_type(tag_name)               # str | MyDict

   When I do reveal_type(tag_name.get("asd")), that happens in a new type context, and works as expected. But when I do tag_name = tag_name.get("asd"), the type of tag_name is taken from the type context as a suggestion for what the type of tag_name.get("asd") might be. To make that happen, mypy resolves T to appropriately.

   I think the bug is that str | MyDict is still in the type context by the time we get to the type-narrowed part of the code. When the type narrowing happens, the type of tag_name should become MyDict, and whatever gets str | MyDict into the type context is buggy: it's not looking up the type of the newly-narrowed local variable correctly.
5. RedKnite5 commented on Aug 19, 2022

   

   RedKnite5

   on Aug 19, 2022

   Last edited by RedKnite5

   More actions

   I think its about the assignment back to the typed expression. This also exhibits the same behavior and follows a similar data path, but doesn't have type narrowing in the same way.

   from typing import TypeVar
   T = TypeVar("T")
   
   def func(default: T) -> str | T: ...
   
   def example() -> None:
   	tag_name: str | int
   	tag_name = func("a")
   	reveal_type(tag_name)
   

   Its the union of a type and a generic that turns out to be the same type assigned back to a variable with the union type.
   Or maybe I'm misunderstanding how the context stack is supposed to work.

   If the type hint on tag_name is omitted:

   def example() -> None:
   	tag_name = func("a")
   	reveal_type(tag_name)
   

   then when it gets to the function infer_function_type_arguments_using_context it returns early without inferring anything and proceeds to get the right answer. So it could be that str | int shouldn't be in the context any more, but I'm not sure.

   Edit: Made the example more minimal
6. hauntsaninja

   added

   topic-type-contextType context / bidirectional inferenceType context / bidirectional inference

   on Aug 19, 2022
7. Akuli commented on Aug 20, 2022

   

   Akuli

   on Aug 20, 2022

   ContributorAuthor

   More actions

   In my example the type of tag_name should be MyDict, not str | MyDict, because type narrowing has happened. While it would be nice for tag_name = func("a") to narrow to str, it's not what this issue is about: here the narrowing has happened already, but the type context isn't picking it up.

   Your example, modified to match what I consider to be this issue's scope, would be:

   from typing import TypeVar
   T = TypeVar("T")
   
   def func(default: T) -> str | T: ...
   
   def example() -> None:
   	tag_name: str | int
   	if isinstance(tag_name, str):
   		tag_name = func("a")
   		reveal_type(tag_name)  # str | int :(
8. AlexWaygood commented on Nov 22, 2022

   

   AlexWaygood

   on Nov 22, 2022

   Member

   More actions

   Fixed in #14151
9. AlexWaygood

   closed this as completedon Nov 22, 2022