False positive when mixing Mapping and Union #6001

Activity

1. JelleZijlstra commented on Dec 4, 2018

   

   JelleZijlstra

   on Dec 4, 2018

   Member

   More actions

   This is because Mapping is invariant in its key type. See https://mypy.readthedocs.io/en/latest/common_issues.html#invariance-vs-covariance.
2. JelleZijlstra

   closed this as completedon Dec 4, 2018
3. wkschwartz commented on Dec 4, 2018

   

   wkschwartz

   on Dec 4, 2018

   ContributorAuthor

   More actions

   I'm having a hard time understanding how this isn't a bug:

   1. str should satisfy Union[str, int]: str is not a subclass of Union[str, int] so variance shouldn't matter.
   2. g({'b': 'c'}) should be equivalent to d = {'b': 'c'}; g(d).
   3. The docs you linked says using immutable types mitigates the unexpected behavior—Mapping is an immutable type.
4. ilevkivskyi commented on Dec 4, 2018

   

   ilevkivskyi

   on Dec 4, 2018

   Last edited by ilevkivskyi

   Member

   More actions

   Mutability is not a root cause, the root cause is invariance. Mutable collections are typically invariant, but an immutable collection can be invariant as well.
5. JelleZijlstra commented on Dec 4, 2018

   

   JelleZijlstra

   on Dec 4, 2018

   Member

   More actions

   python/typing#273 discusses why Mapping's key type is invariant.
6. wkschwartz commented on Dec 4, 2018

   

   wkschwartz

   on Dec 4, 2018

   ContributorAuthor

   More actions

   What about items 1 and 2? How can invariance prevent str from satisfying Union[str, int]? Why would it affect variables but not literals?
7. JelleZijlstra commented on Dec 4, 2018

   

   JelleZijlstra

   on Dec 4, 2018

   Member

   More actions

   2. is due to type context, where mypy infers a different type if you're accessing something in a context where a particular type is expected.

   3. My understanding is that str is a subtype (though not subclass) of Union[str, int].
8. bluenote10 commented on Jun 20, 2022

   

   bluenote10

   on Jun 20, 2022

   Contributor

   More actions

   FYI related discussion python/typing#445 and feature request python/typing_extensions#5
9. randolf-scholz commented on Nov 30, 2022

   

   randolf-scholz

   on Nov 30, 2022

   Last edited by randolf-scholz

   Contributor

   More actions

   This is not type safe for the following reason:

   • g may be using the arg.__getitem__ function.
   • Imagine, we had a UserMapping class that is a Mapping[UserString, Any] type that used UserString keys, a subclass of str with an additional method transliterate. Furthermore, assume UserMapping made use of that UserString.transliterate method in the UserMapping.__getitem__ call. (Imagine for example that the Mapping transliterates Greek to Latin characters before attempting a lookup.)
   • Then, if we pass an instance of this UserMapping to g, g might attempt to perform a lookup with a regular string that lacks the .transliterate method. Thus it is not type safe.

   In essence, because Mapping has both methods where the keys appear as input and output, Mapping needs to be invariant in the keys.

   ---

   A way around this is:

   from typing import Any, Mapping, Union, TypeVar
   
   StrVar = TypeVar("StrVar", bound=str)
   StrOrIntVar = TypeVar("StrOrIntVar", bound=Union[str, int])
   
   def f(arg: Mapping[      StrVar      , Any]) -> None: pass
   def g(arg: Mapping[StrOrIntVar, Any]) -> None: pass
   f({'b': 'c'})
   g({'b': 'c'})
   d = {'b': 'c'}
   f(d)
   g(d)

   Using a TypeVar kind of acts as an @overload for every possible str class.

   ---

   Of course, in the particular case of str as a subtype of Union[str, int] it would be nice if the type-checker were a bit smarter. Because here really str is kind of stronger than just a subtype.