TypeError from sorting incompatible types isn't flagged #9122

Activity

1. hauntsaninja commented on Jul 10, 2020

   

   hauntsaninja

   on Jul 10, 2020

   Last edited by hauntsaninja

   Collaborator

   More actions

   Hmm, there have been some typeshed changes aimed at fixing this, eg: python/typeshed#4192 (that haven't made it to the latest mypy release). Note that they've definitely helped when passing key, but clearly haven't fixed your case:

   ~/tmp λ cat test19.py                                      
   z = [1, None]
   reveal_type(z)
   z.sort()
   
   ~/tmp λ mypy test19.py                                     
   test19.py:2: note: Revealed type is 'builtins.list[Union[builtins.int, None]]'
   ~/tmp λ mypy test19.py --custom-typeshed-dir ~/dev/typeshed
   test19.py:2: note: Revealed type is 'builtins.list[Union[builtins.int, None]]'
   
   ~/tmp λ vim test19.py
   ~/tmp λ cat test19.py
   z = [1, None]
   reveal_type(z)
   z.sort(key=lambda x: x)
   
   ~/tmp λ mypy test19.py                                     
   test19.py:2: note: Revealed type is 'builtins.list[Union[builtins.int, None]]'
   ~/tmp λ mypy test19.py --custom-typeshed-dir ~/dev/typeshed
   test19.py:2: note: Revealed type is 'builtins.list[Union[builtins.int, None]]'
   test19.py:3: error: Argument "key" to "sort" of "list" has incompatible type "Callable[[Optional[int]], Optional[int]]"; expected "Callable[[Optional[int]], _SupportsLessThan]"
   test19.py:3: error: Incompatible return value type (got "Optional[int]", expected "_SupportsLessThan")
   Found 2 errors in 1 file (checked 1 source file)
   

   The typeshed change looks correct to me now and looked correct to me at the time. Maybe mypy doesn't deal with typevar upper bounded by protocols in self types correctly?
   https://github.com/python/typeshed/blob/028f0d52931fe1f96bb25d066186961159c1f801/stdlib/2and3/builtins.pyi#L961
   https://github.com/python/typeshed/blob/028f0d52931fe1f96bb25d066186961159c1f801/stdlib/2and3/builtins.pyi#L84
2. akaptur commented on Jul 11, 2020

   

   akaptur

   on Jul 11, 2020

   Last edited by akaptur

   Author

   More actions

   Oh, interesting - the key version of this is actually more similar to my original code. For example, neither of these sorts produces a warning:

   from typing import Optional
   
   class A:
   
       def __init__(self, thing: Optional[int]) -> None:
           self.thing = thing
   
   
   
   sorted([A(1), A(None)], key=lambda a: a.thing)
   
   # or without the lambda
   def sort_func(a: A) -> Optional[int]:
       return a.thing
   
   sorted([A(1), A(None)], key=sort_func)
3. hauntsaninja commented on Jul 11, 2020

   

   hauntsaninja

   on Jul 11, 2020

   Collaborator

   More actions

   The next release of mypy will complain about both of those sorts :-) If you point mypy at a copy of typeshed master using --custom-typeshed-dir you can get those errors today.

   It's unfortunate that in the key-less case, mypy doesn't seem to be able to use the newly more accurate typeshed stubs, so we should still keep this issue open. Seems mypy has some issue with typevars upper bounded by protocols in self types, but that sentence covers a lot of surface area, so would need to do some digging to actually see where things go wrong 🤷‍♂️
4. mortoray commented on Jul 27, 2020

   

   mortoray

   on Jul 27, 2020

   More actions

   Here's another simple example that came up in our code:

   from typing import List, Union
   
   
   def main() -> None:
       values: List[Union[float, str]] = [1, "hi", 2, "zen"]
       print(sorted(values))
   
   
   main()
   

   Will the next release catch this one?
5. JukkaL commented on Sep 11, 2020

   

   JukkaL

   on Sep 11, 2020

   Collaborator

   More actions

   @mortoray That probably won't be caught, since both float and str are comparable (they just can't be compared with each other). We could support this via a plugin for sorted (and sort) that does some special casing.
6. JukkaL

   added

   feature

   on Sep 11, 2020
7. hauntsaninja commented on Sep 11, 2020

   

   hauntsaninja

   on Sep 11, 2020

   Collaborator

   More actions

   The next version of mypy will actually catch:

   values: List[Union[float, str]] = [1, "hi", 2, "zen"]
   values.sort(lambda x: x)
   

   But not other ways of sorting. I believe this is due to a mypy bug. #9122 (comment)