Repository navigation
problem with the type definition of sum #7974
Description
Everything is explained in microsoft/pylance-release#2865. I copy it to avoid redirection.
Environment data
Language Server version: 2022.5.3
OS and version: macOS 10.14.6 (Mojave)
Python version (& distribution if applicable, e.g. Anaconda): 3.10.4
Code Snippet
def test(N:int) -> bool:
from random import choice
return sum(a for a in [choice([True,False]) for _ in range(N)])==2Expected behavior
This code should be perfectly fine.
Actual behavior
Problem detected on the last line
Pylance computes that it will always return False. Because it implies that sum on bool is always 0 or a bool.
Condition will always evaluate to False since the types "bool | Literal[0]" and "Literal[2]"
have no overlapPylance[reportUnnecessaryComparison]
(https://github.com/microsoft/pylance-release/blob/main/DIAGNOSTIC_SEVERITY_RULES.md#diagnostic-severity-rules)
Two problems here :
1/ bool should be considered as int, because they are ! (True+True==2)
2/ Even with ==1 at the end of the line, pylance seems to think that (True==1) is False.
Answer of Eric Traut when I posted this bug report on pylance :
I'm glad to see that you're using the reportUnnecessaryComparison diagnostic check! It's a strict-mode check, so most pylance users are not aware of it.
This looks like a bug in the type definition of sum, which is defined in the typeshed stubs here:
def sum(__iterable: Iterable[_SumT]) -> _SumT | Literal[0]: ...That leads to the following:
x = sum([True, True, True])
reveal_type(x) # bool | Literal[0]However, the correct (runtime) type of this expression is int.
So the bug will need to be fixed in the typeshed stubs. It will probably require a special-case overload for sum to properly model its behavior. Please file a bug (or a PR) in the typeshed project.
Here are a few suggested workarounds:
Use ints rather than bools in your code:
def test(N: int) -> bool:
return sum([choice([1, 0]) for _ in range(N)]) == 2Add a # pyright: ignore[reportUnnecessaryComparison] or # type: ignore comment to the end of the line.
Disable that diagnostic check for the file by adding a # pyright: reportUnnecessaryComparison=false comment at the top of the file.
Activity
AlexWaygood commentedon May 28, 2022
We can add another overload to the stub here to fix this special case. I do agree with @erictraut that it's probably more idiomatic to use
ints here, though.ChristophePapazian commentedon May 28, 2022
I want to count the number of True in a generator. Sum does that.
What's really wrong to me is that python considers
Trueas1subtyped as aboolandFalseas0subtyped asbool. So3-Trueis perfectly ok (because bool are int) but the type system doesn't seem to treatboolas a subtype ofintAlexWaygood commentedon May 28, 2022
This comment of mine was incorrect; I deleted it shortly after posting it :)
Not quite. The type system does recognise
boolas a subtype ofint; that's not what causes the issue here.Here's the stub as it exists currently.
_SumThere is solved as beingbool, which is what is causing the error:typeshed/stdlib/builtins.pyi
Lines 1548 to 1552 in 3191c1a
But when you add
True + False, the result is of typeint, not typebool:None of the
sumoverloads currently take account of the fact that adding two objects of the same type does not always result in the same type being returned. This is a general problem that's quite hard to solve here, but we can certainly add a special case for this specific instance.`builtins.sum`: Add overload for `bool` special case
ChristophePapazian commentedon May 28, 2022
Perfect ! Thank you 😀
`builtins.sum`: Add overload for `bool` special case (#7975)