mypy doesn't understand partial unpacking with star arguments

Bug Report

While working on qutebrowser/qutebrowser#5317 I think I may have found a bug in mypy. The following code looks correct, and runs fine on my machine.

To Reproduce

Save the following code to file /tmp/reproducer.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: fenc=utf-8:et:ts=4:sts=4:sw=4:fdm=marker
from typing import Optional, Sequence, Mapping
import collections

class ModuleInfo:
    def __init__(
        self,
        name: str,
        version_attributes: Sequence[str],
        min_version: Optional[str] = None
    ):
        self.name = name
        self._version_attributes = version_attributes
        self.min_version = min_version

MODULE_INFO: Mapping[str, ModuleInfo] = collections.OrderedDict([
    (name, ModuleInfo(name, *args))
    for (name, *args) in
    [
        ('sip', ['SIP_VERSION_STR']),
        ('colorama', ['VERSION', '__version__']),
        ('pypeg2', ['__version__']),
        ('jinja2', ['__version__']),
        ('pygments', ['__version__']),
        ('yaml', ['__version__']),
        ('adblock', ['__version__'], "0.3.2"),
        ('cssutils', ['__version__']),
        ('attr', ['__version__']),
        ('PyQt5.QtWebEngineWidgets', []),
        ('PyQt5.QtWebEngine', ['PYQT_WEBENGINE_VERSION_STR']),
        ('PyQt5.QtWebKitWidgets', []),
    ]
])

Run mypy /tmp/reproducer.py.

Expected Behavior

There should be no type errors.

Actual Behavior

The output of the command in step 2 returns:

reproducer.py:19: error: List comprehension has incompatible type List[Tuple[Sequence[str], ModuleInfo]]; expected List[Tuple[str, ModuleInfo]]
reproducer.py:19: error: Argument 1 to "ModuleInfo" has incompatible type "Sequence[str]"; expected "str"
reproducer.py:19: error: Argument 2 to "ModuleInfo" has incompatible type "*List[Sequence[str]]"; expected "Optional[str]"
Found 3 errors in 1 file (checked 1 source file)

Your Environment

Mypy version used: mypy 0.790
Mypy command-line flags: mypy /tmp/reproducer.py
Mypy configuration options from mypy.ini (and other config files): None, I think
Python version used: Python 3.8.6
Operating system and version: Gentoo Linux

This happens because your iterable type is not infered correctly:

from typing import Optional, Sequence, Mapping, Tuple, Union

items = [
    ('sip', ['SIP_VERSION_STR']),
    ('colorama', ['VERSION', '__version__']),
    ('pypeg2', ['__version__']),
    ('jinja2', ['__version__']),
    ('pygments', ['__version__']),
    ('yaml', ['__version__']),
    ('adblock', ['__version__'], "0.3.2"),
    ('cssutils', ['__version__']),
    ('attr', ['__version__']),
    ('PyQt5.QtWebEngineWidgets', []),
    ('PyQt5.QtWebEngine', ['PYQT_WEBENGINE_VERSION_STR']),
    ('PyQt5.QtWebKitWidgets', []),
]

reveal_type(items)
# note: Revealed type is 'builtins.list[builtins.tuple*[typing.Sequence[builtins.str]]]'

And:

from typing import Optional, Sequence, Mapping, Tuple, Union

items: Sequence[Union[
    Tuple[str, Sequence[str]],
    Tuple[str, Sequence[str], str],
]] = [
    ('sip', ['SIP_VERSION_STR']),
    ('colorama', ['VERSION', '__version__']),
    ('pypeg2', ['__version__']),
    ('jinja2', ['__version__']),
    ('pygments', ['__version__']),
    ('yaml', ['__version__']),
    ('adblock', ['__version__'], "0.3.2"),
    ('cssutils', ['__version__']),
    ('attr', ['__version__']),
    ('PyQt5.QtWebEngineWidgets', []),
    ('PyQt5.QtWebEngine', ['PYQT_WEBENGINE_VERSION_STR']),
    ('PyQt5.QtWebKitWidgets', []),
]

reveal_type(items)
# note: Revealed type is 'typing.Sequence[Union[Tuple[builtins.str, typing.Sequence[builtins.str]], Tuple[builtins.str, typing.Sequence[builtins.str], builtins.str]]]'

When I try the following

MODULE_INFO: Mapping[str, ModuleInfo] = collections.OrderedDict([
    (name, ModuleInfo(name, *args))
    for (name, *args) in cast(
        Sequence[Union[Tuple[str, Sequence[str]], Tuple[str, Sequence[str], str]]],
        [
            ('sip', ['SIP_VERSION_STR']),
            ('colorama', ['VERSION', '__version__']),
            ('pypeg2', ['__version__']),
            ('jinja2', ['__version__']),
            ('pygments', ['__version__']),
            ('yaml', ['__version__']),
            ('adblock', ['__version__'], "0.3.2"),
            ('cssutils', ['__version__']),
            ('attr', ['__version__']),
            ('PyQt5.QtWebEngineWidgets', []),
            ('PyQt5.QtWebEngine', ['PYQT_WEBENGINE_VERSION_STR']),
            ('PyQt5.QtWebKitWidgets', []),
        ]
    )
])

I still get the error

reproducer.py:19: error: Argument 2 to "ModuleInfo" has incompatible type "*List[Sequence[str]]"; expected "Optional[str]"
Found 1 error in 1 file (checked 1 source file)

Ok, I followed all the way down to the rabbit hole.

This happens because is_subtype_ignoring_tvars returns that str is a subtype of Sequence. And [Sequence[str], str] gets joined into Sequence[str]. This is happing in the check_multi_assignment_from_union method of Checker

In contrast to

from typing import Optional, Sequence, Mapping, Tuple, Union

items: Sequence[Union[
    Tuple[str, Sequence[str]],
    Tuple[str, Sequence[str], int],
]]

for a, *args in items:
    reveal_type(a)
    reveal_type(args)

# ex.py:25: note: Revealed type is 'builtins.str'
# ex.py:26: note: Revealed type is 'Union[builtins.list[typing.Sequence*[builtins.str]], builtins.list[builtins.object*]]'

Navigation Menu

Repository navigation

mypy doesn't understand partial unpacking with star arguments #9706

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

mypy doesn't understand partial unpacking with star arguments #9706

Description

Activity

sobolevn commented on Nov 6, 2020

ArniDagur commented on Nov 6, 2020

sobolevn commented on Nov 6, 2020

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions