Skip to content

Failing to recognize generic constraints in tuples #7702

Closed

Description

@mikeknep
  • Are you reporting a bug, or opening a feature request? Bug

The basic context here is I need to map data into different shapes using various Mappings, and then write those shapes to different targets using various Writers. I want to ensure that when I define a job configuration, I get a Mapping and Writer that are compatible with one another (i.e. they map to and write the same shape).

Initially I tried returning a Tuple[Mapping[E], Writer[E]], but found that mypy does not report an error when I deliberately mismatch a Mapping and Writer. Fortunately, however, an error is raised when I define a (data)class to hold these two objects instead of just a tuple. I think using a dataclass here is probably a good idea regardless (for reader comprehension purposes), but I would nevertheless expect the tuple implementation to behave the same way. It seems like someone could try the tuple implementation, not see any errors, and incorrectly assume they have a typechecking safety net that isn't actually there.

from dataclasses import dataclass
from typing import Generic, Tuple, TypeVar
from typing_extensions import Protocol


M = TypeVar("M", covariant=True)
class Mapping(Protocol[M]):
    def map(self) -> M:
        ...


W = TypeVar("W", contravariant=True)
class Writer(Protocol[W]):
    def write(self, shape: W):
        ...


class Circle:
    pass


class CircleMapping:
    def map(self) -> Circle:
        return Circle()


class CircleWriter:
    def write(self, shape: Circle):
        print("in CircleWriter.write")


class Square:
    pass


class SquareMapping:
    def map(self) -> Square:
        return Square()


class SquareWriter:
    def write(self, shape: Square):
        print("in SquareWriter.write")


E = TypeVar("E")


## Tuple strategy
Tools = Tuple[Mapping[E], Writer[E]]

def get_tools(shape_name: str) -> Tools:
    if shape_name == "square":
        return (SquareMapping(), CircleWriter())   # mypy does not flag an error here!
    else:
        return (CircleMapping(), CircleWriter())

def run_1(tools: Tools):
    (mapping, writer) = tools
    shape = mapping.map()
    writer.write(shape)



## Dataclass strategy
@dataclass
class EtlTools(Generic[E]):
    mapping: Mapping[E]
    writer: Writer[E]

def get_etl_tools(shape_name: str) -> EtlTools:
    if shape_name == "square":
        return EtlTools(mapping=SquareMapping(), writer=CircleWriter()) # mypy error, cannot infer type argument 1 of EtlTools
    else:
        return EtlTools(mapping=CircleMapping(), writer=CircleWriter())

def run_2(etl_tools: EtlTools):
    shape = etl_tools.mapping.map()
    etl_tools.writer.write(shape)
  • What is the actual behavior/output?
    Mypy only errors on the dataclass implementation, not the tuple implementation

  • What is the behavior/output you expect?
    The tuple implementation should report an error as well

  • What are the versions of mypy and Python you are using?
    mypy = "==0.720"
    python version 3.7.4

  • What are the mypy flags you are using? (For example --strict-optional)
    None

Activity

  1. JelleZijlstra commented on Oct 11, 2019

    @JelleZijlstra
    Member

    This is probably because you're missing a type argument to Tools, which is a generic alias. Try returning Tools[E].

    The --disallow-any-generics option (which ideally should be on by default) would have told you this.

  2. mikeknep commented on Oct 11, 2019

    @mikeknep
    Author

    @JelleZijlstra Thanks for the flag tip. I'm trying your suggestion, but here with correct pairings:

    def get_tools(shape_name: str) -> Tools[E]:
        if shape_name == "square":
            return (SquareMapping(), SquareWriter())
        else:
            return (CircleMapping(), CircleWriter())
    

    Error message from mypy:

    Incompatible return value type (got "Tuple[SquareMapping, SquareWriter]", expected "Tuple[Mapping[E], Writer[E]]")
    Incompatible return value type (got "Tuple[CircleMapping, CircleWriter]", expected "Tuple[Mapping[E], Writer[E]]")
    
  3. JelleZijlstra commented on Oct 11, 2019

    @JelleZijlstra
    Member

    Interesting, that seems like a bug to me.

  4. JukkaL commented on Oct 14, 2019

    @JukkaL
    Collaborator

    Mypy treats a type variable in the return type (but not in an argument type) as meaning that the value of that variable is specified by the calling context. A typical example is set(), which returns Set[T] for arbitrary T. In the example get_tools doesn't return Tools[E] for an arbitrary E, and this is what mypy is complaining about. This is probably poorly documented, though.

  5. ilevkivskyi commented on Oct 14, 2019

    @ilevkivskyi
    Member

    @JukkaL I think this is then a duplicate of #2885 (added the docs label there).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions