Skip to content

mypy caches types of attributes for too long, ignoring side effects of methods #14010

Closed as not planned

Description

@finite-state-machine

Bug Report

Consider the following code [link to mypy-play.net]:

# run mypy with '--show-error-codes --strict --warn-unreachable'

class SomeClass:

    cache_hot: bool = False

    def some_method(self) -> None:
        # expensive computation goes here
        self.cache_hot = True


def test_some_class() -> None:

    instance = SomeClass()
    assert not instance.cache_hot

    instance.some_method()
            # mypy fails to notice that 'cache_hot' has changed

    reveal_type(instance.cache_hot)
            # mypy: note: Revealed type is "Literal[False]"
            # (this is incorrect)
    assert instance.cache_hot  # mypy treats this as 'assert False'

    assert isinstance(instance, SomeClass)
            # mypy: error: Statement is unreachable  [unreachable]

mypy is caching the type of instance.cache_hot, which is narrowed by the assert statement to Literal[False]; it relies on this type even after side effects of some_method() have changed the value of that attribute. This makes is difficult or impossible to test that the attribute is changed as a side-effect of calling some_method().

The issue affects properties as well as attributes.

Workarounds

Edit: see much better suggestions in comments, below.

mypy does not cache function return values (provided they're not decorated to be a @property).

To Reproduce

See above.

Expected Behavior

If mypy cannot notice the side effect of some_method(), there should be some way to reset the type of attributes on a particular instance.

Actual Behavior

The exact output is:

thecodeabove.py:22: note: Revealed type is "Literal[False]"
thecodeabove.py:27: error: Statement is unreachable  [unreachable]
Found 1 error in 1 file (checked 1 source file)

Your Environment

  • Mypy version used: master, 0.981
  • Mypy command-line flags: --show-error-codes --strict --warn-unreachable
  • Mypy configuration options from mypy.ini (and other config files): N/A
  • Python version used: 3.8, 3.10

Activity

  1. Avasam commented on Nov 4, 2022

    @Avasam
    SponsorContributor

    Not sure it could even analyze that statically (in more complex cases) because of the mutation.
    You can, however, decide to never typeguard directly on your mutated property. This way mypy and pyright will keep seeing it as a bool.

    from typing_extensions import reveal_type
    
    class SomeClass:
    
        cache_hot: bool = False
    
        def some_method(self) -> None:
            # expensive computation goes here
            self.cache_hot = True
    
    
    def test_some_class() -> None:
    
        instance = SomeClass()
        cache_hot = instance.cache_hot
        assert not cache_hot
    
        instance.some_method()
    
        cache_hot = instance.cache_hot
        reveal_type(cache_hot)
        assert cache_hot
    
        assert isinstance(instance, SomeClass)
  2. hauntsaninja commented on Nov 4, 2022

    @hauntsaninja
    Collaborator

    Yeah, this is a heuristic mypy uses that is needed in practice. Other type checkers do this too. Would need some complicated type system feature to make this expressible.

    There are a couple workarounds, but one good one is a dummy re-assignment instance = instance

  3. finite-state-machine commented on Nov 4, 2022

    @finite-state-machine
    Author

    I agree this would be very difficult to definitively fix, but I was surprised I couldn't find an issue (open or closed) describing this problem or ways to work around it.

    @Avasam @hauntsaninja thank you both for the very helpful suggestions!

  4. finite-state-machine commented on Nov 5, 2022

    @finite-state-machine
    Author

    I wonder: for the benefit of others who encounter this issue in future, is there some way to make this thread (or the gist of it) easier to find? I anticipate most people won't search for closed issues.

  5. ShadowLNC commented on Dec 3, 2022

    @ShadowLNC

    Yeah this issue is difficult to find, I nearly ended up creating a duplicate.
    Interestingly upgrading from mypy version 0.910 to 0.991 and from Python 3.8 to 3.11 caused some similar issues for us - we're getting a false positive about Statement is unreachable too.

    This self-contained example does the same thing (although I know mypy struggles with @property sometimes):

    import time
    
    
    class X:
        start: float
    
        def __init__(self) -> None:
            self.start = time.time()
    
        @property
        def has_time_passed(self) -> bool:
            return (time.time() - self.start) >= 3
    
        def sleep(self) -> None:
            time.sleep(3)
    
    
    val = X()
    assert not val.has_time_passed, "..."
    val.sleep()
    assert val.has_time_passed, "..."
    assert val.has_time_passed, "..."

    Our actual code is slightly more complex of the form:

    assert not x.some_prop, "..."
    assert not y.some_prop, "..."
    assert not z.some_prop, "..."
    
    x.some_func("...", y.other_prop, z.other_prop)
    assert x.some_prop, "..."
    # Flagged as unreachable, all 3 statements here assume `some_prop` is `false`.
    assert y.some_prop, "..."
    assert z.some_prop, "..."

    I find it a little strange that it didn't previously narrow this type (Python 3.8, mypy 0.910) and I'm not sure when it changed given how many releases have happened since then. In any case, the upgrade has fixed some of our other type: ignore statements, so on balance I think it's still a win 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugmypy got something wrong

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions