Skip to content

functools.lru_cache doesn't work with generic functions #1515

Closed

Description

@rwbarton
from typing import TypeVar
import functools

T = TypeVar('T')

@functools.lru_cache()
def f(x: T) -> T: pass

reveal_type(f) # E: Revealed type is 'functools._lru_cache_wrapper[T`-1]'
reveal_type(f(1)) # E: Revealed type is 'T`-1'

These types are wrong: types are not allowed to contain unbound type variables.

There isn't really a clearly correct thing to do here though. The stubs for lru_cache are defined in a way that does not retain the relationship between the input and output types of the function being decorated. Options here include

  • Find a way to retain the entire Callable type of the decorated function while having the result of lru_cache's __call__ simultaneously be an instance of _lru_cache_wrapper. As far as I can tell, this is currently impossible, but would be the ideal long-term outcome.

  • Give up on _lru_cache_wrapper and CacheInfo (for now?) and just give lru_cache the type

    def lru_cache(f: _TC, maxsize: int = ..., typed: bool = ...) -> _TC: ...

    where _TC = TypeVar(_TC, bound=Callable), basically reverting python/typeshed@a40418e.

  • Have mypy infer the type _lru_cache_wrapper[Any] for this f on the grounds that the type variable _T is undetermined. But normally in this situation (where an undetermined type variable appears in an inferred type) mypy either uses 'None' or complains about being unable to infer a type. What makes this case special?

This kind of situation occurs in stdlib-samples/3.2/fnmatch.py:

@functools.lru_cache(maxsize=250)
def _compile_pattern(pat: AnyStr,
                     is_bytes: bool = False) -> Callable[[AnyStr],
                                                         Match[AnyStr]]:
    ...

It accidentally happens to not cause an error at the uses of _compile_pattern because of another issue, #1261.

Activity

  1. JukkaL commented on May 10, 2016

    @JukkaL
    Collaborator

    Interestingly, this is an example where intersection types would be useful. We could annotate lru_cache like this:

    def lru_cache(
        f: _TC, 
        maxsize: int = ..., 
        typed: bool = ...) -> Intersection[_TC, <wrapper type>]: ...
    
  2. ilevkivskyi commented on May 18, 2018

    @ilevkivskyi
    Member

    I think this is a duplicate of #1317

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions