type hint: how should type of lru_cache be defined?

Note: if you are reporting a wrong signature of a function or a class in
the standard library, then the typeshed tracker is better suited
for this report: https://github.com/python/typeshed/issues

Please provide more information to help us understand the issue:

Are you reporting a bug, or opening a feature request? bug
Please insert below the code you are checking with mypy,
or a mock-up repro if the source is private. We would appreciate
if you try to simplify your case to a minimal repro.

@lru_cache(maxsize=1)
def get_response_from_api() -> List[ApiObject]:
    url = _get_api_url()
    response = requests.get(url).text
    return json.loads(response, object_hook=_create_api_obj)

What is the actual behavior/output?

error: Untyped decorator makes function "get_response_from_api" untyped

What is the behavior/output you expect?
it should not emit an error, alternatively there should be instructions what to do.
What are the versions of mypy and Python you are using?
mypy 0.600
python 3.6.5
Do you see the same issue after installing mypy from Git master? did not try
What are the mypy flags you are using? (For example --strict-optional)
mypy predictor --ignore-missing-imports --strict
If mypy crashed with a traceback, please paste
the full traceback below.

Also asked in SO: https://stackoverflow.com/questions/50495759/type-hint-how-should-type-of-lru-cache-be-defined

The underlying issue here is that the type variable for the lru_cache decorator becomes Any since there are no arguments to the function. I think it would be good to handle this case. For now, I'd recommend type ignoring it.

I believe variadic function parameters could help with this.

Well, almost. The type variable only represents the return type; the decorator leaves the arguments unspecified, and the "Untyped decorator makes function "{}" untyped" error (generated by --disallow-untyped-decorators) considers that untyped. Since the function in this case has no arguments it is arguably wrong, but in general the lru_cache decorator does drop the types of the arguments (alas), so `# type: ignore` is still the best you can do ATM.

I don't know if I was just unlucky but I encountered this bug few minutes after attempting to start using mypy for the first time on a project. Lucky me..., I had one function to check, and that one was using @lru_cache.

What is the workaround? (code still supports py27).

Since lru_cache is mostly transparent, maybe this?

from typing import TYPE_CHECKING, TypeVar
if TYPE_CHECKING:
    F = TypeVar('F', Callable)
    def lru_cache(f: F) -> F: pass
else:
    from functools import lru_cache

Or for the parameterised variant:

    def lru_cache(maxsize: int = 128, typed: bool = False) -> Callable[[F], F]:
        pass

(And note also that F should be bound=Callable.)

~~no idea how correct this is, but I took a stab at improving typeshed to "fix" this python/typeshed#3880~~ (didn't work for classes, nevermind)

What about the ability to access the decorated function in order to access cache info, clear the cache, etc.? The type hints for lru_cache decorator should reflect those are available as well so mypy won't emit errors when accessing them. What do you think?

Avoided lru_cache due mypy problems

[pre-commit.ci] pre-commit autoupdate (#142)

Is there a workaround for this issue?

I tried using a type: ignore on the line with @lru_cache but that didn't work. Can I add a type shed for this to my code?

@gvanrossum

The problem with the lru_cache stubs is that we can't get all of the features at the same time. We have to choose one or two of the following:

A) preserving the type signature of the decorated function
B) support for cache_clear methods, etc
C) warning about non-hashable arguments.

The stubs in the typeshed chose B and C, which are arguably valuable, but it's a shame that we have to choose.

The suggestion from @gvanrossum chooses A, which is also valuable:

from typing import TYPE_CHECKING, TypeVar
if TYPE_CHECKING:
    F = TypeVar('F', Callable)
    def lru_cache(f: F) -> F: pass
else:
    from functools import lru_cache

It would be nice if ParamSpec had a basic mechanism to enforce types of arguments, like TypeVar(bound=...):

The end result might look like this:

_P = ParamSpec('P', bound=Hashable)  # <-- enforce all args are hashable
_T = TypeVar('T')

@final
class _lru_cache_wrapper(Generic[_P, _T]):
    def __call__(self, *args: _P.args, **kwargs: _P.kwargs) -> _T: ...
    def cache_info(self) -> _CacheInfo: ...
    def cache_clear(self) -> None: ...
    def __copy__(self) -> _lru_cache_wrapper[_P, _T]: ...
    def __deepcopy__(self, __memo: Any) -> _lru_cache_wrapper[_P, _T]: ...

def lru_cache(maxsize: int | None) -> Callable[Callable[_P, _T], Callable[_P, _T]]
    ...

Shouldn't this be moved to the typeshed repo?

Shouldn't this be moved to the typeshed repo?

We don't currently have the features we need to make the stubs any better. We have to choose the least bad solution, and I think the case could be made in either direction, so there's not a strong argument to break backward compatibility of the stubs.

We either need a new feature (maybe something for ParamSpec) or we need a mypy plugin for lru_cache/cache.

Does anyone have any good ideas about what a new feature to solve this problem would look like? Does my ParamSpec(bound=...) idea make sense? It's limited in usefulness but it seems like it would work.

In Python 3.11 this feature appears to be working correctly

I don't think there's anything actionable for mypy here in any case.

I don't think there's anything actionable for mypy here in any case.

functools.partial was recently special-cased, is there any appetite to do the same for lru_cache, given it's comparably widespread usage?

Navigation Menu

Repository navigation

type hint: how should type of lru_cache be defined? #5107

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

type hint: how should type of lru_cache be defined? #5107

Description

Activity

emmatyping commented on May 24, 2018

gvanrossum commented on May 24, 2018

ssbarnea commented on Sep 8, 2019

gvanrossum commented on Sep 9, 2019

OJFord commented on Feb 19, 2020

asottile commented on Mar 23, 2020

EldarSehayekZenity commented on Feb 17, 2022

JakeSummers commented on Sep 28, 2022

chadrik commented on Dec 17, 2022

gvanrossum commented on Dec 17, 2022

chadrik commented on Dec 17, 2022

JotaRata commented on Aug 19, 2023

JelleZijlstra commented on Aug 19, 2023

gandhis1 commented on Aug 1, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions