Proposal: treat "obvious" return type as annotated

(Not sure if it should be here or in typing)

Consider this function:

def f():
    return "hello"
def g(x):
    if x: return A(1)
    else: return A(2)

f obviously (without any deep analysis) returns str, and g returns A. Why not use this information? This pattern is very common, and taking advantage of it can help in precise checking and remove clutter (for example -> Union[Tuple[int, int, str], Tuple[int, str, bool]]).

I propose treating calls to functions whose return expressions consists solely of literals and constructor calls as if they were declared with the returned type (join or union of the return types).

Well, I always felt that the main reason for this was to allow mypy to be used on already-existing codebases, so the unannotated functions don't need to be type-safe (yet). This would change that behavior.

@kirbyfan64

@kirbyfan64 I think I agree with what you are saying, but I also think there isn't a reason something like this couldn't be behind a flag, I've heard from several people that this is a significant pain point, as people don't want to have to do a lot of work to annotate obvious functions, so big +1 from me.

@kirbyfan64

@kirbyfan64 the proposal does not require unannotated functions to be type-safe. It simply allows annotated functions that use them to be slightly more type safe (peek into the immediate return expressions, if there is one).

To elaborate:

class A: pass

def f(): return A(1)  # no error here, even though A has no no-arg __init__

def g() -> None:
    f().bar()  # error here

However when is it obvious enough? Where do you draw the line?

…

On Dec 25, 2017 13:43, "Elazar Gershuni" ***@***.***> wrote: To elaborate: class A: pass def f(): return A(1) # no error here, even though A has no no-arg __init__ def g() -> None: f().bar() # error here — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4409 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACwrMlh2ba7M5ihwiKF6_8U6p7N_2f9Kks5tEAjhgaJpZM4RMQzD> .

Either a literal or a direct constructor call on a globally defined class. (Part of the idea is that the return statements are also the documentation).

Expression atoms would be straightforward enough (not certain about identifiers however). At some point this could probably be expanded to entire expressions I believe (down the road).

Yes, that could be a nice path. Atoms, then (recursively) tuples/lists, then simple names. My hope is this could be done as part of semantic analysis, just like the analysis of the annotations themselves.

Could a director constructor call really be safe though? e.g.:

def func():
    A = lambda x: x
    return A(None)

Or even:

def func():
    return A(1)

func.__globals__['A'] = lambda x: x

Of course, this isn't ideal, but it happens in unannotated code.

I think it will be as safe as any other analysis performed by mypy. The first example is not a call to a global constructor; the second example defeats any other analysis.

I guess the discussion can be separated to two orthogonal questions:

What should be considered as "obvious" return type
How this mechanism is enabled: a flag for unannotated functions, a flag for partially-annotated functions, using an Infer annotation, or some other means

Note there is an old proposal python/typing#276 to make this explicit. During the discussion it became clear that inferring types for arguments can be very hard, but for return types it is probably OK.

Recently, additional interest in this appeared in the context of data classes (PEP 557), where annotations are syntactically required, but people may be lazy to write exact types, so that this could be allowed:

@dataclass
class C:
    x: ... = 5  # equivalent to 'x = 5' (without an annotation) for a type checker

while for function this will work as

def func() -> ...:
    return 5

reveal_type(func())  # Revealed type is 'builtins.int'

An additional argument in favour of ellipsis is that libraries in numeric stack use ellipsis for inferred number of dimensions.

One problem with annotations is that they mean that the function will be checked. It also does not completely address the issue of clutter and laziness.

I agree that if this is the direction, then the ellipsis idea is nice. But two minor points should be noted: its meaning is unrelated to Tuple[int, ...], and ... everywhere might make the code look bad.

@elazarg

@elazarg

One problem with annotations is that they mean that the function will be checked.

It is maybe a matter of taste, but I think it is rather a plus, since I like to keep the current separation between annotated and non-annotated functions.

The idea is to not to make the human hands robotic, but to give them fingernails 😄

Behind a flag, the taste will be the user's.

@kstauffer

@kstauffer That doesn't sound like a useful solution at all. It disables a whole bunch of warnings for all affected code, rather than one specific warning for just the functions where it is necessary. All it will lead to is other legitimate warnings being missed.

@djmattyg007

@djmattyg007

That doesn't sound like

(Emphasis added.) Did you actually try it? It disables precisely the warning you don't care about and only in the files where you don't care about it (your test files). My config was just an example. You can tailor yours to your needs. https://mypy.readthedocs.io/en/stable/config_file.html

I don't want to disable those settings on a per-file basis. I want to disable them on a per-function basis.

TypeScript's inference is really great in this regard, with strict mode enabled you don't have to annotated the return types of functions. I would love some level of support for this in mypy.

It may be obvious to most humans that f should be marked as returning str, but it would be more precise to say it returns Literal["hello"]. Maybe a slightly more complex example could be said to return an enum of possible string literals rather than any arbitrary str. It is also not incorrect to say that f returns Any or object. Automatically inferring the return type could be problematic when the developer intentions are unclear.

@ryanc-bs

@ryanc-bs

It may be obvious to most humans that f should be marked as returning str, but it would be more precise to say it returns Literal["hello"]

The following definitions can be considered as equal:

def f():
    return "hello"

def f():
    x = "hello"
    reveal_type(x) #  Revealed type is "builtins.str"
    return x

mypy already infers type for local variable, and I don't see why rule for inferring return type should be different. For comparison, in both cases pyright infers return type of f as Literal["hello"] (see https://microsoft.github.io/pyright/#/type-inference?id=return-type-inference and https://microsoft.github.io/pyright/#/type-inference?id=literals).

Having to write -> None on every function that can only return None because it has no explicit return statements is quite annoying.

In general, if MyPy can infer the return type accurately, then it should. There is no benefit to not doing so.
It doesn't have to report the error, if not in --strict mode, but there seem no reason not to use the information that MyPy already knows.

For example:

def x():
     return 1

def y() -> str:
    return "hello " + x()

The above code is clearly wrong, yet without --strict, x() has the type Any and "hello " + x() isn't checked.
With --strict, MyPy complains that x() is not annotated, which is a distraction from the actual error.

In general, if MyPy can infer the return type accurately, then it should. There is no benefit to not doing so.

There probably is a benefit: performance. I'd assume inferring the return types where missing requires a bit of extra work, because currently mypy can just skip checking all returns and their types, which should be faster.

But nonetheless it would be a very nice feature. If performance is a factor it could simply be made an opt-in feature.

Ok I think I am going to sit down and commit time to work on an implementation of this. It's a prerequisite to #6646 IMO, and I think fixing that is very important.

I think I will try an implementation with the full inference engine, but I also want to try a more minimal approach, as I think it is less risky.

I was gonna ask for some sort of no-untyped-def check (disallow_incomplete_defs or disallow_untyped_defs) that only raises for methods that mypy won't analyze (ie: 0 non-self parameters w/ missing return type, or missing parameter annotations).

But this request would alleviate some of the pain with disallow_incomplete/untyped_defs and reduce a lot of Function is missing a return type annotation

I think one potential problem that might come up with adding a configuration option for turning this on is that this changes how a given type signature is interpreted by mypy, unlike most of the other configuration options, so if a library you were using turned that option on while you had it off or you turned it on while the library turned it off then mypy would do something that wasn't intended. In the former case, mypy would silently infer an Any return type if the library authors were relying on the return type being inferred, while in the latter case mypy would, if I'm understanding the proposal correctly, start emitting errors relating to that inferred return type being used incorrectly, even if the library authors intentionally avoided annotating that since they were relying on it getting inferred as Any.

In my experience, I've definitely run into errors similar to the latter case with pyright, since (if I recall correctly) I usually have reasonably strict settings turned on while I sometimes use libraries without type annotations, and libraries that have complicated return types tend to result in pyright inferring a return type that leads to a false positive error. To some extent that's an understandable design decision since someone turning on stricter settings might prefer false positives over false negatives, but I think it's a risk to keep in mind here.

As far as I can tell, the only precedent for a config option that changes how types are interpreted by mypy is implicit_optional, and I think the historical context behind that makes it reasonably different than this case, since if you and a library disagree on whether to set that flag, there is a "right" option, while in the case of this option, libraries might be expected to support both it being set and unset, which would end up with them getting the worst of both worlds for their external API at least (you'd need to add return types for functions with obvious return types to make sure mypy/other type checkers inferred them correctly, but you'd also probably want to annotate non-obvious return types with Any explicitly so that mypy didn't infer the wrong thing).

This proposal does not require the full inference of the return type. Only the obvious one in simple cases, like if a function does not have a return statement, infer it as -> None, or if it has only one return statement, that's the return type of the function.

@eternal-sorrow

@eternal-sorrow

or if it has only one return statement,

It looks like artificial limitation. For example, in the following example

def func1(val: int):
    if val > 3:
        return ""
    elif val < 1:
        return True

return type is str | bool | None. It seems fairly obvious. Pyright and TypeScript can handle such cases.

@ikokostya

@ikokostya while this function is similar to the one I have described at the beginning of this issue, I am against inferring its type. This feature should be about (a) performing zero computation and (b) having the return expression itself serve as an explicit annotation.

Navigation Menu

Repository navigation

Proposal: treat "obvious" return type as annotated #4409

22 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Proposal: treat "obvious" return type as annotated #4409

Description

Activity

refi64 commented on Dec 25, 2017

emmatyping commented on Dec 25, 2017

elazarg commented on Dec 25, 2017

elazarg commented on Dec 25, 2017

gvanrossum commented on Dec 25, 2017

elazarg commented on Dec 25, 2017

emmatyping commented on Dec 25, 2017

elazarg commented on Dec 25, 2017

refi64 commented on Dec 25, 2017

elazarg commented on Dec 25, 2017

elazarg commented on Dec 26, 2017

ilevkivskyi commented on Dec 26, 2017

elazarg commented on Dec 27, 2017

ilevkivskyi commented on Dec 28, 2017

elazarg commented on Dec 28, 2017

22 remaining items

djmattyg007 commented on Jan 20, 2022

kstauffer commented on Jan 20, 2022

djmattyg007 commented on Jan 23, 2022

michaeloliverx commented on Jul 8, 2022

ryanc-bs commented on Nov 8, 2023

ikokostya commented on Nov 8, 2023

markshannon commented on Dec 1, 2023

bluenote10 commented on Dec 1, 2023

emmatyping commented on Apr 29, 2024

Avasam commented on May 23, 2024

pranavrajpal commented on Jul 8, 2024

eternal-sorrow commented on Jul 8, 2024

ikokostya commented on Jul 8, 2024

elazarg commented on Jul 9, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions