Skip to content

Inference from unannotated functions #4455

Open

Description

@elazarg

Mypy uses the body of unannotated functions for inference and to understand what attributes an object has. For example:

d = {}
e = {}

def f():
    d['x'] = 1

d['y'] = 2
e['y'] = 2

reveal_type(d)  # dict[Any, Any]
reveal_type(e)  # dict[str, int]

I couldn't find any mention of it on the docs. I think this behaviour needs to be documented. Perhaps as a separate section, to make it searchable.

Related: #4434 #4409

Activity

  1. ilevkivskyi commented on Jan 10, 2018

    @ilevkivskyi
    Member

    I would actually say this is a bug. We shouldn't use Any types (even explicit probably) for partial types, and require an annotation instead (if there are no other assignments in scope). This can lead to false negatives:

    d = {}  # We should require an annotation here, if the second assignment below is absent...
    
    def g():
        d['x'] = 1
    
    d[1] = 'x'  # ...instead of inferring 'Dict[Any, Any]' that may be unsafe.
  2. changed the title [-]Document inference from unannotated functions[/-] [+]Inference from unannotated functions[/+] on Jan 10, 2018
  3. elazarg commented on Jan 10, 2018

    @elazarg
    ContributorAuthor

    IMO it is unclear that the error is at the last line. I would expect that line to define the type, and the error to be inside the function, since it is unannotated and therefore assumed to be too clever for the type checker.

    I am confused though. I agree that we shouldn't use (implicit) Any for partial types, but from the discussion on #4434 I understood that this is by design.

    This issue was meant to be more general: I am trying to understand how things should behave. What are the rules.

  4. ilevkivskyi commented on Jan 11, 2018

    @ilevkivskyi
    Member

    IMO it is unclear that the error is at the last line.

    I was probably not very clear in my example.

    At least one of the two assignments is probably erroneous. But it would be safer to use the second for inference. The only downside of this is that after annotating the function, an error will appear in previously valid code. But this is better than the current (clearly unsafe) behavior. In case if there is no second assignment we should give a normal Need type annotation for variable error.

    I am confused though. I agree that we shouldn't use (implicit) Any for partial types, but from the discussion on #4434 I understood that this is by design.

    I don't think current behavior is really intentional, @JukkaL may correct me.

  5. JukkaL commented on Jan 11, 2018

    @JukkaL
    Collaborator

    The current behavior seems accidental or at least not very well thought out. I think that it would be better to require all parts of a partial type definition to be located within a single scope. So this shouldn't be valid:

    d = {}  # We should require an annotation here
    
    def f():
        # This is a different scope from the initial assignment so can't affect the partial type
        d['x'] = 1

    In the original example, we should probably use the second assignment for inference (d['y'] = 2). There probably shouldn't be an error, since the function is unannotated and we can just give d type Any within the function, independent of whether it has a partial type or not. If the function is annotated, we can defer the function and re-check it after we've finished with the module top level. This would conform to how fine-grained incremental works, I think.

  6. elazarg commented on Jan 11, 2018

    @elazarg
    ContributorAuthor

    Jukka, I agree. So - what does mypy do inside unannotated functions?

  7. JukkaL commented on Jan 11, 2018

    @JukkaL
    Collaborator

    Here are some things (probably incomplete):

    • Mypy infers the existence of attributes through self assignments.
    • Mypy follows imports within unannotated functions.
    • Mypy does some reachability analysis (sys.version_info checks, for example), but this seems inconsistent.
    • Does some analysis of nested, annotated functions (again this seems inconsistent).
    • As discussed above, it does some partial type related analysis, but this should probably be removed.

    Much of this isn't very principled, to be honest.

  8. elazarg commented on Jan 11, 2018

    @elazarg
    ContributorAuthor

    Seems like the not-inconsistent part is straight-forward semantic analysis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions