Skip to content

New semantic analyzer: conditional function definitions - #6429

Merged
JukkaL merged 10 commits into
masterfrom
semanal-cond-func
Feb 19, 2019
Merged

JukkaL merged 10 commits into
masterfrom
semanal-cond-func

Conversation

@JukkaL

@JukkaL JukkaL commented Feb 18, 2019

Copy link
Copy Markdown
Collaborator

To semantically analyze conditional function definitions (and function
redefinitions) we add redefinitions to a symbol table with a mangled
name of form name-redefinition[N]. This is necessary since
functions are only analyzed if they can be accessed through a symbol
table.

I checked that this fixes most relevant test cases in
check-functions.test. Using a decorated function in a redefinition
is still not working perfectly. That is also partially implemented in
the old semantic analyzer. Conditionally defining an overloaded
function is also not supported. I'll create follow-up issues for these
remaining known limitations.

Fixes #6318.

@JukkaL
JukkaL requested a review from ilevkivskyi February 18, 2019 18:18

@ilevkivskyi ilevkivskyi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this tricky part! I have few comments.

# Add module-level function to symbol table on first iteration
# (recurse_into_functions == False). Also add nested functions
# always, since they won't be processed as separate targets.
if not self.recurse_into_functions or len(self.function_stack) > 0:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This explanation is not very clear. I would rather say that we don't add function top-level functions when we just enter each of it during second phase of analysis -- processing of functions.

Also I would rather explain why this is needed (I agree that this looks reasonable).

from m import f
else:
@dec
def f(x: int) -> None:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a test case for when this causes a type error? (Decorated function signature differs from original one.)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work correctly with the old semantic analyzer either. I'll create an issue about this.

1() # E: "int" not callable
reveal_type(f) # E: Revealed type is 'def (x: builtins.str)'
[file m.py]
def f(x: str) -> None: pass

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a test that a function with same name as a class is still semantically analyzed. In general, do we have enough tests for something like:

x = 1
class C:
    pass
class C:
    x + 'no way'

etc?

else:
def f(x: str) -> None:
...
reveal_type(f) # E: Revealed type is 'def (builtins.str)'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does old semantic analyzer support the opposite order? I mean:

if int():
    def f(x: str) -> None:
        ...
else:
      f: Callable[[str], None]

If yes, then the new one should probably support this too. Otherwise, this is probably not important.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This generates an error with the old semantic analyzer as well.

@ilevkivskyi ilevkivskyi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updates!

@JukkaL
JukkaL merged commit ffd8e89 into master Feb 19, 2019
@JelleZijlstra
JelleZijlstra deleted the semanal-cond-func branch February 19, 2019 18:07
ilevkivskyi added a commit that referenced this pull request Apr 17, 2019
… or classes (#6681)

Fixes #6432

This is a hot fix that simply extends the solution in PR #6429 to more situations. We agreed with @JukkaL that he will clean this up when he is back from PTO.

Note that we still give the errors for redefinitions that we consider invalid (see tests), but we don't crash now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New semantic analyzer: conditionally defined functions

2 participants