Skip to content

Allow nested classes in NamedTuple bodies - #15776

Open
sobolevn wants to merge 6 commits into
masterfrom
issue-15775
Open

sobolevn wants to merge 6 commits into
masterfrom
issue-15775

Conversation

@sobolevn

@sobolevn sobolevn commented Jul 29, 2023 •

Copy link
Copy Markdown
Member

Closes #15775
Closes #15752
Closes #15064
Refs #15680

I've taken test cases from #15064
Thanks a lot to @hmc-cs-mdrissi for working on it!

@github-actions

This comment has been minimized.

@sobolevn

Copy link
Copy Markdown
Member Author

This does not look good, fixing.

@github-actions

This comment has been minimized.

@sobolevn
sobolevn requested a review from ilevkivskyi July 29, 2023 14:24
@github-actions

This comment has been minimized.

1 similar comment
@github-actions

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

sobolevn added a commit that referenced this pull request Jul 29, 2023
While working on #15776 I've noticed
that some `RuntimeError` do not have enough metadata to understand what
is going on.
CI:
https://github.com/python/mypy/actions/runs/5700479199/job/15450345887

This PR adds more context to error messages.

@hauntsaninja hauntsaninja left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@sobolevn

Copy link
Copy Markdown
Member Author

I want to hear from @ilevkivskyi as well, because of recent #14119

Thanks, @hauntsaninja, for the review :)

@sobolevn

Copy link
Copy Markdown
Member Author

After reading feedback in #15064 I realize that this needs some more work.

@sobolevn

Copy link
Copy Markdown
Member Author

This is going to be much harder than I expected :)

@sobolevn

Copy link
Copy Markdown
Member Author

Solution suggested by @JukkaL gave this idea: we can split the semanal passes over named tuple class definition. I think the best option is:

  1. Analyze nested classes
  2. Analyze fields and methods
  3. Re-analyze everything

@sobolevn

Copy link
Copy Markdown
Member Author

Well, to do this I had to refactor almost all logic related to NamedTuple class definitions.
I don't really think that it is worth it right now :(

@sobolevn

Copy link
Copy Markdown
Member Author

I think that this PR can be reviewed separately: it still allow defining a class inside a NamedTuple, but it does not allow using this class in annotations, which is very rare.

@sobolevn

Copy link
Copy Markdown
Member Author

This is the example of how much changes are needed (just a single method):

def analyze_namedtuple_classdef(
        self, defn: ClassDef, tvar_defs: list[TypeVarLikeType]
    ) -> bool:
        def analyze_class_internal(
            defn: ClassDef, info: TypeInfo | None = None, *, first_pass: bool = False
        ) -> None:
            if first_pass:
                self.prepare_class_def(defn, info=info, custom_names=True)
            self.setup_type_vars(defn, tvar_defs)
            self.setup_alias_type_vars(defn)
            with self.scope.class_scope(defn.info):
                if first_pass:
                    for deco in defn.decorators:
                        self.analyze_class_decorator(defn, deco)
                self.analyze_class_body_common(defn)

        if defn.namedtuple_body_semanal is None:
            info = self.named_tuple_analyzer.analyze_namedtuple_classdef_first_pass(
                defn, self.is_func_scope()
            )
            analyze_class_internal(defn, info, first_pass=True)

        # Second pass with actual fields:
        self.enter_class(defn.info)
        is_incomplete = self.named_tuple_analyzer.analyze_namedtuple_classdef(defn)
        self.leave_class()
        if is_incomplete:
            self.defer(defn)
            return

        with self.named_tuple_analyzer.save_namedtuple_body(info):
            analyze_class_internal(defn)
        defn.namedtuple_body_semanal = None

@ilevkivskyi ilevkivskyi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I can almost guarantee this will lead to crashes, but it is also hard to predict in which exact scenarios. My only suggestion would be to add a bunch of coarse grained and fine grained incremental tests, where these nested classes are added/modified (e.g. made generic)/deleted/renamed/replaced with a non-class with same name etc.

But also a more broad question: what is the value of supporting this? Are people asking for this often? (This is an honest question, e.g. a lot of people are asking for nested TypedDicts support)

@hmc-cs-mdrissi

hmc-cs-mdrissi commented Jul 31, 2023 •

Copy link
Copy Markdown
Contributor

My primary motivation for wanting this was code like this. Using older style namedtuple it is possible to define nested namedtuple and have code that becomes unclear how to write type stubs for. Maybe it would be fine from typing perspective to define class outside but to allow defining type alias to original nested class to allow any existing code that referenced nested class to continue to work.

This is only place I have ever come across nested namedtuples.

@ilevkivskyi

Copy link
Copy Markdown
Member

OK, I see, this use case looks similar to nested TypeDicts.

The problem with type aliases is that supporting type aliases defined inside NamedTuples may be ~same amount of work @sobolevn mentioned to allow nested classes to be usable in annotations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NamedTuples cannot have nested classes [stubgen] Nested classes in NamedTuples are not produced

4 participants