Skip to content

mypy 0.990 is 1000x slow on pydantic codebase vs 0.982 #14034

Closed

Description

@samuelcolvin

Bug Report

Running mypy on the current pydantic codebase (specifically the code as of pydantic/pydantic#4735), give the following run times:

  • running mypy pydantic with mypy 0.982, and no .mypy_cache: 8s
  • running mypy pydantic with mypy 0.982, .mypy_cache present: 0.17s
  • running mypy pydantic with mypy 0.990, and no .mypy_cache: 3m19s
  • running mypy pydantic with mypy 0.990, .mypy_cache present: 3m6s

(1000x comes from (3 * 60 + 6) / 0.17)

There are quite a few errors, mostly due to no_implicit_optional=True, but still - this seems like a significant regression.

While mypy 0.990 is running, I see 100% usage on 1 CPU.

To Reproduce

Should be as simple as cloning pydantic, checkout that branch (uprev-mypy), install requirements from requirements/linting.txt, run mypy.

Your Environment

  • Mypy version used: 0.990
  • Mypy command-line flags: just mypy pydantic
  • Mypy configuration options from mypy.ini (and other config files): see this bit of pyproject.toml
  • Python version used:3.10.0 (just realised how old my version of 3.10 is, I've upgraded to 3.10.8, same result)

Activity

  1. samuelcolvin commented on Nov 8, 2022

    @samuelcolvin
    SponsorContributorAuthor

    As with #14027, would it be worth adding pydantic to mypy_primer @hauntsaninja?

    As with sqlalachemy, pydantic pushes python's type system fairly hard, in particular, the schema for pydantic-core, is defined as a recursive union of TypedDicts.

  2. ilevkivskyi commented on Nov 8, 2022

    @ilevkivskyi
    Member

    Note that when an error is present in a file, mypy will not use cache for that file.

  3. ilevkivskyi commented on Nov 8, 2022

    @ilevkivskyi
    Member

    Btw can you try --disable-recursive-aliases? Such big recursive types are known to be slow.

  4. samuelcolvin commented on Nov 8, 2022

    @samuelcolvin
    SponsorContributorAuthor

    Yup, that makes a big different, time reduces to 9s.

  5. ilevkivskyi commented on Nov 8, 2022

    @ilevkivskyi
    Member

    Also, just to be sure, can you also try --enable-recursive-aliases on 0.982?

  6. samuelcolvin commented on Nov 8, 2022

    @samuelcolvin
    SponsorContributorAuthor

    Takes 261s as you predicted - seems that's the problem.

  7. ilevkivskyi commented on Nov 8, 2022

    @ilevkivskyi
    Member

    OK, then this will be not easy to fix. The problem is that you have that big union of 40 items. Previously mypy didn't support recursive types, so most likely all recursive types were either plain Any or shallow TypedDicts with Any. Mypy used to give an error about this however. I guess this error was not shown because pydantic_core is checked as third party dependency? You can try --no-silence-site-packages when using --disable-recursive-aliases. (Btw everything on PYTHONPATH is considered third party, cc @JukkaL)

    So now when recursive types are properly supported imagine you have a triple nested dictionary typed as CoreSchema. Mypy will need to try every item in the union until it finds a matching one. This will take on average 20 * 20 * 20 = 8000 type checks for a single dict. To make things worse, TypedDicts are structural, and some have overlapping keys so it may take a while to figure out that a given item doesn't actually match.

    We will still try to profile this. Maybe there is some room for optimizations.

  8. samuelcolvin commented on Nov 8, 2022

    @samuelcolvin
    SponsorContributorAuthor

    Thanks so much for the explanation.

    Simple answer might be to move to pyright, which we already use to check pydantic-core and seems to do a reasonably good job (although that union found a bug in pyright too). (I don't mean this to sound hostile, mypy is amazing and I'd love to continue to use it, but I understand if this is a fundamental limitation of the library)

    If it helps, all those TypedDicts have a type key which can (mostly) be used as a discriminator - that's how we validate pydantic-core schema using pydantic-core, see here. Using that discriminator, the number of types checked could be massively reduced, but I guess that's too specific for mypy to be able to pick up on.

  9. ilevkivskyi commented on Nov 8, 2022

    @ilevkivskyi
    Member

    Btw if you fix the errors coming from no_implicit_optional=True the cache should became effective again. I guess even with recursive types enabled, a typical run should take just few seconds with warm cache.

  10. samuelcolvin commented on Nov 8, 2022

    @samuelcolvin
    SponsorContributorAuthor

    ye, I thought that, but it's still not really acceptable for either CI or new contributors for mypy to take minutes.

    I guess we'll have to use --disable-recursive-aliases.

  11. hauntsaninja commented on Nov 8, 2022

    @hauntsaninja
    Collaborator

    Thanks, pydantic is already in mypy_primer :-) However, we don't currently monitor perf regressions in mypy_primer... it's a feature I've been thinking about though

  12. adriangb commented on Mar 15, 2023

    @adriangb
    Contributor

    I did some testing and came up with a self-contained example that may be helpful.

    It seems that both the size of the union and how many times SchemaXX() gets called both matter. Using non-overlapping keys (e.g. type_99) doesn't seem to matter. Removing the recursive references does indeed improve things.

    Long code block
    from typing import Union
    from typing_extensions import TypedDict, Literal, NotRequired
    
    
    class Schema0(TypedDict):
        type: Literal['schema-0']
        schema: NotRequired['Schema']
    
    
    class Schema1(TypedDict):
        type: Literal['schema-1']
        schema: NotRequired['Schema']
    
    
    class Schema2(TypedDict):
        type: Literal['schema-2']
        schema: NotRequired['Schema']
    
    
    class Schema3(TypedDict):
        type: Literal['schema-3']
        schema: NotRequired['Schema']
    
    
    class Schema4(TypedDict):
        type: Literal['schema-4']
        schema: NotRequired['Schema']
    
    
    class Schema5(TypedDict):
        type: Literal['schema-5']
        schema: NotRequired['Schema']
    
    
    class Schema6(TypedDict):
        type: Literal['schema-6']
        schema: NotRequired['Schema']
    
    
    class Schema7(TypedDict):
        type: Literal['schema-7']
        schema: NotRequired['Schema']
    
    
    class Schema8(TypedDict):
        type: Literal['schema-8']
        schema: NotRequired['Schema']
    
    
    class Schema9(TypedDict):
        type: Literal['schema-9']
        schema: NotRequired['Schema']
    
    
    class Schema10(TypedDict):
        type: Literal['schema-10']
        schema: NotRequired['Schema']
    
    
    class Schema11(TypedDict):
        type: Literal['schema-11']
        schema: NotRequired['Schema']
    
    
    class Schema12(TypedDict):
        type: Literal['schema-12']
        schema: NotRequired['Schema']
    
    
    class Schema13(TypedDict):
        type: Literal['schema-13']
        schema: NotRequired['Schema']
    
    
    class Schema14(TypedDict):
        type: Literal['schema-14']
        schema: NotRequired['Schema']
    
    
    class Schema15(TypedDict):
        type: Literal['schema-15']
        schema: NotRequired['Schema']
    
    
    class Schema16(TypedDict):
        type: Literal['schema-16']
        schema: NotRequired['Schema']
    
    
    class Schema17(TypedDict):
        type: Literal['schema-17']
        schema: NotRequired['Schema']
    
    
    class Schema18(TypedDict):
        type: Literal['schema-18']
        schema: NotRequired['Schema']
    
    
    class Schema19(TypedDict):
        type: Literal['schema-19']
        schema: NotRequired['Schema']
    
    
    class Schema20(TypedDict):
        type: Literal['schema-20']
        schema: NotRequired['Schema']
    
    
    class Schema21(TypedDict):
        type: Literal['schema-21']
        schema: NotRequired['Schema']
    
    
    class Schema22(TypedDict):
        type: Literal['schema-22']
        schema: NotRequired['Schema']
    
    
    class Schema23(TypedDict):
        type: Literal['schema-23']
        schema: NotRequired['Schema']
    
    
    class Schema24(TypedDict):
        type: Literal['schema-24']
        schema: NotRequired['Schema']
    
    
    class Schema25(TypedDict):
        type: Literal['schema-25']
        schema: NotRequired['Schema']
    
    
    class Schema26(TypedDict):
        type: Literal['schema-26']
        schema: NotRequired['Schema']
    
    
    class Schema27(TypedDict):
        type: Literal['schema-27']
        schema: NotRequired['Schema']
    
    
    class Schema28(TypedDict):
        type: Literal['schema-28']
        schema: NotRequired['Schema']
    
    
    class Schema29(TypedDict):
        type: Literal['schema-29']
        schema: NotRequired['Schema']
    
    
    class Schema30(TypedDict):
        type: Literal['schema-30']
        schema: NotRequired['Schema']
    
    
    class Schema31(TypedDict):
        type: Literal['schema-31']
        schema: NotRequired['Schema']
    
    
    class Schema32(TypedDict):
        type: Literal['schema-32']
        schema: NotRequired['Schema']
    
    
    class Schema33(TypedDict):
        type: Literal['schema-33']
        schema: NotRequired['Schema']
    
    
    class Schema34(TypedDict):
        type: Literal['schema-34']
        schema: NotRequired['Schema']
    
    
    class Schema35(TypedDict):
        type: Literal['schema-35']
        schema: NotRequired['Schema']
    
    
    class Schema36(TypedDict):
        type: Literal['schema-36']
        schema: NotRequired['Schema']
    
    
    class Schema37(TypedDict):
        type: Literal['schema-37']
        schema: NotRequired['Schema']
    
    
    class Schema38(TypedDict):
        type: Literal['schema-38']
        schema: NotRequired['Schema']
    
    
    class Schema39(TypedDict):
        type: Literal['schema-39']
        schema: NotRequired['Schema']
    
    
    class Schema40(TypedDict):
        type: Literal['schema-40']
        schema: NotRequired['Schema']
    
    
    class Schema41(TypedDict):
        type: Literal['schema-41']
        schema: NotRequired['Schema']
    
    
    class Schema42(TypedDict):
        type: Literal['schema-42']
        schema: NotRequired['Schema']
    
    
    class Schema43(TypedDict):
        type: Literal['schema-43']
        schema: NotRequired['Schema']
    
    
    class Schema44(TypedDict):
        type: Literal['schema-44']
        schema: NotRequired['Schema']
    
    
    class Schema45(TypedDict):
        type: Literal['schema-45']
        schema: NotRequired['Schema']
    
    
    class Schema46(TypedDict):
        type: Literal['schema-46']
        schema: NotRequired['Schema']
    
    
    class Schema47(TypedDict):
        type: Literal['schema-47']
        schema: NotRequired['Schema']
    
    
    class Schema48(TypedDict):
        type: Literal['schema-48']
        schema: NotRequired['Schema']
    
    
    class Schema49(TypedDict):
        type: Literal['schema-49']
        schema: NotRequired['Schema']
    
    
    class Schema50(TypedDict):
        type: Literal['schema-50']
        schema: NotRequired['Schema']
    
    
    class Schema51(TypedDict):
        type: Literal['schema-51']
        schema: NotRequired['Schema']
    
    
    class Schema52(TypedDict):
        type: Literal['schema-52']
        schema: NotRequired['Schema']
    
    
    class Schema53(TypedDict):
        type: Literal['schema-53']
        schema: NotRequired['Schema']
    
    
    class Schema54(TypedDict):
        type: Literal['schema-54']
        schema: NotRequired['Schema']
    
    
    class Schema55(TypedDict):
        type: Literal['schema-55']
        schema: NotRequired['Schema']
    
    
    class Schema56(TypedDict):
        type: Literal['schema-56']
        schema: NotRequired['Schema']
    
    
    class Schema57(TypedDict):
        type: Literal['schema-57']
        schema: NotRequired['Schema']
    
    
    class Schema58(TypedDict):
        type: Literal['schema-58']
        schema: NotRequired['Schema']
    
    
    class Schema59(TypedDict):
        type: Literal['schema-59']
        schema: NotRequired['Schema']
    
    
    class Schema60(TypedDict):
        type: Literal['schema-60']
        schema: NotRequired['Schema']
    
    
    class Schema61(TypedDict):
        type: Literal['schema-61']
        schema: NotRequired['Schema']
    
    
    class Schema62(TypedDict):
        type: Literal['schema-62']
        schema: NotRequired['Schema']
    
    
    class Schema63(TypedDict):
        type: Literal['schema-63']
        schema: NotRequired['Schema']
    
    
    class Schema64(TypedDict):
        type: Literal['schema-64']
        schema: NotRequired['Schema']
    
    
    class Schema65(TypedDict):
        type: Literal['schema-65']
        schema: NotRequired['Schema']
    
    
    class Schema66(TypedDict):
        type: Literal['schema-66']
        schema: NotRequired['Schema']
    
    
    class Schema67(TypedDict):
        type: Literal['schema-67']
        schema: NotRequired['Schema']
    
    
    class Schema68(TypedDict):
        type: Literal['schema-68']
        schema: NotRequired['Schema']
    
    
    class Schema69(TypedDict):
        type: Literal['schema-69']
        schema: NotRequired['Schema']
    
    
    class Schema70(TypedDict):
        type: Literal['schema-70']
        schema: NotRequired['Schema']
    
    
    class Schema71(TypedDict):
        type: Literal['schema-71']
        schema: NotRequired['Schema']
    
    
    class Schema72(TypedDict):
        type: Literal['schema-72']
        schema: NotRequired['Schema']
    
    
    class Schema73(TypedDict):
        type: Literal['schema-73']
        schema: NotRequired['Schema']
    
    
    class Schema74(TypedDict):
        type: Literal['schema-74']
        schema: NotRequired['Schema']
    
    
    class Schema75(TypedDict):
        type: Literal['schema-75']
        schema: NotRequired['Schema']
    
    
    class Schema76(TypedDict):
        type: Literal['schema-76']
        schema: NotRequired['Schema']
    
    
    class Schema77(TypedDict):
        type: Literal['schema-77']
        schema: NotRequired['Schema']
    
    
    class Schema78(TypedDict):
        type: Literal['schema-78']
        schema: NotRequired['Schema']
    
    
    class Schema79(TypedDict):
        type: Literal['schema-79']
        schema: NotRequired['Schema']
    
    
    class Schema80(TypedDict):
        type: Literal['schema-80']
        schema: NotRequired['Schema']
    
    
    class Schema81(TypedDict):
        type: Literal['schema-81']
        schema: NotRequired['Schema']
    
    
    class Schema82(TypedDict):
        type: Literal['schema-82']
        schema: NotRequired['Schema']
    
    
    class Schema83(TypedDict):
        type: Literal['schema-83']
        schema: NotRequired['Schema']
    
    
    class Schema84(TypedDict):
        type: Literal['schema-84']
        schema: NotRequired['Schema']
    
    
    class Schema85(TypedDict):
        type: Literal['schema-85']
        schema: NotRequired['Schema']
    
    
    class Schema86(TypedDict):
        type: Literal['schema-86']
        schema: NotRequired['Schema']
    
    
    class Schema87(TypedDict):
        type: Literal['schema-87']
        schema: NotRequired['Schema']
    
    
    class Schema88(TypedDict):
        type: Literal['schema-88']
        schema: NotRequired['Schema']
    
    
    class Schema89(TypedDict):
        type: Literal['schema-89']
        schema: NotRequired['Schema']
    
    
    class Schema90(TypedDict):
        type: Literal['schema-90']
        schema: NotRequired['Schema']
    
    
    class Schema91(TypedDict):
        type: Literal['schema-91']
        schema: NotRequired['Schema']
    
    
    class Schema92(TypedDict):
        type: Literal['schema-92']
        schema: NotRequired['Schema']
    
    
    class Schema93(TypedDict):
        type: Literal['schema-93']
        schema: NotRequired['Schema']
    
    
    class Schema94(TypedDict):
        type: Literal['schema-94']
        schema: NotRequired['Schema']
    
    
    class Schema95(TypedDict):
        type: Literal['schema-95']
        schema: NotRequired['Schema']
    
    
    class Schema96(TypedDict):
        type: Literal['schema-96']
        schema: NotRequired['Schema']
    
    
    class Schema97(TypedDict):
        type: Literal['schema-97']
        schema: NotRequired['Schema']
    
    
    class Schema98(TypedDict):
        type: Literal['schema-98']
        schema: NotRequired['Schema']
    
    
    class Schema99(TypedDict):
        type: Literal['schema-99']
        schema: NotRequired['Schema']
    
    
    Schema = Union[
        Schema0,
        Schema1,
        Schema2,
        Schema3,
        Schema4,
        Schema5,
        Schema6,
        Schema7,
        Schema8,
        Schema9,
        Schema10,
        Schema11,
        Schema12,
        Schema13,
        Schema14,
        Schema15,
        Schema16,
        Schema17,
        Schema18,
        Schema19,
        Schema20,
        Schema21,
        Schema22,
        Schema23,
        Schema24,
        Schema25,
        Schema26,
        Schema27,
        Schema28,
        Schema29,
        Schema30,
        Schema31,
        Schema32,
        Schema33,
        Schema34,
        Schema35,
        Schema36,
        Schema37,
        Schema38,
        Schema39,
        Schema40,
        Schema41,
        Schema42,
        Schema43,
        Schema44,
        Schema45,
        Schema46,
        Schema47,
        Schema48,
        Schema49,
        Schema50,
        Schema51,
        Schema52,
        Schema53,
        Schema54,
        Schema55,
        Schema56,
        Schema57,
        Schema58,
        Schema59,
        Schema60,
        Schema61,
        Schema62,
        Schema63,
        Schema64,
        Schema65,
        Schema66,
        Schema67,
        Schema68,
        Schema69,
        Schema70,
        Schema71,
        Schema72,
        Schema73,
        Schema74,
        Schema75,
        Schema76,
        Schema77,
        Schema78,
        Schema79,
        Schema80,
        Schema81,
        Schema82,
        Schema83,
        Schema84,
        Schema85,
        Schema86,
        Schema87,
        Schema88,
        Schema89,
        Schema90,
        Schema91,
        Schema92,
        Schema93,
        Schema94,
        Schema95,
        Schema96,
        Schema97,
        Schema98,
        Schema99,
    ]
    
    schema: Schema = {'type': 'schema-0'}
    Schema0(type='schema-0', schema=schema)
    Schema1(type='schema-1', schema=schema)
    Schema2(type='schema-2', schema=schema)
    Schema3(type='schema-3', schema=schema)
    Schema4(type='schema-4', schema=schema)
    Schema5(type='schema-5', schema=schema)
    Schema6(type='schema-6', schema=schema)
    Schema7(type='schema-7', schema=schema)
    Schema8(type='schema-8', schema=schema)
    Schema9(type='schema-9', schema=schema)
    Schema10(type='schema-10', schema=schema)
    Schema11(type='schema-11', schema=schema)
    Schema12(type='schema-12', schema=schema)
    Schema13(type='schema-13', schema=schema)
    Schema14(type='schema-14', schema=schema)
    Schema15(type='schema-15', schema=schema)
    Schema16(type='schema-16', schema=schema)
    Schema17(type='schema-17', schema=schema)
    Schema18(type='schema-18', schema=schema)
    Schema19(type='schema-19', schema=schema)
    Schema20(type='schema-20', schema=schema)
    Schema21(type='schema-21', schema=schema)
    Schema22(type='schema-22', schema=schema)
    Schema23(type='schema-23', schema=schema)
    Schema24(type='schema-24', schema=schema)
    Schema25(type='schema-25', schema=schema)
    Schema26(type='schema-26', schema=schema)
    Schema27(type='schema-27', schema=schema)
    Schema28(type='schema-28', schema=schema)
    Schema29(type='schema-29', schema=schema)
    Schema30(type='schema-30', schema=schema)
    Schema31(type='schema-31', schema=schema)
    Schema32(type='schema-32', schema=schema)
    Schema33(type='schema-33', schema=schema)
    Schema34(type='schema-34', schema=schema)
    Schema35(type='schema-35', schema=schema)
    Schema36(type='schema-36', schema=schema)
    Schema37(type='schema-37', schema=schema)
    Schema38(type='schema-38', schema=schema)
    Schema39(type='schema-39', schema=schema)
    Schema40(type='schema-40', schema=schema)
    Schema41(type='schema-41', schema=schema)
    Schema42(type='schema-42', schema=schema)
    Schema43(type='schema-43', schema=schema)
    Schema44(type='schema-44', schema=schema)
    Schema45(type='schema-45', schema=schema)
    Schema46(type='schema-46', schema=schema)
    Schema47(type='schema-47', schema=schema)
    Schema48(type='schema-48', schema=schema)
    Schema49(type='schema-49', schema=schema)
    Schema50(type='schema-50', schema=schema)
    Schema51(type='schema-51', schema=schema)
    Schema52(type='schema-52', schema=schema)
    Schema53(type='schema-53', schema=schema)
    Schema54(type='schema-54', schema=schema)
    Schema55(type='schema-55', schema=schema)
    Schema56(type='schema-56', schema=schema)
    Schema57(type='schema-57', schema=schema)
    Schema58(type='schema-58', schema=schema)
    Schema59(type='schema-59', schema=schema)
    Schema60(type='schema-60', schema=schema)
    Schema61(type='schema-61', schema=schema)
    Schema62(type='schema-62', schema=schema)
    Schema63(type='schema-63', schema=schema)
    Schema64(type='schema-64', schema=schema)
    Schema65(type='schema-65', schema=schema)
    Schema66(type='schema-66', schema=schema)
    Schema67(type='schema-67', schema=schema)
    Schema68(type='schema-68', schema=schema)
    Schema69(type='schema-69', schema=schema)
    Schema70(type='schema-70', schema=schema)
    Schema71(type='schema-71', schema=schema)
    Schema72(type='schema-72', schema=schema)
    Schema73(type='schema-73', schema=schema)
    Schema74(type='schema-74', schema=schema)
    Schema75(type='schema-75', schema=schema)
    Schema76(type='schema-76', schema=schema)
    Schema77(type='schema-77', schema=schema)
    Schema78(type='schema-78', schema=schema)
    Schema79(type='schema-79', schema=schema)
    Schema80(type='schema-80', schema=schema)
    Schema81(type='schema-81', schema=schema)
    Schema82(type='schema-82', schema=schema)
    Schema83(type='schema-83', schema=schema)
    Schema84(type='schema-84', schema=schema)
    Schema85(type='schema-85', schema=schema)
    Schema86(type='schema-86', schema=schema)
    Schema87(type='schema-87', schema=schema)
    Schema88(type='schema-88', schema=schema)
    Schema89(type='schema-89', schema=schema)
    Schema90(type='schema-90', schema=schema)
    Schema91(type='schema-91', schema=schema)
    Schema92(type='schema-92', schema=schema)
    Schema93(type='schema-93', schema=schema)
    Schema94(type='schema-94', schema=schema)
    Schema95(type='schema-95', schema=schema)
    Schema96(type='schema-96', schema=schema)
    Schema97(type='schema-97', schema=schema)
    Schema98(type='schema-98', schema=schema)
    Schema99(type='schema-99', schema=schema)
  13. 2 remaining items

  14. hauntsaninja commented on Apr 7, 2023

    @hauntsaninja
    Collaborator

    Hmm...

    from pydantic import BaseModel
    
    class MyModel(BaseModel):
        x: int
    

    Running mypy on this file takes about 16s for me with no cache (with both mypy versions 1.1.1 and 1.2 both compiled, pydantic version 1.10.7, no extra mypy configuration)

    Obviously it would be preferable to actually solve the issue, but note that one potential hack is to use a MYPY constant. mypy will always treat this as True, so you could do something like:

    MYPY = False
    if MYPY:
        BigSchemaType = Any  # or possibly a slightly better fallback
    else:
        BigSchemaType = Union[...]
    
  15. adriangb commented on Apr 7, 2023

    @adriangb
    Contributor

    The issue is that if you use those types that 16s grows exponentially (give or take).

    one potential hack is to use a MYPY constant

    We are indeed exploring that: pydantic/pydantic-core#528. Is this approach documented anywhere?

  16. hauntsaninja commented on Apr 7, 2023

    @hauntsaninja
    Collaborator

    Yeah, makes sense, sounds like Samuel didn't paste the full example.py he was running against in #14034 (comment) then

    MYPY constant is documented, but not particularly thoroughly, e.g. at https://mypy.readthedocs.io/en/stable/common_issues.html#python-version-and-system-platform-checks Too much of its use would probably be bad for the typing ecosystem

  17. cdce8p commented on Apr 7, 2023

    @cdce8p
    Collaborator

    Running mypy on this file takes about 16s for me with no cache (with both mypy versions 1.1.1 and 1.2 both compiled, pydantic version 1.10.7, no extra mypy configuration)

    I believe the issue is only present in the current dev builds for pydantic as a result of the upcoming move to version 2. I can reproduce the issue with pydantic==2.0a1 and --no-incremental.

    To reproduce
    1. Create a new venv (I'm using Python 3.11)
    2. pip install -U pip wheel setuptools (might not be necessary)
    3. pip install mypy==1.2.0 pydantic==2.0a1
    4. mypy -v --no-incremental example.py
    # example.py
    from pydantic import BaseModel
    
    class MyModel(BaseModel):
        x: int

    Mypy starts to hang at

    LOG:  Processing SCC singleton (pydantic_core.core_schema) as inherently stale with stale deps (__future__ builtins datetime sys typing)
    
  18. samuelcolvin commented on Apr 7, 2023

    @samuelcolvin
    SponsorContributorAuthor

    Correct @cdce8p, my mistake - I was running on main but the same will be true on 2.0a1 - or indeed any V2 release until we use a hack like if not MYPY: on this line, or mypy fixes some of these performance issues.

  19. hauntsaninja commented on Apr 7, 2023

    @hauntsaninja
    Collaborator

    @cdce8p weird, I still can't repro

    ~/test λ pip freeze                             
    annotated-types==0.4.0
    mypy==1.2.0
    mypy-extensions==1.0.0
    pydantic==2.0a1
    pydantic_core==0.23.0
    tomli==2.0.1
    typing_extensions==4.5.0
    ~/test λ mypy --version                     
    mypy 1.2.0 (compiled: yes)
    λ cat example.py
    from pydantic import BaseModel
    
    class MyModel(BaseModel):
        x: int
    ~/test λ time mypy --no-incremental example.py   
    Success: no issues found in 1 source file
    mypy --no-incremental example.py  2.96s user 0.16s system 99% cpu 3.124 total
    
  20. hauntsaninja commented on Apr 7, 2023

    @hauntsaninja
    Collaborator

    Oh looks like pydantic_core made a new release (presumably including the if MYPY hack)

  21. adriangb commented on Apr 7, 2023

    @adriangb
    Contributor

    Yes, we basically used that constant to the disable type checking for mypy and will have to tell our users that if they want proper type checkin they need to use a different type checker 😔

  22. hauntsaninja commented on Apr 21, 2023

    @hauntsaninja
    Collaborator

    Okay, took a look tonight and found a simple patch that makes this terminate (but is still extremely slow).

    Note for self: pip install 'pydantic-core==0.22.0' 'pydantic==2.0a1' to get the right versions.

  23. added a commit that references this issue on Apr 23, 2023
    d89b317
  24. added a commit that references this issue on Apr 23, 2023
    bd6ce23
  25. JukkaL commented on Apr 24, 2023

    @JukkaL
    Collaborator

    #14150 further improved performance of the simple example by 3.5x for me (when not using a compiled mypy).

  26. added a commit that references this issue on Apr 24, 2023
    d887e9c
  27. hauntsaninja commented on Apr 26, 2023

    @hauntsaninja
    Collaborator

    #15128 should be another 25% for me on the pydantic codebase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions