Skip to content

Use worker threads for launching conda and windows registry when discovering #22146

Closed
#22964
Closed
Use worker threads for launching conda and windows registry when discovering#22146
#22964
Assignees
karrtikr
Labels
area-environmentsFeatures relating to handling interpreter environmentsfeature-requestRequest for new features or functionalityneeds PRReady to be worked onverification-neededVerification of issue is requestedverifiedVerification succeeded
Milestone
March 2024

Description

@karrtikr
Kartik Raj (karrtikr)
opened on Oct 3, 2023

#21755

Spike results

Based on CPU profiles in #21755 (comment) discovery is mainly blocked by:

  • Launching conda binary
  • Launching REG binary for windows registry
  • Windows Path discovery

Note spawning python binaries is fine, it blocks very little of the time.

Activity

  1. Kartik Raj (karrtikr)
    added
    feature-requestRequest for new features or functionality
    area-environmentsFeatures relating to handling interpreter environments
    needs PRReady to be worked on
    on Oct 3, 2023
  2. Kartik Raj (karrtikr)
    self-assigned this
    on Oct 3, 2023
  3. Kartik Raj (karrtikr)
    added this to the October 2023 milestone on Oct 3, 2023
  4. Kartik Raj (karrtikr)
    modified the milestones: October 2023, November 2023 on Oct 23, 2023
  5. Kartik Raj (karrtikr)
    changed the title [-]Use webworker for launching conda and windows registry when discovering[/-] [+]Use worker threads for launching conda and windows registry when discovering[/+] on Nov 15, 2023

  6. Tyriar commented on Nov 15, 2023

    @Tyriar
    Daniel Imms (Tyriar)
    on Nov 15, 2023

    Robo (@deepak1556) do you know why spawn would be blocking the thread for so long here?

    image

    Profiles at #21755 (comment)

  7. karrtikr commented on Nov 16, 2023

    @karrtikr
    Kartik Raj (karrtikr)
    on Nov 16, 2023
    Author

    Note this is when spawning the conda binary, similarly spawning REG binaries can seldomly be like this. But python binaries are much cheaper for some reason.

  8. Kartik Raj (karrtikr)
    added a commit that references this issue on Nov 16, 2023

    Use worker threads for fetching Windows Registry interpreters (#22479)

    e27185a

  9. karrtikr commented on Nov 16, 2023

    @karrtikr
    Kartik Raj (karrtikr)
    on Nov 16, 2023
    Author

    It's not just us, the git extension while spawning also runs into this:

    image

  10. DonJayamanne commented on Nov 16, 2023

    @DonJayamanne
    Don Jayamanne (DonJayamanne)
    on Nov 16, 2023

    do you know why spawn would be blocking the thread for so long here?

    This was my observation a few years ago as well ( #15813 (comment))
    Back then I reached out to jrieken , and he indicated that in node, spawning processes can block the main thread for a few ms.
    However we didn't try to dig further into why spawning conda was blocking for a few seconds, primarily because the suggestion was not being actively worked on back then

  11. karrtikr commented on Nov 16, 2023

    @karrtikr
    Kartik Raj (karrtikr)
    on Nov 16, 2023
    Author

    Performance issue filled for other cases here: microsoft/vscode#198384

  12. Tyriar commented on Nov 16, 2023

    @Tyriar
    Daniel Imms (Tyriar)
    on Nov 16, 2023

    Yeah a few ms is expected, but it's very different from over 2 seconds which is what the first spawn takes in my screenshot.

  13. 62 remaining items

  14. Daniel Imms (Tyriar)
    reopened this on Mar 25, 2024
  15. Daniel Imms (Tyriar)
    added
    verification-foundIssue verification failed
    on Mar 25, 2024

  16. karrtikr commented on Mar 25, 2024

    @karrtikr
    Kartik Raj (karrtikr)
    on Mar 25, 2024
    Author

    but there's an exception launching conda (which I don't think is installed) and there's no "worker id"

    Sorry, this is actually expected, missed to update the verification steps from before. The important thing to verify here is that the number of environments discovered are still the same, once discovery finishes in both cases.

  17. Kartik Raj (karrtikr)
    added
    verifiedVerification succeeded
    and removed
    verification-foundIssue verification failed
    on Mar 25, 2024
  18. Kartik Raj (karrtikr)
    closed this as completedon Mar 27, 2024
  19. Wes McKinney (wesm)
    added 4 commits that reference this issue on Mar 28, 2024

    Merged PR posit-dev/positron-python#277: Merge vscode-python 2023.22 …

    7dd0e33

    Merged PR posit-dev/positron-python#395: Merge upstream: `v2024.2.0`

    d822c25

    Merged PR posit-dev/positron-python#277: Merge vscode-python 2023.22 …

    439aafc

    Merged PR posit-dev/positron-python#395: Merge upstream: `v2024.2.0`

    c976f74
  20. Wes McKinney (wesm)
    added 4 commits that reference this issue on Apr 5, 2024

    Fix duplicated environments in interpreter list (microsoft/vscode-pyt…

    8e48cb8

    Include windows registry perf improvement in `pythonDiscoveryUsingWor…

    9824f09

    Fix duplicated environments in interpreter list (microsoft/vscode-pyt…

    7103a45

    Include windows registry perf improvement in `pythonDiscoveryUsingWor…

    27a1f4d
  21. github-actions
    locked as resolved and limited conversation to collaborators on Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-environmentsFeatures relating to handling interpreter environmentsfeature-requestRequest for new features or functionalityneeds PRReady to be worked onverification-neededVerification of issue is requestedverifiedVerification succeeded

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions