Skip to content

Add support for "conda run" in PythonExecutionFactory - #8259

Merged
Kim-Adeline Miguel (kimadeline) merged 70 commits into
microsoft:masterfrom
kimadeline:7696-python-execution-service-conda-run
Nov 21, 2019
Merged

Kim-Adeline Miguel (kimadeline) merged 70 commits into
microsoft:masterfrom
kimadeline:7696-python-execution-service-conda-run

Conversation

@kimadeline

@kimadeline Kim-Adeline Miguel (kimadeline) commented Oct 28, 2019 •

Copy link
Copy Markdown

First PR for #7696 (terminal execution update will happen separately)

  • Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
  • Title summarizes what is changing
  • Has a news entry file (remember to thank yourself!)
  • Appropriate comments and documentation strings in the code
  • Has sufficient logging.
  • Has telemetry for enhancements.
  • Unit tests & system/integration tests are added/updated
  • Test plan is updated as appropriate
  • package-lock.json has been regenerated by running npm install (if dependencies have changed)
  • The wiki is updated with any design decisions/details.

@kimadeline Kim-Adeline Miguel (kimadeline) changed the title Add "conda run" support Add support for "conda run" Oct 28, 2019
@codecov-io

Codecov (codecov-io) commented Oct 28, 2019 •

Copy link
Copy Markdown

Codecov Report

Merging #8259 into master will increase coverage by 0.07%.
The diff coverage is 92.3%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8259      +/-   ##
==========================================
+ Coverage   59.18%   59.25%   +0.07%     
==========================================
  Files         521      522       +1     
  Lines       23977    24030      +53     
  Branches     3873     3882       +9     
==========================================
+ Hits        14190    14239      +49     
- Misses       8870     8875       +5     
+ Partials      917      916       -1
Impacted Files Coverage Δ
src/client/interpreter/contracts.ts 100% <ø> (ø) ⬆️
src/client/common/process/types.ts 100% <ø> (ø) ⬆️
...reter/locators/services/cacheableLocatorService.ts 85.52% <0%> (ø) ⬆️
src/client/common/process/pythonDaemon.ts 23.07% <0%> (-0.17%) ⬇️
src/client/common/process/pythonDaemonPool.ts 70.4% <0%> (-0.57%) ⬇️
src/client/interpreter/locators/index.ts 89.74% <0%> (ø) ⬆️
...ient/interpreter/locators/services/condaService.ts 84.33% <100%> (ø) ⬆️
src/client/terminals/codeExecution/repl.ts 100% <100%> (ø) ⬆️
src/client/common/process/pythonProcess.ts 96% <100%> (+0.44%) ⬆️
...rc/client/common/process/pythonExecutionFactory.ts 97.61% <100%> (+4.07%) ⬆️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97d017c...0f93c43. Read the comment docs.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (aside from 2 little things)

Thanks for making those changes. I'm approving the PR, but would still like you to consider the 2 comments before merging. :)

Comment thread src/client/common/process/pythonExecutionFactory.ts Outdated
Comment thread src/client/interpreter/contracts.ts
@kimadeline
Kim-Adeline Miguel (kimadeline) force-pushed the 7696-python-execution-service-conda-run branch from 091957a to b6b3fd3 Compare November 16, 2019 06:40
public execObservable(args: string[], options: SpawnOptions): ObservableExecutionResult<string> {
const opts: SpawnOptions = { ...options };
// Cannot use this.getExecutionInfo() until 'conda run' can be run without buffering output.
// See https://github.com/microsoft/vscode-python/issues/8473

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please ensure you use createActivatedEnvironment instead of procService.
Else this is totally useless.
When running conda we must either use conda run or at a minimium revert to the old hack, activate the environment then grab the env variables and run in the context of those variables.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because today we use createActivatedEnvironment when running tools such as pytest, unittest, etc.
If we are going with conda run then createActivatedEnvironment becomes obsolete, BUT ONLY if this (above mentioned) change is added.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is that old hack? I could only find the following createActivatedEnvironment calls on master:
image

In this PR execObservable and execModuleObservable use the old code, I've only added a comment.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Further to this, createActivatedEnvironment was created to get around conda run. It allows code to execute python code within an activated python environment context (which is what conda run does).

So, anywhere createActivatedEnvironment is being used, we need to ensure it uses condaRun.

We have a couple of options:

  • Return conda execution service when calling createActivatedEnvironment
  • Drop createActivatedEnvironemnt (or make it private)
  • Anything that requires observable output, then the conda execution service will fallback to using the service returned by createActivatedEnvironment.
  • Please remember, conda run isn't the only special case. We could have pyenv and others. pyenv requires initializing an environment variable to run the correct python interpreter (they use shims).
  • Hence conda run is an implementation detail when it comes to createActivatedEnvironment.

Finally, i'd prefer if these were done as separate PRs (issues), as opposed to doing them all in one go in a large PR...I.e. first implement conda exec service, then another, then integrate in a few places, etc... Else it just feels too large (to review)

Karthik Nadig (@karthiknadig) /cc

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets talk

@kimadeline
Kim-Adeline Miguel (kimadeline) merged commit b543408 into microsoft:master Nov 21, 2019
@kimadeline
Kim-Adeline Miguel (kimadeline) deleted the 7696-python-execution-service-conda-run branch November 21, 2019 22:17
@ericsnowcurrently

Copy link
Copy Markdown

hurray!

@lock lock Bot locked as resolved and limited conversation to collaborators Nov 29, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants