Repository navigation
Python Language Server uses too much CPU #479
Description
Activity
On #3653 (duplicate)
Just in case the differences matter:
I'll have to check on CPU use, but I didn't notice it. I'd guess it's usually reasonable, even when memory use 10GB or higher (most of the time).
I'm on Python 3 not 2, and have only that version of Python installed.
Reacted by Velizar VESSELINOV and Bruno BarbosaI have couple projects. On python3 is ok. But on python2 cpu usage skips to 100%.
Language Server: 0.1.65
Python output, Extension Output, etc. Is empty, no error messages
Reacted by Velizar VESSELINOV and Jean-Philippe PellerinMost probably similar to #495
i can repro this:
- vscode 1.30.0
- ms-ms-python.python 4.5 (only extension enabled)
git https://github.com/mcedit/mcedit2.git --depth 1- hit
f12on a symbol
Reacted by Velizar VESSELINOV and AhuigoFWIW I have the same behavior on python3.7 on the xarray repo
I've tried both the current release and the daily channel (
Microsoft Python Language Server version 0.1.73.0), and I 100% CPU on bothReacted by Velizar VESSELINOVSeems like I'm running into the same issue with scikit-learn. 100% and minimal RAM.
Reacted by Velizar VESSELINOV and AndersTo all folks: we made some tactical fixes, but there is inherent issue to the analysis engine approach.
We are currently working on major restructuring of the static analysis engine that should address the issue with both CPU and memory. If you install Zenhub extension to Chrome, you should be able to track epic #425. The work is not trivial and is not a bug fix level change.
WIP PR #465.
Thanks you for using the LS, the feedback is much appreciated.
Reacted by Velizar VESSELINOV, Maximilian Roos, Dror Atariah, Mark Adamson, Jean-Philippe Pellerin, Ogi Moore and Jyri LautalaJust as a simple not-so-workaround, how about assigning a lower priority to the spawned process?
On Linux all processes have a value called
nicewhich indicates its priority. A "normal" user process usually uses nice=0. System important processes use lower nice, such as -20; you need to be root to lower the nice. Background, not-so-important processes are expected to use a higher value, such as nice=5.By doing this very simple patch, we don't fix the CPU leak, but at least we tell the CPU to not block more important processes just for the sake of keep running the language server. Basically, the users wouldn't notice so much this leak. And in any case, it's a good practice for background processes.
I usually
renicethe LS process and at least it doesn't disturb so much.I bet other OS have a similar system.
That might be possible. Given this is in C#, we can do:
Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.BelowNormal;
Which has the effect of setting the nice to 10 on Unix-y platforms.
On second thought, it seems like that'd only help in cases where the system is already using all of its cores at once. If the system has a free core to schedule the process on, it's going to do that even if the nice value is high. I'm not sure that would affect other processes if there are already free cycles...
- it can't hurt
- cache-locality matters
- "using all of its cores" happens more often than you might think on a real machine.
Reacted by Jairo LlopisNot sure if you're able to reproduce this issue still or not, I'm happy to provide some log/debug info
OS: macOS 10.14.2
VSCode: 1.30.2
MS-Python Extension: 2019.1.0
Python Version 3.7.2 (via pipenv)The virutalenv has a large number of dependencies (pyside2, numpy, scipy, pandas..); the codebase isn't particularly large. Python is the only extension currently installed (debug step was to remove VSCode entirely and try again).
Below is everything I could think of as being relevant, but if there is something else that would be helpful, let me know.
User Settings
{ "python.jediEnabled": false, "python.analysis.logLevel": "Trace", "python.formatting.provider": "black" }workplace settings:
{ "python.linting.flake8Enabled": true, "python.linting.flake8Path": ".venv/bin/flake8", "python.linting.flake8Args": ["--config=setup.cfg"], "python.linting.pylintEnabled": false, "python.unitTest.pyTestEnabled": true, "python.unitTest.pyTestPath": ".venv/bin/pytest", "python.formatting.blackPath": ".venv/bin/black", "python.pythonPath": ".venv/bin/python", "python.sortImports.path": ".venv/bin/isort", "python.sortImports.args": ["--settings-path ./pyproject.toml"] }View -> Output -> Python
Starting Microsoft Python language server. Downloading https://pvsc.azureedge.net/python-language-server-stable/Python-Language-Server-osx-x64.0.1.78.nupkg... complete Unpacking archive... done Microsoft Python Language Server version 0.1.78.0 Initializing for /Users/ognyan/Developer/fred/.venv/bin/python ##########Linting Output - flake8########## ##########Linting Output - flake8##########Log (Extension host)
[2019-02-01 08:25:33.511] [exthost] [info] extension host started [2019-02-01 08:25:33.522] [exthost] [info] ExtensionService#_doActivateExtension vscode.debug-auto-launch {"startup":true,"activationEvent":"*"} [2019-02-01 08:25:33.522] [exthost] [info] ExtensionService#loadCommonJSModule /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/debug-auto-launch/dist/extension [2019-02-01 08:25:33.524] [exthost] [info] ExtensionService#_doActivateExtension vscode.emmet {"startup":true,"activationEvent":"*"} [2019-02-01 08:25:33.524] [exthost] [info] ExtensionService#loadCommonJSModule /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/emmet/dist/extension [2019-02-01 08:25:33.528] [exthost] [info] ExtensionService#_doActivateExtension vscode.git {"startup":true,"activationEvent":"*"} [2019-02-01 08:25:33.528] [exthost] [info] ExtensionService#loadCommonJSModule /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/main [2019-02-01 08:25:33.549] [exthost] [info] ExtensionService#_doActivateExtension vscode.merge-conflict {"startup":true,"activationEvent":"*"} [2019-02-01 08:25:33.549] [exthost] [info] ExtensionService#loadCommonJSModule /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/merge-conflict/dist/extension [2019-02-01 08:25:33.551] [exthost] [info] eager extensions activated [2019-02-01 08:25:44.582] [exthost] [info] ExtensionService#_doActivateExtension vscode.configuration-editing {"startup":false,"activationEvent":"onLanguage:jsonc"} [2019-02-01 08:25:44.582] [exthost] [info] ExtensionService#loadCommonJSModule /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/configuration-editing/dist/extension [2019-02-01 08:25:44.590] [exthost] [info] ExtensionService#_doActivateExtension vscode.json-language-features {"startup":false,"activationEvent":"onLanguage:jsonc"} [2019-02-01 08:25:44.590] [exthost] [info] ExtensionService#loadCommonJSModule /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/json-language-features/client/dist/jsonMain [2019-02-01 08:25:44.646] [exthost] [info] ExtensionService#_doActivateExtension vscode.typescript-language-features {"startup":false,"activationEvent":"onLanguage:jsonc"} [2019-02-01 08:25:44.646] [exthost] [info] ExtensionService#loadCommonJSModule /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/typescript-language-features/dist/extension [2019-02-01 08:26:57.140] [exthost] [info] ExtensionService#_doActivateExtension vscode.python {"startup":false,"activationEvent":"onLanguage:python"} [2019-02-01 08:26:57.140] [exthost] [info] ExtensionService#loadCommonJSModule /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/python/out/pythonMain [2019-02-01 08:26:57.142] [exthost] [info] ExtensionService#_doActivateExtension ms-python.python {"startup":false,"activationEvent":"onLanguage:python"} [2019-02-01 08:26:57.142] [exthost] [info] ExtensionService#loadCommonJSModule /Users/ognyan/.vscode/extensions/ms-python.python-2019.1.0/out/client/extension [2019-02-01 08:27:12.143] [exthost] [error] [ms-python.python] provider FAILED [2019-02-01 08:27:12.143] [exthost] [error] TypeError: Cannot read property 'map' of undefined at asCompletionResult (/Users/ognyan/.vscode/extensions/ms-python.python-2019.1.0/out/client/extension.js:83:96546) at <anonymous> [2019-02-01 08:27:27.567] [exthost] [error] [ms-python.python] provider FAILED [2019-02-01 08:27:27.567] [exthost] [error] TypeError: Cannot read property 'map' of undefined at asCompletionResult (/Users/ognyan/.vscode/extensions/ms-python.python-2019.1.0/out/client/extension.js:83:96546) at <anonymous> [2019-02-01 08:27:28.768] [exthost] [error] [ms-python.python] provider FAILED [2019-02-01 08:27:28.768] [exthost] [error] TypeError: Cannot read property 'map' of undefined at asCompletionResult (/Users/ognyan/.vscode/extensions/ms-python.python-2019.1.0/out/client/extension.js:83:96546) at <anonymous>}}Log (Window)
[2019-02-01 08:25:33.472] [renderer3] [info] no standard startup: not just one window [2019-02-01 08:27:12.144] [renderer3] [error] Cannot read property 'map' of undefined: TypeError: Cannot read property 'map' of undefined at asCompletionResult (/Users/ognyan/.vscode/extensions/ms-python.python-2019.1.0/out/client/extension.js:83:96546) at <anonymous> [2019-02-01 08:27:27.568] [renderer3] [error] Cannot read property 'map' of undefined: TypeError: Cannot read property 'map' of undefined at asCompletionResult (/Users/ognyan/.vscode/extensions/ms-python.python-2019.1.0/out/client/extension.js:83:96546) at <anonymous> [2019-02-01 08:27:28.769] [renderer3] [error] Cannot read property 'map' of undefined: TypeError: Cannot read property 'map' of undefined at asCompletionResult (/Users/ognyan/.vscode/extensions/ms-python.python-2019.1.0/out/client/extension.js:83:96546) at <anonymous>Log (Shared)
[2019-02-01 08:25:06.261] [sharedprocess] [info] main {"machineId":"78630bf7d35a0ab3bd8c5a2362a1bcc05eac83d48c7423f7ea566f5fce023116"} [2019-02-01 08:25:06.287] [sharedprocess] [info] Scanned user extensions: 0 [2019-02-01 08:25:06.311] [sharedprocess] [info] Scanned user extensions: 0 [2019-02-01 08:25:06.319] [sharedprocess] [info] Scanned user extensions: 0 [2019-02-01 08:25:06.328] [sharedprocess] [info] Scanned user extensions: 0 [2019-02-01 08:25:06.380] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:06.388] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:06.390] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:14.994] [sharedprocess] [info] Scanned user extensions: 0 [2019-02-01 08:25:14.999] [sharedprocess] [info] Scanned user extensions: 0 [2019-02-01 08:25:15.000] [sharedprocess] [info] Scanned user extensions: 0 [2019-02-01 08:25:15.010] [sharedprocess] [info] Scanned user extensions: 0 [2019-02-01 08:25:15.072] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:15.074] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:15.077] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:19.176] [sharedprocess] [info] Installing extension: python [2019-02-01 08:25:19.178] [sharedprocess] [info] Scanned user extensions: 0 [2019-02-01 08:25:19.771] [sharedprocess] [info] Downloaded extension: python /var/folders/49/mw4m6yx97t3dv9y_pjvdbm400000gn/T/27bacb49-061c-4630-90a8-8c9eb5d05f86 [2019-02-01 08:25:20.754] [sharedprocess] [info] Extracted extension to /Users/ognyan/.vscode/extensions/.ms-python.python-2019.1.0: ms-python.python-2019.1.0 [2019-02-01 08:25:20.755] [sharedprocess] [info] Renamed to /Users/ognyan/.vscode/extensions/ms-python.python-2019.1.0 [2019-02-01 08:25:20.758] [sharedprocess] [info] Installation completed. ms-python.python-2019.1.0 [2019-02-01 08:25:20.793] [sharedprocess] [info] Extensions installed successfully: ms-python.python [2019-02-01 08:25:20.905] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:20.907] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:20.937] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:20.939] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:22.651] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:22.652] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:22.654] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:22.666] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:22.668] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:22.729] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:22.731] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:22.733] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:22.735] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:22.744] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:33.225] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:33.227] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:33.282] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:33.285] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:25:33.504] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:36.611] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:25:46.265] [sharedprocess] [info] Starting to clean up unused language packs. [2019-02-01 08:26:57.143] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:26:57.163] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 08:30:33.510] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:35:33.505] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:40:33.503] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:45:33.498] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:50:33.496] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 08:55:33.491] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:00:33.489] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:05:33.482] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:10:33.480] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:15:33.475] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:16:04.085] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:16:04.088] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:16:04.090] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:16:04.099] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:16:04.160] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 09:16:04.162] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 09:16:04.164] [sharedprocess] [info] Scanned system extensions: 71 [2019-02-01 09:20:33.469] [sharedprocess] [info] Scanned user extensions: 1 [2019-02-01 09:25:33.502] [sharedprocess] [info] Scanned user extensions: 1Log (Main)
[2019-02-01 08:25:02.715] [main] [info] update#setState idle [2019-02-01 08:25:32.719] [main] [info] update#setState checking for updates [2019-02-01 08:25:32.873] [main] [info] update#setState idle [2019-02-01 09:25:32.571] [main] [info] update#setState checking for updates [2019-02-01 09:25:32.700] [main] [info] update#setState idleEDIT: Let my machine run over the weekend; still running at ~100% CPU
3 remaining items
Please forgive me for commenting on a closed issue.
Congratulations on that pull request, that looks like a ton of work went into this, thank you. I am curious if you know know what version of the Language Server will incorporate that pull request or when it will go live?
Thanks again for this awesome tool.
If you install Zenhub extension to Chrome, you should be able to see our schedule. Specifically, the Release report.
Reacted by Ogi MooreI am also have the same problem with the following version
Version: 1.32.3
Commit: a3db5be9b5c6ba46bb7555ec5d60178ecc2eaae4
Date: 2019-03-14T23:38:49.842Z
Electron: 3.1.6
Chrome: 66.0.3359.181
Node.js: 10.2.0
V8: 6.6.346.32
OS: Linux x64 4.18.19-100.fc27.x86_64this sucks !!!
Reacted by Velizar VESSELINOV, Владимир Маничев and ArthurReacted by Maximilian Roos, Jairo Llopis and Artem ErmulinMikhail Arkhipov (@MikhailArkhipov) Brett Cannon (@brettcannon)
It is exceptional to see so badly managing resource piece of software in production with known bugs (and/or bad design) since months and still not fixed. Not sure what it is wrong the management of the project or people working on this project or the testing, but it is very difficult for users of VS Code (Python on Mac...) to trust the capacity of Microsoft to produce good quality software.
What is the required hardware capacity to run VS Code with Python (on Mac)? 70 GiB RAM for VS Code?
Reacted by Jairo Llopis, Anders and Richard HöchenbergerThis is not normal, it is a bug. We have been working for many weeks on performance improvements, including memory usage. At the latest revision, analyzing tensorflow on my machine only uses 400 MB, as opposed to the multiple gigabytes it used to take. If you have a project which reproduces it, we'd love to test it and figure out why that would occur.
Also, try switching up to the beta release, which has many more improvements not yet in stable.
"python.analysis.downloadChannel": "beta"
Jake Bailey (@jakebailey) I understand that this is a bug, I'm happy to see that you are working on fixing it, but the problem of abusive usage of resources is not new. This ticket is more six months old, and there was other as well earlier Q4 last year. I'm surprised that it is taking you more than six months to fix it, plus have the feeling that is going backward not reducing the resources used but increasingly. 20-40 GB was the "normal" level a few months ago, and now it is 70 GB.
I hope that soon you will fix it, and not again add small patches that solve few loading pick issues. The hidden requirement to restart many times per day VS Code reminds me of the old times of Windows.
Reacted by Jairo Llopis, Maximilian Roos, Brett Cannon, Tony Benoy, Yulong Jing and Anders
Analyzing workspace of Microsoft.Python.LanguageServer at the early start of opening, is a killing resources machines, is just like a python surrounding your neck, iI disabled other extensions, Jedi False, and the server just takes all available resources as a desperately hungry that just return from the jungle ...
Please let me know if you have some tips, tricks or guidance to resolve or how we can help to resolve this.
Reacted by Velizar VESSELINOVReading related reports issues, and testing, I got an improvement that helps me a few.
Setting the download channel to "beta" version, the log level "Trace" and Jedi False btw."python.analysis.logLevel": "Trace",
"python.analysis.downloadChannel": "beta",
"python.jediEnabled": falseReacted by Velizar VESSELINOVThank you for the work on python language server.
I am a happy user of vscode, but when it comes to python code in my project, I have to switch to sublime etc., because python package is bundled together with language server (I might be wrong), and now I also have this issue of very high cpu usage, (I have relative faster macbook air with i7 processor)...
Is it possible to separate the python extension and python language server?
You can disable the language server by setting
python.jediEnabledto true. The language server is not enabled by default in the Python extension; you would have needed to enable it explicitly to have it run.Setting
python.jediEnabledto true works for mebut it keeps showing this....
Is there a way to ignore this permanently?
You can't use Jedi with IntelliCode; IntelliCode's Python support is an extension to the language server. That gear button may offer a way, but past that I do not know other than to remove IntelliCode.
Be aware that we've just built 0.3.22, which is available in the beta/daily channels. It contains some fixes which can help. You can set this and reload to update:
"python.analysis.downloadChannel": "beta"Reacted by Arthur Chien
Velizar VESSELINOV (@VelizarVESSELINOV) commented on Tue Dec 04 2018
Environment data
Expected behaviour
< 1% CPU
Actual behaviour
> 99% CPU
Steps to reproduce:
Logs
Output for
Pythonin theOutputpanel (View→Output, change the drop-down the upper-right of theOutputpanel toPython)Brett Cannon (@brettcannon) commented on Wed Dec 05 2018
Do you have
"python.autoComplete.extraPaths"defined?Michał Mrówka (@PushUpek) commented on Thu Dec 06 2018
I have the same issue, but only with python2
Velizar VESSELINOV (@VelizarVESSELINOV) commented on Thu Dec 06 2018
Brett Cannon (@brettcannon) only in the default user settings defined as
[]Velizar VESSELINOV (@VelizarVESSELINOV) commented on Thu Dec 06 2018
It is the biggest CPU consumer of all the processes: >21 hours
Side comment that maybe it is the case of Michał Mrówka (@PushUpek) I have a lot of files and folders in my current folder that is not Python code or libraries, questioning if Microsoft Python Server is not trying to parse GiB of data continuously in the current folder of VS Code project.
Jonas Witt (@jonaswitt) commented on Mon Dec 10 2018
+1 - I have the same problem with
VS Code 1.29.1
Microsoft Python Language Server version 0.1.65.0
Python Extension 2018.11.0
Python 2.7.15 (venv-based environment)
My
python.autoComplete.extraPathssetting is also[]. I tried to disable Jedi and Linting. I keep killing the process to keep my Macbook's fans from spinning, but it always respawns at 100% CPU usage.Is there any way to disable the language server? I believe my only option right now is to disable the python extension completely.
Martin Häcker (@dwt) commented on Mon Dec 10 2018
This happens for me too. Python 2.7 with a moderately big project. What Info can I provide to help debug this?