Repository navigation
High memory usage in 0.3.40(memory leak?) #1298
Description
Activity
After working on the project for some time
How much time have you used it?
Sorry I should have clarified.
I usually notice the issue after 1-4h of the last reload of vscode or kill of the language service.
But I'm estimating here, I have not traked it preciselyI've checked and I still had vscode open after today. Attached is the log from the python output.
vscode python log.txt
I've killed it at 12, 15:30, then later this evening when I resumed the pc from sleepOk, thank you!
We just built 0.3.22, which is available in the beta/daily channels. It contains some fixes which we believe will help solve some of the leaks. You can set this and reload to update:
"python.analysis.downloadChannel": "beta"Reacted by Jongwook Choi and Federico CaselliI'll try it and report back. Thanks for the quick feedback 👍
I still have the same problem with the 0.3.22.
This is after about 1h of work.
Reinstalling the extension or deleting the folder of the language server may help?
I've logged the memory usage with the performance monitor and this is the graph (in MB)
- changed the title
[-]High memory usage in 0.3.20 (memory leak?)[/-][+]High memory usage in 0.3.22 (memory leak?)[/+]on Jul 10, 2019 Federico Caselli (@CaselIT) , If you have a venv, can you create
requirements.txtand attach it to the bug?Attached is a pip freeze of the current environment.
I use conda to manage the environments, but I've used pip to install the packages, so all should be there
pip freeze.txtI'm noticing the same memory issue. I'll leave VS Code running for about 1hr and it will end up using 10gb of RAM.
OS Version: Version Windows 10.0.18362.175
Python Version: 2.7.16
VS Code version: 1.36.1Reacted by Areeb Jamal and Areeb JamalPlease try the daily build, which has a few more fixes (mainly #1316).
"python.analysis.downloadChannel": "daily"Currently 0.3.28.
53 remaining items
On 0.3.46 the leaking behavior seems to be solved. Now, the LS reaches around 1GB while analyzing the files, and after it, it drops to around 400 MB. Multiple
pip installof the project do not accumulate memory as before, getting exactly the same memory behavior every install cycle.Reacted by Jake Bailey and Sumeet RankaSorry for the long absence, but I've been on holiday and then had to work on another project for a bit.
This week I've returned to work to the project that has been causing problems, and I still have large memory issues.
This is a dump from
Microsoft Python Language Server version 0.3.72.0with a size of ~6gb
dupheap stas.5.9gb.txtI'm not been tracking its memory usage but I've noticed that sometimes its memory usage decreases: a few minutes before dumping the memory to collect the stats the language server was using ~8gb, so some progress there seems to have happened.
Hope it helps. I can track its memory usage it may help
Reacted by Amir AlaviJust a follow up.
I'm currently working on the sqlalchemy library so I have it installed in editable mode with pip (
pip install -e .in the sqlalchemy folder) and after about 1h of working on it, mainly navigating between classes, the language server is at about 4.3 GB or used ram. I'm on Version 0.4.71.0I was using sqlalchemy also on my original project. I still cannot trigger it on demand, but at least this is now happening on an open source project, so it may be easier to reproduce
I'm using a env only for this, so the packages installed are not that many (sadly I had jupyterlab installed in the same env, so this increases the number of packages, but I'm not using it or referencing it in the project):
Packages
Package Version Location ------------------ ------------ ------------------------------------------------- apipkg 1.5 appdirs 1.4.3 asn1crypto 1.2.0 atomicwrites 1.3.0 attrs 19.2.0 backcall 0.1.0 black 19.3b0 bleach 3.1.0 certifi 2019.9.11 cffi 1.13.0 Click 7.0 colorama 0.4.1 cryptography 2.7 decorator 4.4.0 defusedxml 0.6.0 entrypoints 0.3 execnet 1.7.1 importlib-metadata 0.23 ipykernel 5.1.2 ipython 7.8.0 ipython-genutils 0.2.0 jedi 0.15.1 Jinja2 2.10.3 json5 0.8.5 jsonschema 3.0.2 jupyter-client 5.3.3 jupyter-core 4.5.0 jupyterlab 1.1.4 jupyterlab-server 1.0.6 MarkupSafe 1.1.1 mistune 0.8.4 mock 3.0.5 more-itertools 7.2.0 nbconvert 5.6.0 nbformat 4.4.0 notebook 6.0.1 packaging 19.2 pandocfilters 1.4.2 parso 0.5.1 pickleshare 0.7.5 pip 19.2.3 pluggy 0.13.0 prometheus-client 0.7.1 prompt-toolkit 2.0.10 psycopg2 2.8.3 py 1.8.0 pycparser 2.19 Pygments 2.4.2 PyMySQL 0.9.3 pyparsing 2.4.2 pyrsistent 0.15.4 pytest 5.2.1 pytest-forked 1.0.2 pytest-xdist 1.30.0 python-dateutil 2.8.0 pywin32 225 pywinpty 0.5.5 pyzmq 18.1.0 Send2Trash 1.5.0 setuptools 41.4.0 six 1.12.0 SQLAlchemy 1.4.0b1.dev0 c:\\sqlalchemy\lib terminado 0.8.2 testpath 0.4.2 toml 0.10.0 tornado 6.0.3 traitlets 4.3.3 wcwidth 0.1.7 webencodings 0.5.1 wheel 0.33.6 wincertstore 0.2 zipp 0.6.0
I'll try recreating the env with only sqlalchemy and its dependencies to check if the problem persists
Cloning and opening
sqlalchemyas a workspace yielded ~500MB RAM consumption. This is 0.5.10Since about version 0.5 I don't seem to notice overly large memory usage, sometimes I've seen a couple of GB, but I have not had to kill the language server for it. I'm not monitoring it closely though
I'm not sure if the underling issue has been solved or not, but I think this can be closed for now. If I notice again the same high usage I'll reopen this (or a new one)
Thanks! 2GB is high-ish, but with large libraries sometimes happens during peak consumption which should be released when analysis is done.
Thanks for the improvements you and your team have doing to the language server 👍
The fixed implemented in #832 worked for some releases, but I'm having again high memory usage problems (10gb+) with version 0.3.20
These seems to be due to a memory leak: if finishes analyzing the project without problems, using less than 1gb in my case. After working on the project for some time it starts using more memory, even more than 10gb+.
I've not kept note of the details regarding after how long or if there is an action that triggers it. I usually notice a slow down, check the task manager and usually the language service is using many gb of memory. I have not kept track to see it the increase in memory usage is sudden or more gradual.
Below are the packages of the project used and some system information. I haven't checked if I can reproduce this issue in other projects
Is there some logging or telemetry I can enable to help with this issue?
Extension version: 2019.6.22090
Microsoft Python Language Server version 0.3.20.0
Python version: 3.6.8
VS Code version: Code 1.36.0 (0f3794b38477eea13fb47fbe15a42798e6129338, 2019-07-03T13:25:46.372Z)
OS version: Windows_NT x64 10.0.18362
requirements of the project
argon2_cffi==18.3.0python-dateutil==2.7.5decorator==4.3.0falcon>=2,<3falcon-auth==1.1.0falcon-cors==1.1.7graphene==2.1.3graphene_sqlalchemy==2.0.0jsonschema==2.6.0keras>=2.1.2numpy>=1.13.3ortools<7.1pandas>=0.22.0,!=0.24.0psycopg2-binary>=2.7.3.2psycopg2<2.8pyjwt>=1.6.4scikit-learn>=0.19.1scipy>=1.0.0SQLAlchemy<1.3.0SQLAlchemy-Utils>=0.32.21sqlalchemy-postgres-copy>=0.5.0simplejson>=3.13.2tensorflow==1.12.0pyDOE>=0.3.8geomdl>=4.1.0pyomo==5.5.0pyutilib>=5.6.3joblib==0.11pytest>=4.4.0pytest-cov>=2.6.1yapf>=0.20.0,!=0.27flake8>=3.7.0matplotlib>=2.2.3waitress==1.1.0pydot==1.2.4System Info
flash_3d: enabled
flash_stage3d: enabled
flash_stage3d_baseline: enabled
gpu_compositing: enabled
multiple_raster_threads: enabled_on
native_gpu_memory_buffers: disabled_software
oop_rasterization: disabled_off
protected_video_decode: enabled
rasterization: enabled
skia_deferred_display_list: disabled_off
skia_renderer: disabled_off
surface_synchronization: enabled_on
video_decode: enabled
viz_display_compositor: disabled_off
webgl: enabled
webgl2: enabled