Skip to content

New language server seems to be frozen at analyzing workspace for more than 30 minutes #2297

Description

Environment data

  • VS Code version: Insiders
  • Extension version (available under the Extensions sidebar): Dev
  • OS and version: Win 10, Mac
  • Python version (& distribution if applicable, e.g. Anaconda): 3.6
  • Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
  • Relevant/affected Python packages and their versions: N/A

Actual behavior

Progress message does not go away

Expected behavior

Progress in status bar must go away.

Steps to reproduce:

  • Open PTVSD repo
  • Watch the progress indicator

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

Diagnostic Code: InvalidEnvironmentPathVariableDiagnostic, Message: The environment variable 'Path' seems to have some paths containing the '"' character. The existence of such a character is known to have caused the Python extension to not load. If the extension fails to load please modify your paths to remove this '"' character.
Starting Microsoft Python language server.
Downloading https://pvsc.blob.core.windows.net/python-language-server/Python-Language-Server-win-x64.0.1.18204.3.nupkg... complete.Unpacking archive... done.
[Info  - 7:05:48 AM] Created Microsoft.PythonTools.Interpreter.Ast.AstPythonInterpreter instance from Microsoft.PythonTools.Interpreter.Ast.AstPythonInterpreterFactory
Initializing for C:\ProgramData\Anaconda3\python.exe
Loading files from c:\Development\VSCode\ptvsd
[Info  - 7:06:05 AM] Reloading modules...
[Info  - 7:06:08 AM] Reloading modules...
[Info  - 7:06:10 AM] Reloading modules...
[Info  - 7:06:23 AM] Reloading modules...
[Info  - 7:06:40 AM] Reloading modules...
[Info  - 7:06:42 AM] Reloading modules...
[Info  - 7:06:49 AM] Reloading modules...
[Error - 7:08:44 AM] One or more errors occurred. (failed to parse file file:///c:/Development/VSCode/ptvsd/one.py)
[Info  - 7:13:29 AM] Reloading modules...
[Info  - 7:13:34 AM] Reloading modules...
[Info  - 7:13:38 AM] Reloading modules...
[Info  - 7:13:40 AM] Reloading modules...
[Info  - 7:13:43 AM] Reloading modules...
[Info  - 7:13:44 AM] Reloading modules...
[Info  - 7:13:47 AM] Reloading modules...

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help)

capture

Activity

  1. xobs commented on Aug 2, 2018

    @xobs

    I'm experiencing this issue as well.

    Relatedly, my vscode is currently in a state where it is not analyzing the workspace, vscode is idle, but Microsoft.Python.LanguageServer.exe is using 22% CPU. So while Analyzing workspace, ${n} items remaining... is one indicator of the issue, it's not the only indicator, and sometimes the language server can spin on its own.

  2. DonJayamanne commented on Aug 3, 2018

    @DonJayamanne
    Author

    Happened again today (on a Mac):
    screen shot 2018-08-03 at 3 40 18 am

  3. derekwallace commented on Aug 3, 2018

    @derekwallace

    I experience this issue aswell.

  4. changed the title [-]New language server seems to frozen at analyzing workspace for more than 30 minutes[/-] [+]New language server seems to be frozen at analyzing workspace for more than 30 minutes[/+] on Aug 3, 2018
  5. ijstokes commented on Aug 4, 2018

    @ijstokes

    Here too. Not sure if it is helpful, but I'm also having this pop up in the OUTPUT tab:

    [Error - 07:34:56] System.NullReferenceException: Object reference not set to an instance of an object.
       at System.Object.GetType()
       at Microsoft.PythonTools.Analysis.Analyzer.ExpressionEvaluator.EvaluateWorker(Node node)
       at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(ClassDefinition node)
       at Microsoft.PythonTools.Parsing.Ast.ClassDefinition.Walk(PythonWalker walker)
       at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(SuiteStatement node)
       at Microsoft.PythonTools.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker)
       at Microsoft.PythonTools.Analysis.Analyzer.DDG.WalkBody(Node node, AnalysisUnit unit)
       at Microsoft.PythonTools.Analysis.ClassAnalysisUnit.AnalyzeWorker(DDG ddg, CancellationToken cancel)
       at Microsoft.PythonTools.Analysis.Analyzer.DDG.Analyze(Deque`1 queue, CancellationToken cancel, Action`1 reportQueueSize, Int32 reportQueueInterval)
       at Microsoft.PythonTools.Analysis.PythonAnalyzer.AnalyzeQueuedEntries(CancellationToken cancel)
       at Microsoft.PythonTools.Intellisense.AnalysisQueue.GroupAnalysis.Analyze(CancellationToken cancel)
       at Microsoft.PythonTools.Intellisense.AnalysisQueue.Worker(Object threadStarted)
    

    (and Don Jayamanne (@DonJayamanne) has Microsoft taken over the vscode-python repo? IOW, is this the main repo to track and report issues to?)

  6. ijstokes commented on Aug 4, 2018

    @ijstokes
  7. ijstokes commented on Aug 4, 2018

    @ijstokes
  8. DonJayamanne commented on Aug 7, 2018

    @DonJayamanne
    Author
  9. MikhailArkhipov commented on Aug 7, 2018

    @MikhailArkhipov

    Steve Dower (@zooba) may be interested in microsoft/vscode#55815 as well.

  10. 16 remaining items

  11. sneridagh commented on Sep 13, 2018

    @sneridagh

    Same here, CPU spinning to 100% "Analyzing workspace". I have a large one, but who that works at a professional level doesn't?

    screen shot 2018-09-13 at 10 46 53

    What did it "analyze"?

  12. allenyllee commented on Sep 13, 2018

    @allenyllee

    I've encountered the same problem in my ubuntu 18.04 64bit with vscode 1.27.1, vscode-python 2018.8.0 and python 3.6.5

    here is the trace output log:

    output log
    Starting Microsoft Python language server.
    Initializing for /usr/bin/python3
    Loading files from /home/allenyl/anaconda3/lib/python3.6/site-packages/jieba
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/_compat.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__init__.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__main__.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/prob_emit.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/__init__.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/prob_trans.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/viterbi.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/char_state_tab.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/prob_start.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/tfidf.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/__init__.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/textrank.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/analyzer.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/prob_emit.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/__init__.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/prob_trans.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/prob_start.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/_compat.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/_compat.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__init__.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__init__.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__main__.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__main__.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/prob_emit.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/prob_emit.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/__init__.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/__init__.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/prob_trans.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/prob_trans.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/viterbi.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/viterbi.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/char_state_tab.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/char_state_tab.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/prob_start.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/posseg/prob_start.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/tfidf.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/tfidf.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/__init__.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/__init__.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/textrank.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/textrank.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/analyzer.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/analyse/analyzer.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/prob_emit.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/prob_emit.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/__init__.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/__init__.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/prob_trans.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/prob_trans.py
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/prob_start.py at version -1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/finalseg/prob_start.py
    Opening document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__init__.py
    Parsing document file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__init__.py
    Waiting for parsing to complete
    Parsing complete. Waiting for analysis entries to enqueue
    Parse complete for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__init__.py at version 1
    Analysis queued for file:///home/allenyl/anaconda3/lib/python3.6/site-packages/jieba/__init__.py
    Enqueue complete. Waiting for analysis to complete
    [Error - 下午7:50:09] System.NullReferenceException: Object reference not set to an instance of an object.
       at Microsoft.PythonTools.Analysis.ModuleResolver.GetModuleFullName(String originatingModule, String relativePath)
       at Microsoft.PythonTools.Analysis.ModuleResolver.ResolvePotentialModuleNames(String importingFromModuleName, String importingFromFilePath, String relativeModuleName, Boolean absoluteImports)+MoveNext()
       at System.Collections.Generic.LargeArrayBuilder`1.AddRange(IEnumerable`1 items)
       at System.Collections.Generic.EnumerableHelpers.ToArray[T](IEnumerable`1 source)
       at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
       at Microsoft.PythonTools.Analysis.Analyzer.DDG.TryImportModule(String modName, Boolean forceAbsolute, ModuleReference& moduleRef, IReadOnlyList`1& remainingParts)
       at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(FromImportStatement node)
       at Microsoft.PythonTools.Parsing.Ast.FromImportStatement.Walk(PythonWalker walker)
       at Microsoft.PythonTools.Analysis.Analyzer.DDG.Walk(SuiteStatement node)
       at Microsoft.PythonTools.Parsing.Ast.SuiteStatement.Walk(PythonWalker walker)
       at Microsoft.PythonTools.Parsing.Ast.PythonAst.Walk(PythonWalker walker)
       at Microsoft.PythonTools.Analysis.AnalysisUnit.AnalyzeWorker(DDG ddg, CancellationToken cancel)
       at Microsoft.PythonTools.Analysis.Analyzer.DDG.Analyze(Deque`1 queue, CancellationToken cancel, Action`1 reportQueueSize, Int32 reportQueueInterval)
       at Microsoft.PythonTools.Analysis.PythonAnalyzer.AnalyzeQueuedEntries(CancellationToken cancel)
       at Microsoft.PythonTools.Intellisense.AnalysisQueue.GroupAnalysis.Analyze(CancellationToken cancel)
       at Microsoft.PythonTools.Intellisense.AnalysisQueue.HandleAnalyzable(IAnalyzable item, AnalysisPriority priority, CancellationToken cancellationToken)
       at Microsoft.PythonTools.Intellisense.AnalysisQueue.ConsumerLoop()
    

    and the result of top command:

    deepin _gnome-terminal-server_20180913201617

    Although CPU has idled while this small project, the memory consumption seems too high.

    the project is https://github.com/fxsjy/jieba

  13. MikhailArkhipov commented on Sep 13, 2018

    @MikhailArkhipov

    Remaining items in the status bar may be UI issue if CPU load is low. If LS was restarted b/c of exception (most probable cause is microsoft/python-language-server#72) then UI just shows string from the previous instance.

  14. richardwb commented on Sep 14, 2018

    @richardwb

    One thing I noticed here is some symlink folder structures can cause really deep parsing queues to form.

    Recursive/looping symlinks can end up propagating pretty deeply and if there's actual content anywhere along the way it will take a long time. For example, given a simple folder structure like this:

    /code
      /test
        /some
          test.py
          /folder
            /symlink -> ../../some
            __init__.py (empty)
            lib.py (empty)
    

    And test.py is simply "import folder.lib" so we start traversing the tree, I see the following trace output:

    Initializing for /usr/local/opt/python/bin/python2.7
    Loading files from /Users/username/code/test/some
    Parsing document file:///Users/username/code/test/some/test.py
    Parsing document file:///Users/username/code/test/some/folder/lib.py
    Parsing document file:///Users/username/code/test/some/folder/__init__.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/test.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/lib.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/__init__.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/test.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/lib.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/__init__.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/test.py
    Parse complete for file:///Users/username/code/test/some/test.py at version -1
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
    Analysis queued for file:///Users/username/code/test/some/test.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/test.py
    Parsing document file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/lib.py
    <more>
    Parse complete for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py at version -1
    Analysis queued for file:///Users/username/code/test/some/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/symlink/folder/__init__.py
    

    I managed to work around this by just adding the suspect symlinks to my VSCode excludes.

    There's also some other odd behavior with dead symlinks, but it doesn't seem to act as poorly.

  15. added a commit that references this issue on Sep 15, 2018
    3c44aa4
  16. sneridagh commented on Sep 17, 2018

    @sneridagh

    Symlink could cause it... I have tons of them (I use zc.buildout, with an extension (omelette) that makes all the eggs used by the project available in the same tree, using symlinks).

    I'm not the only one using it... in Zope/Plone world is quite usual that devs use these tools.

    It would be great if we can fix it somehow.

  17. samuelpcabral commented on Sep 17, 2018

    @samuelpcabral

    Same thing here! Windows 10, with Python 2.7
    capture

    and it looks like it's consuming far more memory than it should.

    capture2

  18. mungojam commented on Sep 17, 2018

    @mungojam

    I wondered if it is processing my ./.venv pipenv package folder or some other temporary sub-folder. The number of items it said it was processing seemed much larger than my code base.

  19. sneridagh commented on Sep 17, 2018

    @sneridagh

    True, I didn't take into account that virtualenv also uses symlinks... don't know about pipenv, but I guess it also does it. So, there's a major problem then that might affect the vast majority of people.

  20. xobs commented on Sep 17, 2018

    @xobs

    It's happening on Windows as well, which doesn't use symlinks for virtualenv (that I know of). Also, my project doesn't use virtualenv.

    However, it does use environment variables and mess with PYTHONPATH, which may be related to the cause.

  21. added a commit that references this issue on Sep 17, 2018
    44c9396
  22. locked as resolved and limited conversation to collaborators on Oct 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIssue identified by VS Code Team member as probable bugimportantIssue identified as high-priority

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions