Repository navigation
Speed up linting #444
Description
Environment data
VS Code version: 1.19.0 (1.19.0)
Python Extension version: 0.9.0
Python Version: Anaconda 2.7.14
OS and version: mac OS 10.13.2
Actual behavior
Updates to Pylint messages are very slow even after saving, it takes several seconds for messages to change. For example, if a variable is undefined and then defined, the linter will take several seconds to update it's undefined variable message.
Expected behavior
Messages should update instantly when the linter is run.
Steps to reproduce:
- Have a Pylint message
- Fix the error associated with the message
- Save to lint
- Message will take some time to disappear
Logs
Output from Python output panel
##########Linting Output - pylint##########
No config file found, using default configuration
************* Module scheduler
304,0,convention,C0303:Trailing whitespace
313,0,convention,C0303:Trailing whitespace
316,0,convention,C0303:Trailing whitespace
1,0,convention,C0111:Missing module docstring
73,0,convention,C0111:Missing function docstring
78,4,convention,C0103:Invalid variable name "df"
84,4,convention,C0103:Invalid variable name "df"
107,0,convention,C0111:Missing function docstring
114,0,convention,C0103:Invalid argument name "df"
114,0,convention,C0111:Missing function docstring
116,11,convention,C0103:Invalid variable name "c"
120,0,convention,C0103:Invalid argument name "m"
120,0,convention,C0111:Missing function docstring
122,4,refactor,R1705:Unnecessary "else" after "return"
122,19,error,E1101:Module 'gurobi' has no 'GRB' member
124,21,error,E1101:Module 'gurobi' has no 'GRB' member
135,0,convention,C0111:Missing function docstring
135,21,warning,W0613:Unused argument 'data'
135,27,warning,W0613:Unused argument 'prediction_file'
141,0,convention,C0111:Missing function docstring
141,0,refactor,R0914:Too many local variables (36/15)
164,4,convention,C0103:Invalid variable name "m"
164,8,error,E1101:Module 'gurobi' has no 'Model' member
167,4,convention,C0103:Invalid variable name "x"
167,53,error,E1101:Module 'gurobi' has no 'GRB' member
168,4,convention,C0103:Invalid variable name "z"
169,4,convention,C0103:Invalid variable name "f"
184,8,convention,C0103:Invalid variable name "t"
188,14,error,E1101:Module 'gurobi' has no 'quicksum' member
202,8,convention,C0122:Comparison should be gb.quicksum([le for le in x.loc[data[PACU_FLAG],:].mul(late_pacu).values.flatten() if le.getCoeff(0) != 0.0]) == 0
202,13,error,E1101:Module 'gurobi' has no 'quicksum' member
209,8,convention,C0103:Invalid variable name "ti"
209,12,convention,C0103:Invalid variable name "t"
212,12,convention,C0103:Invalid variable name "r"
214,22,error,E1101:Module 'gurobi' has no 'quicksum' member
223,9,convention,C0122:Comparison should be x.loc[(p, data.loc[(p, PRIOR_START_TIME)])] == 1
230,8,convention,C0122:Comparison should be gb.quicksum([le for le in x.loc[~data[FIXED_START],:].mul(too_early).values.flatten() if le.getCoeff(0) != 0.0]) == 0
230,13,error,E1101:Module 'gurobi' has no 'quicksum' member
240,15,error,E1101:Module 'gurobi' has no 'GRB' member
245,16,convention,C0103:Invalid variable name "fv"
251,16,convention,C0103:Invalid variable name "zv"
255,16,convention,C0103:Invalid variable name "fv"
260,16,convention,C0103:Invalid variable name "fv"
267,16,convention,C0103:Invalid variable name "p"
268,23,convention,C0103:Invalid variable name "t"
141,0,refactor,R0912:Too many branches (15/12)
141,0,refactor,R0915:Too many statements (55/50)
274,0,convention,C0111:Missing function docstring
275,16,convention,C0121:Comparison to True should be just 'expr' or 'expr is True'
289,0,convention,C0111:Missing function docstring
300,0,convention,C0111:Missing function docstring
2,0,warning,W0611:Unused datetime imported from datetime
------------------------------------------------------------------
Your code has been rated at 5.46/10 (previous run: 5.36/10, +0.10)
Output from Console window (Help->Developer Tools menu)
TypeError: Cannot read property 'TokenType' of undefined
at CompletionSource.isPositionInsideStringOrComment (/Users/mfairley/.vscode/extensions/ms-python.python-0.9.0/out/client/providers/completionSource.js:115:44)
at CompletionSource.<anonymous> (/Users/mfairley/.vscode/extensions/ms-python.python-0.9.0/out/client/providers/completionSource.js:78:22)
at Generator.next (<anonymous>)
at __awaiter (/Users/mfairley/.vscode/extensions/ms-python.python-0.9.0/out/client/providers/completionSource.js:9:71)
at Promise (<anonymous>)
at __awaiter (/Users/mfairley/.vscode/extensions/ms-python.python-0.9.0/out/client/providers/completionSource.js:5:12)
at CompletionSource.getCompletionResult (/Users/mfairley/.vscode/extensions/ms-python.python-0.9.0/out/client/providers/completionSource.js:68:16)
at CompletionSource.<anonymous> (/Users/mfairley/.vscode/extensions/ms-python.python-0.9.0/out/client/providers/completionSource.js:40:39)
at Generator.next (<anonymous>)
at __awaiter (/Users/mfairley/.vscode/extensions/ms-python.python-0.9.0/out/client/providers/completionSource.js:9:71)
(node:13752) DeprecationWarning: Calling an asynchronous function without callback is deprecated.
t.log @ /Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:249
11:18:45.522 console.ts:123 [Extension Host] (node:13752) DeprecationWarning: Calling an asynchronous function without callback is deprecated.
t.log @ console.ts:123
Activity
DonJayamanne commentedon Dec 17, 2017
I've created another issue to track the bug, #447
DonJayamanne commentedon Dec 17, 2017
Please could you give us an idea of how long this is?
mfairley commentedon Dec 17, 2017
About 10 seconds
MikhailArkhipov commentedon Feb 1, 2018
Michael Fairley (@mfairley) - Is time the same if you run pylint from command line?
We can probably improve this a bit by keeping pylint up rather than running it every time, but generally speed is limited by what it takes pylint to get through the file.
mfairley commentedon Feb 1, 2018
MikhailArkhipov commentedon Feb 6, 2018
We can save some time on not launching pylint every time, but I don't expect the process to be lightning fast. Pylint needs to fully parse all related files, build ASTs and analyze their structure. All that is done in Python. Depending on the code size it may take a while.
Ex full C# code analysis in VS C# solution (say, using Resharper) takes quite a while (~10 seconds+). There are different approaches such as building own AST in a compiled language with incremental updates (PyCharm comes to mind), but this is a large endeavor.
jeyanthimuthuram commentedon Feb 21, 2018
I don't know too much about the internals it, But how py-charm is doing linting on the real time ?
If possible why can't we use the same way they are using ? Just asking .
brettcannon commentedon Feb 21, 2018
Jeyanthinath Muthuram (@jeyanthimuthuram) as Mikhail Arkhipov (@MikhailArkhipov) said, we could do what PyCharm does but it requires re-implementing Pylint. So it's doable, but not an easy thing to do.
jeyanthimuthuram commentedon Feb 21, 2018
Brett Cannon (@brettcannon) I totally agree, but I feels like it should be one of the very key features of this plugin ( as python is now natively supported by vscode )
But I do toally agree there is a lot of work need to be put behind this.
Don Jayamanne (@DonJayamanne) Mikhail Arkhipov (@MikhailArkhipov) do we have any roadmap on this ?
brettcannon commentedon Feb 21, 2018
Jeyanthinath Muthuram (@jeyanthimuthuram) the roadmap is "we're working on it" 😉 , otherwise we don't announce anything more on purpose as these things have a tendency to slip since we're talking about such large, critical aspects of the extension.
jeyanthimuthuram commentedon Feb 21, 2018
Brett Cannon (@brettcannon) Thanks for the response, I know you guys can make it work 👍
Looking forward for this feature .
MikhailArkhipov commentedon Feb 21, 2018
Jeyanthinath Muthuram (@jeyanthimuthuram) - PyCharm is not using Jedi. JetBrains are professionals in languages :-) and build their own parsers. Same with Python Tools in VS or R Tools in VS that we own. Both have async parsing and code analysis as you type on unsaved files as do C#, C++ etc. We have code (but its in C#) and experience, but it takes time :-)
MikhailArkhipov commentedon Apr 3, 2018
New Python Analysis engine does syntax check as you type. Closing since the engine was integrated with #1231