Repository navigation
Python Testing: Codelens above parametrized tests doesn't open dropdown for selecting tests #8627
Description
- VS Code version: 1.40.1
- Extension version: 2019.10.44104
- OS and version: Windows 10 Education 1909
Expected behaviour
Clicking on the codelens above a parametrized test function/method or a whole test class (Run Test (Multiple) | Debug Test (Multiple)) opens a dropdown where one can select the test to run/debug.
Actual behaviour
When clicking on the codelens above a parametrized test function/method or a whole test class (Run Test (Multiple) | Debug Test (Multiple)) nothing happens.
Steps to reproduce:
On windows 10 (!):
Take any python package repository that contains tests and uses pytest, discover the tests and click on the appearing codelens above a parametrized test.
Note
There is no error message displayed in the various output panels (Protocol, Python Testing Log or others). Simply nothing happens.... Very confusing.
BUT!
While trying to find out the source of the error (because i like this codelens feature for running/debugging a single test very much) i cloned the vscode-python repo and set it up to debug the extension. I found out that the current code can't find a match between fileName and item.name or item.fullPath in the method displayFunctionTestPickerUI from the class TestDisplay. In my case fileName had the value c:\somepath\some_test_file.py and item.fullPath C:\somepath\some_test_file.py. The mismatch was the drive letter. When the drive letter is set to uppercase in fileName it works as expected and the dropdown menu is shown. Otherwise testFile is undefined an the method returns.
Maybe this helps sorting out this bug. I did not find out exactly where the path from tests.TestFiles[]/item.fullPath is generated but after some investigation i assume that it comes from the discover python module in the folder pythonFiles.
Below the source code of that method:
public displayFunctionTestPickerUI(cmdSource: CommandSource, wkspace: Uri, rootDirectory: string, file: Uri, testFunctions: TestFunction[], debug?: boolean) {
const tests = this.testCollectionStorage.getTests(wkspace);
if (!tests) {
return;
}
const fileName = file.fsPath;
const testFile = tests.testFiles.find(item => item.name === fileName || item.fullPath === fileName);
if (!testFile) {
return;
}
const flattenedFunctions = tests.testFunctions.filter(fn => {
return fn.parentTestFile.name === testFile.name &&
testFunctions.some(testFunc => testFunc.nameToRun === fn.testFunction.nameToRun);
});
this.appShell.showQuickPick(buildItemsForFunctions(rootDirectory, flattenedFunctions, undefined, undefined, debug),
{ matchOnDescription: true, matchOnDetail: true })
.then(testItem => testItem ? onItemSelected(this.commandManager, cmdSource, wkspace, testItem, debug) : Promise.resolve());
}Worth to mention that this works as expected when using vscode-python inside WSL.
Activity
Decapitalize drive letter in rootFolder during test collection
Decapitalize drive letter in rootFolder during test collection
ericsnowcurrently commentedon Nov 19, 2019
Philipp Loose (@phloose), thanks for letting us know about this and for being so proactive about finding a solution. Just to be sure I've understood, the problem is that the code lens tries to run a test with the wrong casing (for the drive letter)? This isn't specific to parameterized tests, is it?
Also, popping up a picker UI for parameterized tests sounds more like a feature request. I recommend opening a separate issue for that one.
DonJayamanne commentedon Nov 19, 2019
This is how it used to behave (displaying a pocket for parameterized tests). Not sure what had changed in the extension.
phloose commentedon Nov 19, 2019
The problem i discovered is that the listed code tries to match a path (returned from
Uri.fsPath) for an actual testfile, in which a codelens is used on a parametrized test, with a path for a testfile which was build during test collection. TherebyUri.fsPathreturns a lowercase drive letter for a windows path (like described in the api docs) while the path generated during test collection has an uppercase drive letter. The rest of both path is exactly the same.This applies ONLY to parametrized tests/tests that have a codelens that shows Run Test (Multiple), other tests work as expected.
Yeah i also remember that it worked correctly. In WSL/Linux and MacOS it works as expected.
Decapitalize drive letter in rootFolder during test collection
Add news entry for microsoft#8627
ericsnowcurrently commentedon Nov 25, 2019
Ah, thanks for clarifying. I was able to reproduce the problem using parameterized tests (only on WIndows). This is a know issue with VS Code. In the meantime, I expect the extension code needs to use
FileSystem.arePathsSame()(e.g. indisplayFunctionTestPickerUI).FWIW, here's what I tried, with the following code in a top-level file:
Linux
Test discovery worked fine and the codelens showed up. When I clicked on
Run Test (Multiple), a picker popped up with the 2 parameterized values as options. Selecting one of them ran the tests successfully.Windows
Test discovery worked fine and the codelens showed up. When I clicked on
Run Test (Multiple), nothing happened.6 remaining items
phloose commentedon Nov 25, 2019
Exactly as i discovered. This happens ONLY on Windows.
For a possible fix look at my pull request #8647. It doesn't use that method (which just uppercases all path elements for the comparison when the platform is windows) but cuts of only the drive letter and uppercases that. There i should also add that this should only be done if the path is NOT relative, otherwise strange thinks happen...
ericsnowcurrently commentedon Nov 25, 2019
I've left a review on your PR.
Compare paths of testfiles with FileSystem.arePathsSame
Add unit test for microsoft#8627
Compare paths of testfiles with FileSystem.arePathsSame
Add unit test for microsoft#8627
Add unit and functional tests for microsoft#8627
Add unit and functional tests for microsoft#8627
Fix codelens action for opening a dropdown on parametrized tests in w…