Repository navigation
Bug: Symlinks in Testing #22658
Description
Please see the first comment on this issue for the proposed solution.
Description of Issue:
In investigating the symlink issue affecting Python testing in VS Code, I uncovered several contributing factors. The problem begins during the test discovery phase: the Python extension initiates a subprocess which uses the symlink as its current working directory (since the symlink or at least a path relative to the symlink workspace is set as the cwd). Then when node calls the python subprocess which run pytest discovery, Node resolves the symlink cwd to target path. This means the subprocess which pytest run on is on the target path of the symlinked workspace. Consequently, when the PyTest extension is activated, it encounters already-resolved symlinks. This leads to the collection of test data with paths relative to the symlink's target value, not its relative location. These paths are then utilized to populate the Test Explorer in VS Code. As a result, when a user attempts to open a file from the Test Explorer, VS Code erroneously navigates to the target path which is incorrect for the workspace since it is workspace opened at the location of a symlink. Also each test then has its path set as the target path to its file, instead of the symlink, meaning the test is no longer associated with the given workspace it is in.
Given these findings, a design decision is needed to address and rectify the symlink issue in the context of testing within VS Code. This decision will determine the most effective solution or workaround to ensure accurate and efficient test handling.
I will explore how other symlink items are handled in vscode but some ideas include:
- passing the symlink into the subprocess and having the plugin handle returning data relative to the symlink
- editing resulting data on the extension side to substitute symlinks in for the target link if present.
Test Case Design:
Below are the steps to make this problem occur for testing and as an example:
- in terminal cd to parent folder
parent - create folder inside parent
target_folderwith tests such astest_example.py(a file with a test inside) - from terminal create a symlink
ln -s target_folder symlink_to_target_folder - open vscode with the folder
symlink_to_target_foldercan be done withcode symlink_to_target_folderin terminal - run test discovery, test UI will be populated
- 🐛 the root folder will be
target_folderinstead ofsymlink_to_target_folderin the test explorer view - 🐛 click on "go to file" on a test, the incorrect file will be opened
- 🐛 try to run tests, nothing will happen
Notable Code:
-
const proc = spawn(file, args, spawnOptions);whenspawnOptionsin rawProcessApis.ts has a valuecwd. On thisspawncall, the symlink is resolved. -
here run requests are evaluated and tests included in the request and checked against known workspace folder to gather correct workspaces. This does not work correctly for symlinks. Code in controller.ts
if (request.include) {
uniq(request.include.map((r) => this.workspaceService.getWorkspaceFolder(r.uri))).forEach((w) => {
if (w) {
workspaces.push(w);
}
});
fs.lstatSync(workspaceFolderPath).isSymbolicLink();a way to check if a given workspace folder is a symlink, possible addition to the code.
Activity
eleanorjboyd commentedon Dec 18, 2023
Proposal for Fix (pytest)
On further investigation I have determined a potential fix to this issue. Below I have outlined how this fix may work as some decisions need to be made around the design and implementation.
As described in the issue description above, the problem stems from the extension running a python subprocess with the symlink as the cwd. The subprocess resolves the cwd to the target when it runs so the subprocess (pytest) has no knowledge it was even started in symlink directory.
A fix for this would be to used the
--rootdirflag for pytest to provide the symlink path to the root. Given the rootdir, the pytest plugin can check to see if a--rootdiris being provided and is a symlink. If so it can then go through and edit all path values to make them relative to the symlink path of the root. This will cause all paths to be correct and all folder creation / nesting to be based on the symlink.Specifics
I propose that before running the subprocess, the extension should check to see if the cwd is a symlink. If it is, then add the arg
--rootdirto the arguments for the pytest request with the rootdir being set to the symlink of the cwd. A question here is what should be done if the--rootdirargument is already specified by the user. For this case I propose to always use the rootdir supplied by the user. The user could have already added a symlink root themselves, this would be fine and would work. There are cases where the user provides a rootdir that is not a symlink but if they have their workspace opened in a symlink then that is already not congruent and they probably expect the rootdir they provided. Given this weird case probably just throw a warning of this and continue with their provided value.This extra work around symlinks should be notified to the user, a log should be included to state that the extension has a symlinked cwd and it is taking necessary steps.
Redoing all paths as we work through building the testing tree for discovery could add time. Since we are already doing the equivalent of a tree search to create "test nodes" for all files and parent folders to the session root, there should not be an additional time complexity but does add other computation item per node.
Outstanding questions:
These require exploration and invite user input.
Required Engineering:
Final question how many people who face this issue use unittest and couldn't use pytest. Looking for numbers here to determine the priority of unittest vs pytest for symlinks. Unittest will require a different solution and I have not investigated the complexity of this.
g24swint commentedon Jan 24, 2024
I followed the conversation in #22518 to this point, and this "symlink" issue seems to be my problem as well. Apart from a virtual "+1", I'm not sure I have much to add except that my symlink is outside / above the repository in which I'm working and is a symlink into an AWS EFS mount point. If I open VS code into my python package subdirectory of my repo using the symlink-to-EFS path, it will not run tests. If I open directly against the mount point in /mnt/efs, it does seem to run the tests correctly.
alanwilter commentedon Feb 12, 2024
Testing
vscode 1.86.1. The issue withTestsand symlink is still there.Interestingly,
vscodeat least now shows the tests, but trying to run them and nothing happens.eleanorjboyd commentedon Feb 12, 2024
Hi Alan Silva (@alanwilter), thanks for the update. The fix for symlinks hasn't been merged yet but it is almost done! I will let you know when it is merged in so you can try it in insiders, thanks!
Fix Bug with Pytest when using symlinked workspaces (#22885)
Merged PR posit-dev/positron-python#395: Merge upstream: `v2024.2.0`
Merged PR posit-dev/positron-python#395: Merge upstream: `v2024.2.0`
eleanorjboyd commentedon Apr 19, 2024
Hello everyone! I have made another fix to resolve this for scenarios where the workspace folder is not a symlink but the parent is. Please let me know if anyone sees problems if they try it on the newest version of insiders and the extension!