Cancel test run doesn't seem to work

Testing #21494

I changed the parameterized test to go from 1 to 10k to 1 to 1k, and then pressed the run button, expecting the test changes to be picked up. However, I noticed the parameterized test was going past 1k because I had the tests expanded while the tests were running. I tried to cancel, but pressing cancel did nothing. I notice now that pressing cancel seems to work after the parameterized test finishes running. I'm wondering if the token can be checked per iteration of a parameterized test?

Noticed the same behavior with vscode crashing after I hit Cancel Test Run in succession. I changed the parametrized test to go from 1 to 10k.

Connor Peet (@connor4312)

On further testing with 10k tests I have found that the cancelation token is called immediately once the stop button is clicked followed by the TestRun object getting set to testRun.end(). I am not sure entirely about the delay, since I see one too and it seems to occur following the cancelation token call. When I observe it locally my extension logs print Setting the state of test "param_folder/test_parameterized.py::test_odd_even[93]" is a no-op after the run ends. for every test that was not finished and instead canceled. This error comes from here. I do notice that it spins for as long as it takes to print these logs which then stop spinning once it has gotten to the final one log but this is likely specific for the debugging mode I was in. Connor Peet (@connor4312) do you have any thoughts?

This is at parity with the current testing infrastructure (rewrite vs non-rewrite code) so it is less urgent to get it out for this release but would be good to get to the bottom of. Thanks!

It seems like the console logs are pretty slow, but there's also a UI stall when ending all those tests. I will have a PR for the latter, and the Python extension should avoid setting states after calling end()

testing: improve performance when ending test with large number of re…

testing: fix state not updating when failing peek opens (#186802)

Connor Peet (@connor4312)

Connor Peet (@connor4312) I thought we did avoid setting states after end() was called. Was there an instance you saw this was not the case?

The console log "Setting the state of test "param_folder/test_parameterized.py::test_odd_even[93]" is a no-op after the run ends" is triggered by setting the state after end gets called. I also observed those logs when I was looking into the issue.

Eleanor Boyd (@eleanorjboyd)

Eleanor Boyd (@eleanorjboyd) I am having the same issue. If I Cancel the Test Run it takes more than 2 minutes for the timer to stop and for the Run Tests button to reappear.

By creating an empty project and using pytest with a test like so:

import time


def test_slow():
    time.sleep(10)
    assert True == True

and clicking Cancel Test Run right after starting, the timer will stop after 10 secs have passed. Setting the time to any other number will lead to the run not being cancelled until such number of seconds has passed.

It seems the test run is actually not cancelled.

Merged PR posit-dev/positron-python#213: Merge python extension relea…

closed by: #23262

also by work to move to using named pipes

Verification:
follow raymond's original comment, let me know how cancelation is and if it feels reactive

It still seems to return the counter to 0/0 and keep the spinner going sometimes.
edit: it stopped at 93.2 seconds, which I can understand considering I changed the parameterized test to run from 1 to 10k, meaning the test runner was loading over 10k tests.
I'll consider the issue verified.

Raymond Zhao (@rzhao271)

Raymond Zhao (@rzhao271) thank you! Yes I think it is just the time it takes to reset all the test items.

Navigation Menu

Repository navigation

Cancel test run doesn't seem to work #21507

45 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Cancel test run doesn't seem to work #21507

Description

Activity

bhavyaus commented on Jun 27, 2023

eleanorjboyd commented on Jun 29, 2023

connor4312 commented on Jun 30, 2023

eleanorjboyd commented on Jul 7, 2023

connor4312 commented on Jul 7, 2023

45 remaining items

edubermejo commented on Feb 1, 2024

eleanorjboyd commented on Apr 23, 2024

eleanorjboyd commented on Apr 23, 2024

rzhao271 commented on Apr 24, 2024

eleanorjboyd commented on Apr 24, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions