New pythonTestAdapter does not work with unittest.SkipTest

Type: Bug

Behaviour

Expected vs. Actual

The new pythonTestAdapter errors out instead of skipping the test.

Steps to reproduce:

Have a file like this in your test folder:

from unittest import SkipTest

raise SkipTest("This is unittest.SkipTest calling")


def test_example():
    assert 1 == 1

Diagnostic data

Python version (& distribution if applicable, e.g. Anaconda): 3.10.12
Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
Value of the python.languageServer setting: Pylance

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-07-18 20:47:13.010 [error] pytest test discovery error 
 unittest.case.SkipTest: This is unittest.SkipTest calling
 Check Python Test Logs for more details.

User Settings


languageServer: "Pylance"

linting
• mypyArgs: "<placeholder>"
• mypyEnabled: true

formatting
• provider: "black"

testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true

experiments
• optInto: ["pythonTestAdapter"]

Extension version: 2023.12.0
VS Code version: Code 1.80.1 (74f6148eb9ea00507ec113ec51c489d6ffb4b771, 2023-07-12T17:22:25.257Z)
OS version: Linux x64 6.2.6-76060206-generic
Modes:

System Info

Item	Value
CPUs	AMD Ryzen 9 5900X 12-Core Processor (24 x 4112)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	2, 2, 2
Memory (System)	125.71GB (99.49GB free)
Process Argv	/home/shh/Development/holoviz/holoviz.code-workspace
Screen Reader	no
VM	0%
DESKTOP_SESSION	pop
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	pop
XDG_SESSION_TYPE	x11

Can you describe what you mean by erroring? When I try this I see it listed as "not run". This is expected because the extension doesn't know the test exists because the skipTest line throws an exception. That folder then can't run since it has no tests and cannot return a skipped value for the icon. Let me know if I misunderstood your bug report, thanks!

Did you have other test files with test functions in the folder?

They should be discoverable, right?

So if you raise a skipTest in the file but not in a function/ class specifically it will skip the whole test file.

for example:

from unittest import SkipTest

raise SkipTest("This is unittest.SkipTest calling")


def test_example():
    assert 1 == 1

def test_two():
    assert True

then if you run unittest from the command line you can see test_folder_unit_skip (unittest.loader.ModuleSkipped) ... skipped 'This is unittest.SkipTest calling' and so it wont recognize the individual tests since it stops when looking in the whole file.

as opposed to

import unittest


def add(x, y):
    return x + y


class SimpleTest(unittest.TestCase):
    @unittest.skip("demonstrating skipping")
    def testadd1(self):
        self.assertEquals(add(4, 5), 9)

class OtherTest(unittest.TestCase):
    def testtwo(self):
        self.assertEquals(add(4, 5), 9)

if __name__ == "__main__":
    unittest.main()

where the testadd1 will be discovered (and labeled as skipped) and testtwo will be discovered and can run since the skip is inside the class. Then if I put these in separate files, the testadd1 will be found but skipped and testtwo will be found and run (so same deal just under different file nodes in the same folder).

I did find one issue with unittest discovery for skip and so I have submitted it here: #21665. This won't change run behavior but will fix the test explorer in that the node isn't correct for file level skip tests. So now if you have root/skip_test_at_file_level and root/no_skip then the node root should exist with two children (which is not happening right now instead there are two top level nodes one of them called "unittest")

I'm seeing this problem when running with pytestEnabled, this is my settings.json:

{
    "python.testing.pytestArgs": [
        "."
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true
}

And I have these two files:

# test/test_example.py
from unittest import SkipTest

raise SkipTest("This is unittest.SkipTest calling")


def test_example():
    assert 1 == 1

# test/test_example2.py
def test_example():
    assert 1 == 1

And in the Python Ouput I get this:

2023-07-20 10:06:36.509 [error] pytest test discovery error 
 unittest.case.SkipTest: This is unittest.SkipTest calling
 Check Python Test Logs for more details.

If I comment out the raise SkipTest I get this:

able to repro now- sorry my settings were wrong. I will put a fix in for this soon! Thanks

putting this here with the info I have. Might be delayed a bit getting to it as this is not a regression and this behavior didn't work before either.

With pytest enabled, files with raise SkipTest("This is unittest.SkipTest calling") should not error instead discovery should work correctly with those tests not being discovered since it is skipped at the file level. Below is a test case that should result in the file being shown as a node but not the error or any tests in the folder. This is not a regression since the old code does not handle this but it discovery should work correctly as you can test in the command line.

import unittest
from unittest import SkipTest

raise SkipTest("This is unittest.SkipTest calling")


class SimpleTest(unittest.TestCase):
    @unittest.skip("demonstrating skipping")
    def testadd1(self):
        assert True

handle skip unittest at file without error (#21678)

Merged PR posit-dev/positron-python#213: Merge python extension relea…

Navigation Menu

Repository navigation

New pythonTestAdapter does not work with unittest.SkipTest #21653

Behaviour

Expected vs. Actual

Steps to reproduce:

Diagnostic data

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

New pythonTestAdapter does not work with unittest.SkipTest #21653

Description

Behaviour

Expected vs. Actual

Steps to reproduce:

Diagnostic data

Activity

eleanorjboyd commented on Jul 19, 2023

hoxbro commented on Jul 19, 2023

eleanorjboyd commented on Jul 19, 2023

hoxbro commented on Jul 20, 2023

eleanorjboyd commented on Jul 21, 2023

eleanorjboyd commented on Jul 21, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions