Failure details in unittest discovery are not always logged. #5889

Activity

1. Eric Snow (ericsnowcurrently)

   added

   bugIssue identified by VS Code Team member as probable bugIssue identified by VS Code Team member as probable bug

   needs PR

   area-testing

   reason-new feature

   on Jun 3, 2019
2. Don Jayamanne (DonJayamanne)

   added

   reason-preexisting

   and removed

   reason-new feature

   on Jun 4, 2019
3. karrtikr commented on Jul 29, 2021

   

   Kartik Raj (karrtikr)

   on Jul 29, 2021

   Last edited by karrtikr

   More actions

   If we have an invalid test to load, say test_2.py.

   """This unit test module can not be discovered.
   """
   import unittest
   x = y +2 # Invalid line as `y` is not defined
   
   
   class Test_ImportOuterModule(unittest.TestCase):
       def test_2_1(self):
           self.assertEqual(1, 2)
           return
   
       def test_2_2(self):
           self.assertEqual(1, 1)
           return
   
   
   if __name__ == "__main__":
       unittest.main()

   The testing script

   "C:/Users/karraj/AppData/Local/Programs/Python/Python36/python.exe" -c "
   import unittest
   import inspect
   
   def get_sourceline(obj):
       s, n = inspect.getsourcelines(obj)
       for i, v in enumerate(s):
           if v.strip().startswith('def'):
               return str(n+i)
       return '*'
   
   def generate_test_cases(suite):
       for test in suite:
           if isinstance(test, unittest.TestCase):
               yield test
           else:
               yield from generate_test_cases(test)
   
   loader = unittest.TestLoader()
   suite = loader.discover("./package/tests", pattern="*test*.py")
   
   print("start") #Don't remove this line
   for s in generate_test_cases(suite):
       tm = getattr(s, s._testMethodName)
       print(s.id().replace('.',':') + ":" + get_sourceline(tm))
   "

   returns

   start
   unittest:loader:_FailedTest:test_1:33
   unittest:loader:_FailedTest:test_2:33
   

   The last 2 entries indicate that the test nodes test_1 and test_2 were not loaded, this should probably be surfaced.
4. Kartik Raj (karrtikr)

   added

   verification-foundIssue verification failedIssue verification failed

   on Jul 29, 2021
5. karthiknadig commented on Aug 3, 2021

   

   Karthik Nadig (karthiknadig)

   on Aug 3, 2021

   Last edited by karthiknadig

   Member

   More actions

   With the latest test-UI this is addressed:
   
6. Karthik Nadig (karthiknadig)

   added

   verifiedVerification succeededVerification succeeded

   and removed

   verification-foundIssue verification failedIssue verification failed

   on Aug 3, 2021
7. Karthik Nadig (karthiknadig)

   added this to the August 2021 milestone on Aug 5, 2021
8. Karthik Nadig (karthiknadig)

   closed this as completedin #16769on Aug 5, 2021
9. github-actions

   locked as resolved and limited conversation to collaborators on Sep 16, 2021