feat(awesome): indicate retry status changes separately - #978
formaceft-93 wants to merge 2 commits into
Conversation
24de585 to
f0395dd
Compare
Allure Report Summary
Reports: Allure2, Awesome, Classic, Dashboard Artifacts used (3)
|
f0395dd to
afed8c5
Compare
There was a problem hiding this comment.
Summary title
I have two issues with the summary title:
- The text is quite long.
- It reads as we're counting status transitions across retries rather than tests with at least one such transition.
I don't have a perfect text in mind. Something like "Changed on retry" fits better (implying "Tests changed on retry"). Maybe an even better alternative exists.
Test-level indicator
The indicator itself is good. There is one minor hiccup though.
Imagine a test with two retries. As long as all retries pass, the test is displayed with the retries indicator and with the total number of retries:
But as soon as we have at least one failed/broken attempt, we lose the information about the number of retries:
But I don't have a good solution in mind. I guess we can left it as is for now. Perhaps, we shouldn't have added these number to begin with.
afed8c5 to
8f5cc7f
Compare
Thanks, agree on both points. I shortened the summary label to “Changed on retry”, and updated the tree indicator so it no longer replaces the retry count. Now tests with changed retry status show both pieces of info: the usual retry count and the status-change marker. |
8f5cc7f to
2ba442f
Compare
This branch has not been deployed
2 participants
Summary
Adds a separate
retriesStatusChangesignal for tests whose visible/latest attempt has a different significant status than at least one retry attempt.This keeps the existing retry indicators and retry totals for measuring retry-policy coverage, while adding a separate way to spot tests that actually changed outcome across attempts.
Semantics
retriesStatusChangeis true when:passed,failed, orbroken;skippedandunknowndo not create a status change and are not used as comparison bases.Issue: #224
Checklist