Allure is a powerful test report instrument (more about it here)
And there are quite a lot of improvements of implementation it into your test framework, and here is one more - annotation @skip
Allure support it's own line of statuses - PASS, FAIL, BROKEN, SKIPPED and UNKNOWN
(UPD: latest version has PASS, FAIL, BROKEN, CANCELLED and PENDING but it does not change much)
And if with PASS, FAIL, BROKEN statuses and how they are set all is obvious, with rest SKIPPED and UNKNOWNwe could work and handle them by our own logic.
And suggested logic is as follow:
If our test case at present moment fails because of some known issue, no reason to see it in each report as a fail, cause we know the problem, so it could be marked in the final report as a 'skipped' grey label.
And if in some moment of time this problem will be solved and the case starts to pass, we will know about it with a 'pending' purple label (this will mean, that we could remove our @skip annotation from this case).
All we need to do is just add implemented annotation @skip before our test method.
Profit👍
And here is an example of how to implement it in Java:
@After("@skip")
public void skipScenario(Scenario scenario) {
log.info("Test case is skipped.");
if (Allure.getLifecycle().getCurrentTestCase().isPresent()) {
String testCaseNameWithUuid = Allure.getLifecycle().getCurrentTestCase().get();
Pattern pattern = Pattern.compile("([a-f0-9]{8}(-[a-f0-9]{4}){4}[a-f0-9]{8})");
Matcher matcher = pattern.matcher(testCaseNameWithUuid);
String uuid = "";
while (matcher.find()) {
uuid = matcher.group(1);
}
if (scenario.isFailed()) {
Allure.getLifecycle().updateTestCase(uuid, testResult -> {
testResult
.setDescription("Test has skip reason (ex. open bug or env problem)")
.setStatus(io.qameta.allure.model.Status.SKIPPED);
});
}
else {
Allure.getLifecycle().updateTestCase(uuid, testResult -> {
testResult.setDescription("Skip reason solved (ex. open bug is fixed)");
testResult.setStatus(null);
});
}
}
else {
log.info("Test case runs locally.");
}
}