Hard and Soft Assertions in test automation

One of the rules of good test automation is to write small fast atomic tests. In other words, small tests, that execute only one test case. And usually, one test case should have only one verification. However, the exception is when we make a few verifications without additional test steps in between verifications and they do not depend on each other (can be executed in any order).

Lets look at Test case 1: Positive LogIn test on our Practice Test Login page.

1. Open page: https://practicetestautomation.com/practice-test-login/
2. Type username student into the [Username field]
3. Type password Password123 into the [Password field]
4. Puch [Submit button]
5. Verify new page URL contains practicetestautomation.com/logged-in-successfully/
6. A new page should contain expected text (‘Congratulations’ or ‘successfully logged in’)
7. Verify button [Log out] is displayed on the new page

Steps 5, 6, and 7 are verifications, and they do not depend on each other. As a result, we can execute these verifications in any order.

In order to execute all three verifications without failing tests, we can use soft assertions instead of hard assertions.

Soft Assert vs Hard Assert

A Hard Assertion is a type of assertion that throws an exception immediately when an assert statement fails. Test steps after hard assertion will not be executed and the next test in the test suite will start.

Soft Assertions are the type of assertions that do not throw an exception immediately when an assertion fails. Therefore, all steps and soft assertions in the automated test will execute before failing the test.

Let’s assume the URL and Success message changed, but the logout button is still there. If we use hard assertions, the test will fail immediately after the URL verification, and all we would know is that the URL changed. We will have to fix the test (change expected URL) and run it again, just to learn that the success message changed too.

If we use soft assertions, we will know that steps 5 and 6 failed, but step 7 passed. So we will be able to fix the expected result for 5 and 6, and that would fix our test. We will not have to rerun the test just to learn that there is something else broken.

Example using Soft and Hard assertions

Different test frameworks implement soft assertions in a different way. Let’s look at the example of how to do this with the TestNG Java testing framework.

driver.get("https://practicetestautomation.com/practice-test-login/");

driver.findElement(By.id("username")).sendKeys("student");
driver.findElement(By.name("password")).sendKeys("Password123");
driver.findElement(By.cssSelector("button#submit").click();

String expectedUrl = "https://practicetestautomation.com/logged-in-successfully/";
String actualUrl = driver.getCurrentUrl();
Assert.assertEquals(actualUrl, expectedUrl, "Actual page url is not the same as expected");

WebElement successMessage = driver.findElement(By.tagName("strong"));
String expectedMessage = "Congratulations Michael Scott. You successfully logged in!";
String actualMessage = successMessage.getText();
Assert.assertTrue(actualMessage.contains(expectedMessage), "Actual message does not contain expected message.\nActual Message: " + actualMessage + "\nExpected Message: " + expectedMessage);

WebElement logOutButton = driver.findElement(By.linkText("Log out"));
Assert.assertTrue(logOutButton.isDisplayed(), "Log Out button is not visible");

This example is using hard assertions. Therefore, the test will fail immediately when the first assertion fails. Instead, let’s see how to change this code to use soft assertions.

The first step is to create an instance of SoftAssert class.

SoftAssert softAssert = new SoftAssert();

After this, we can use this softAssert variable instead of hard assert.

softAssert.assertEquals(actualUrl, expectedUrl, "Actual page url is not the same as expected");

And at the end of the test we need to execute/finalize all soft assertions.

softAssert.assertAll();

Lets see how our code will look now after replacing hard assertions with soft assertions.

driver.get("https://practicetestautomation.com/practice-test-login/");

driver.findElement(By.id("username")).sendKeys("student");
driver.findElement(By.name("password")).sendKeys("Password123");
driver.findElement(By.cssSelector("button#submit").click();

SoftAssert softAssert = new SoftAssert();

String expectedUrl = "https://practicetestautomation.com/logged-in-successfully/";
String actualUrl = driver.getCurrentUrl();
softAssert.assertEquals(actualUrl, expectedUrl, "Actual page url is not the same as expected");

WebElement successMessage = driver.findElement(By.tagName("strong"));
String expectedMessage = "Congratulations Michael Scott. You successfully logged in!";
String actualMessage = successMessage.getText();

softAssert.assertTrue(actualMessage.contains(expectedMessage), "Actual message does not contain expected message.\nActual Message: " + actualMessage + "\nExpected Message: " + expectedMessage);
WebElement logOutButton = driver.findElement(By.linkText("Log out"));
softAssert.assertTrue(logOutButton.isDisplayed(), "Log Out button is not visible");

softAssert.assertAll();

After running the above code, all assertions will be executed, and we will see that steps 5 and 6 failed, and verification on step 7 passed. This will make it easier for us to debug and fix our test.

Learn more about Soft Assertions and many more cool techniques in my Advanced Selenium WebDriver with Java and TestNG course