Soft Assert in Selenium | Difference between Hard and Soft Assert

Soft Assert in Selenium

We have been using the hard assert in our previous tutorials. Assert.fail() is such an example of hard assert which we have used in THIS post. In this tutorial, we will learn what is a soft assert in selenium and how it is different than hard assert.

Assert is a class provided by TestNG to work with Selenium. Asserts are used to validate the major checkpoints of the test case. Assert class provides certain methods which help in validating these checkpoints. For example, we can use assert to verify if two given strings are equal, compare 2 boolean values etc.

Difference between Hard Assert and Soft Assert

Hard Assertion Soft Assertion
It does not allow further execution of test if the line containing hard assert gets failed. Next steps would be executed even if the line containing soft assertion gets failed.
Whole test case gets failed if at least 1 hard assert fails. Extra lines of code are required to track the fail status. When used in raw format, failed test step would also yield in Pass test script.

Let’s understand these assertions using an example. Consider the following 2 tests:

  • @Test1: Verify the login functionality.
  • @Test2: Fill the registration form.

 

Hard Assert

Create a package asserttest and add a class AssertTest to it. Write the following code in this class.

Setup TestNG in eclipse

click HERE
package asserttest;

import org.testng.Assert;
import org.testng.annotations.Test;

public class AssertTest {
    
    @Test(priority=1)
    public void testLogin() {
        System.out.println("Browser Opened");
        
        //Hard Assert
        Assert.assertEquals(true, false);

        //This line won't be printed
        System.out.println("Login Page displayed");
        System.out.println("Title Verified");
        
    }
    
    @Test(priority=2)
    public void register() {
        System.out.println("click on register");
        
        //We can also print messages. Messages are only printed when assert fails
        Assert.assertTrue(false, "This is also hard assert");
        
        //This line won't be executed
        System.out.println("Register page opened");
        
    }

}

  • Assert.assertEquals compare two object values (it could be string, int etc.) and result in pass if both values are equal, else it will fail the test case and no further lines of code would be executed for this test.
  • Assert.assertTrue accepts a boolean value and a string. This string will be printed if the condition gets fail. In this case also, no further lines of code would be executed for this test and the test script would be marked as fail.

Assert class provides many more methods. You may explore them below.

 

All Assert Class Methods

click Here

Create testng.xml for our test

Please click HERE if you have not already gone through testng.xml creation tutorial. Create a file testng_assert.xml at the root of your project and add below lines of code:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="SeleniumTestNG suite">

    <test name="assert">
        <classes>
            <class name="asserttest.AssertTest">

            </class>
        </classes>
    </test>

</suite>


Hard Assert Result

Execute the above XML and you should get following output:

Hard Assert

It is evident from the result that:

  • No further lines of code are executed of @Test if the hard assert gets failed.
  • It would instantly mark the test script fail.
  • In case of failures, our custom message would be printed.

Need of Soft Assert?

You would perhaps be guessing that hard assert is quite aggressive and there are situations where we want selenium to keep executing the test even if certain conditions get fail.

For instance, one of the test steps is validating the page title and our script gets failed for this step. We would still want to execute subsequent steps. But if the test case is of login and it is getting failed, then there is no point in executing other scripts and hard assert is perfect for such scenarios.

Below is the soft assert implementation on the same code. The updated code is highlighted in bold:

package asserttest;

import org.testng.Assert;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;

public class AssertTest {
    
    //Create an object of SoftAssert
    SoftAssert sa = new SoftAssert();
    
    @Test(priority=1)
    public void testLogin() {
        System.out.println("Browser Opened");
        
        //Soft Assert
        sa.assertEquals(true, false);

        // These lines will be printed
        System.out.println("Login Page displayed");
        System.out.println("Title Verified");
        
    }
    
    @Test(priority=2)
    public void register() {
        System.out.println("click on register");
        
        //We can also print messages. Messages are only printed when assert fails
        sa.assertTrue(false, "This is also hard assert");

        
        //This line will be executed
        System.out.println("Register page opened");
        
    }

}

Create an object reference of SoftAssert class at the global level (class variable). Now this object reference has access to every method of Assert class.  The output of testng_assert.xml is:

Soft Assert Result All Pass

The problem with soft assert is that it will pass the whole test script even if any step fails. In the above image, both test cases are pass and every statement is printed on the console. Also, it does not display the custom message which should be printed upon test failure.

To overcome this problem, we will use assertAll method at the end of each @Test. Update code is highlighted in bold:

 

package asserttest;

import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;

public class AssertTest {
    
    //Create an object of SoftAssert
    SoftAssert sa = new SoftAssert();
    
    @Test(priority=1)
    public void testLogin() {
        System.out.println("Browser Opened");
        
        //Soft Assert
        sa.assertEquals(true, false);

        //These line will be printed
        System.out.println("Login Page displayed");
        System.out.println("Title Verified");
        
        //assertAll will update the test status and custom messages in TestNG report
        sa.assertAll();
        
    }
    
    @Test(priority=2)
    public void register() {
        System.out.println("click on register");
        
        //We can also print messages. Messages are only printed when assert fails
        sa.assertTrue(false, "This is also hard assert");

        
        //This line will be executed
        System.out.println("Register page opened");
        sa.assertAll();
        
    }

}

The assertAll() method checks the sa object reference for any assert. If it finds any assert failure, it updates the test result and marks the script fail.

Below is the output of the final program:

Soft Assert Correct Result

The downside of using Soft Assert is that if there are hundreds of tests, then we have to write assertAll() at the end of each test.

You may use multiple hard assert and soft assert inside the same @Test. 

Author: Dhawal Joshi

A post-graduate in MCA, ISTQB & ITIL certified QA with more than 8 years of experience in QA working with a CMMI Level 5 organization as System Analyst. I started my automation journey with HP UFT(formerly known as QTP) and for the past few years, I am using Selenium for automation. I also have experience in Android Application Development, Java, HTML, and VBScript. When I am not working, I like to spend time with my family, cooking and learning new developments in IT.

8 thoughts on “Soft Assert in Selenium | Difference between Hard and Soft Assert

  1. Does your website have a contact page? I’m having problems locating it but, I’d like to shoot you an email.
    I’ve got some suggestions for your blog you might
    be interested in hearing. Either way, great website
    and I look forward to seeing it improve over time. Hi
    there! Someone in my Myspace group shared this website with us so I came to check it out.

    I’m definitely loving the information. I’m book-marking and will be tweeting this to my followers!
    Great blog and wonderful design and style.

    These are in fact fantastic ideas in concerning blogging.
    You have touched some good points here. Any way keep up wrinting.

    http://alienware.com

  2. It’s perfect time to make some plans for the future and
    it’s time to be happy. I’ve read this post and if I could I
    desire to suggest you few interesting things or advice.
    Perhaps you could write next articles referring to this article.
    I desire to read more things about it! I will immediately seize your rss feed as I can not in finding your email subscription hyperlink or e-newsletter service.
    Do you’ve any? Please allow me understand in order that I
    could subscribe. Thanks. I really love your site..
    Pleasant colors & theme. Did you create this site yourself?

    Please reply back as I’m attempting to create my own website
    and would love to learn where you got this from or just what the theme is called.
    Many thanks! http://foxnews.net

  3. Hi there! This blog post could not be written any better! Looking through this article reminds me of my previous roommate! He continually kept talking about this. I’ll send this information to him. Pretty sure he will have a very good read. I appreciate you for sharing!

  4. After looking into a handful of the blog articles on your site, I honestly like your technique of writing a
    blog. I saved as a favorite it to my bookmark webpage list
    and will be checking back in the near future.
    Please check out my website as well and tell me how you feel.

  5. Hey great website! Does running a blog similar to
    this take a great deal of work? I have no knowledge of computer programming however I
    was hoping to start my own blog in the near future.

    Anyway, should you have any ideas or tips for new blog owners please share.
    I understand this is off topic however I simply had to ask.
    Many thanks!

Leave a Reply

Your e-mail address will not be published. Required fields are marked *