Execute Failed Test Cases In Selenium Using TestNG

Re-execute Failed Test Cases

It happens occasionally that a test which was working fine earlier gets failed during execution. If you execute it again, it would get pass. To overcome this issue TestNG provides a listener which would retry to execute failed test cases for a given number of times.

Name of this listener is IRetryAnalyzer and it contains 1 method retry(ITestResult result) which returns true if a test has to re-execute. Another interface IAnnotationTransformer is used to bind the invocation of the class associated with IRetryAnalyzer.

Learn TestNG Listeners

click here

Apart from the above two classes, we will have a test class containing @Test. You may find above two classes in TestNG documentation. No need to remember the logic implemented in these classes as you would only require to create them once and use in any project after minor modifications.

Check The Implementation Of Above 2 Class

Visit HERE

Implementing the Listeners

Create a java project, add package retryFailTest and create 2 classes RetryFailTestCase and RetryTransformListener. Following is the implementation of these 2 classes:

package retryFailTest;

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

// This class is used to force the re-execution of failed test cases

public class RetryFailTestCase implements IRetryAnalyzer {
	
	 private int retryCount = 0;
	 
	 //Max number of times you want to re-execute a test case
	 private static final int maxRetryCount = 1;


	@Override
	public boolean retry(ITestResult result) {
		if (retryCount < maxRetryCount) {
			System.out.println("Retrying test " + result.getName() + " with status "
                    + getResultStatus(result.getStatus()) + " for " + (retryCount+1) + " time(s).");
		      retryCount++;
		      return true;
		    }
		    return false;

	}
	
	//Get the result status
	public String getResultStatus(int status) {
    	String result = null;
    	if(status==1)
    		result = "SUCCESS";
    	if(status==2)
    		result = "FAILURE";
    	if(status==3)
    		result = "SKIP";
		return result;
    }

}
package retryFailTest;

import java.lang.reflect.Constructor;
import java.lang.reflect.Method;

import org.testng.IAnnotationTransformer;
import org.testng.IRetryAnalyzer;
import org.testng.annotations.ITestAnnotation;

public class RetryTransformListener implements IAnnotationTransformer {

	@Override
	public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) {

		IRetryAnalyzer analyzer = annotation.getRetryAnalyzer();

		if (analyzer == null) {
			annotation.setRetryAnalyzer(RetryFailTestCase.class);
		}

	}

}

As mentioned earlier, you would only need this code once, hence it is advised to save these files somewhere for later use. Once these classes are created, next we will create our test class.

The test class has 2 @Test:

  • testLogin(): This @Test would only print some text and should always get pass.
  • failTest(): This test should get re-executed upon failure.
package retryFailTest;

import org.testng.Assert;
import org.testng.annotations.Test;

public class TestClass {
	
	static int counter = 0;

	//This test will always pass
	@Test(priority=1)
	public void testLogin() {
		System.out.println("This is login test");
	}
	
	
	//This test will be executed maxRetryCount number of times, as defined in RetryFailTestCase.java
	@Test(priority=2)
	public void failTest() {
		counter++;
		System.out.println(counter +" Attempt");
		Assert.assertEquals(counter, 2);
		
	}
	
	
}

TestClass.java

failTest() will be executed 1 extra time as the value of maxRetryCount is 1 in RetryFailTestCase class.

  • Value of counter is set to 0.
  • Under failTest(), the value of counter is incremented by 1 and then system prints number of attempts.
  • Assert.assertEquals() would compare counter value (1) with the expected value (2).
  • In the first iteration, this test gets fail. Upon failure, IRetryAnalyzer listener would be invoked and this test is executed for the 2nd time (first retry attempt).
  • This time the value of counter and the expected value is same, hence the test should get pass.

To execute this test we will create a testng_rerunfail.xml as listeners must always be executed using an XML file.

Creating the testng.xml

Please visit THIS tutorial if you are not familiar with testng.xml and click HERE to learn how to implement listeners in testng.xml.

Below is the code to implement the listener in testng_rerunfail.xml.

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="SeleniumTestNG suite">
	<listeners>
		<listener class-name="retryFailTest.RetryTransformListener" />
	</listeners>

	<test name="listeners">
		<classes>
			<class name="retryFailTest.TestClass">

			</class>
		</classes>
	</test>

</suite>

 

Right-click and select Run As TestNGSuite and you should get the following result.

Reexecute Failed Test Cases

You may increase the value of maxRetryCount to execute failed test cases that number of times.

Author: Dhawal Joshi

A post-graduate in MCA, ISTQB & ITIL certified QA with more than 8 years of experience in QA working with a CMMI Level 5 organization as System Analyst. I started my automation journey with HP UFT(formerly known as QTP) and for the past few years, I am using Selenium for automation. I also have experience in Android Application Development, Java, HTML, and VBScript. When I am not working, I like to spend time with my family, cooking and learning new developments in IT.

Leave a Reply

Your email address will not be published. Required fields are marked *