TestNG essentials: annotations, assertions, groups and data

Selenium only controls the browser. It does not decide whether a test passed. A test framework does that. TestNG gives you annotations to mark test methods, setup and cleanup code, assertions to check results, and reports. TestNG is very common in Selenium jobs in India, so you will see it in interviews.

A base class with setup and cleanup

public class BaseTest {
    protected WebDriver driver;

    @BeforeMethod(alwaysRun = true)
    public void setUp() {
        driver = new ChromeDriver();
        driver.get("https://example.com");
    }

    @AfterMethod(alwaysRun = true)
    public void tearDown() {
        if (driver != null) driver.quit();
    }
}

• @Test marks a method as a test.
• @BeforeMethod runs before every test method. @AfterMethod runs after every test method.
• @BeforeClass and @AfterClass run once per class. @BeforeSuite and @AfterSuite run once for the whole suite.
• alwaysRun = true makes the method run even when groups are filtered or an earlier method failed. This helps tearDown always close the browser.

Each test gets a new browser, so tests do not affect each other. If a class has three tests, @BeforeMethod runs three times.

Assertions

An assertion compares the actual result with the expected result. If it does not match, the test fails and the rest of that method stops.

import org.testng.Assert;

public class TitleTest extends BaseTest {
    @Test
    public void titleIsCorrect() {
        Assert.assertEquals(driver.getTitle(), "Example Domain");
        Assert.assertTrue(driver.findElement(By.tagName("p")).isDisplayed());
    }
}

The order in assertEquals is actual first, expected second. The messages use that order, so a wrong order gives confusing failure reports. SoftAssert collects failures and reports them together, but only when you call assertAll() at the end. Without assertAll(), the test passes even if a check failed.

priority, groups, dependsOnMethods and others

@Test(priority = 1, groups = "smoke")
public void openHome() { }

@Test(priority = 2, groups = {"smoke", "regression"})
public void login() { }

@Test(dependsOnMethods = "login")
public void viewProfile() { }

@Test(enabled = false)
public void notReady() { }

@Test(expectedExceptions = NoSuchElementException.class)
public void missingElement() {
    driver.findElement(By.id("nothing"));
}

• By default TestNG runs methods in alphabetical order of method name. Lower priority numbers run first, and the default priority is 0.
• dependsOnMethods skips the method if the one it depends on failed.
• enabled = false turns a test off. invocationCount = 3 runs a test three times.
• expectedExceptions passes the test only when that exception is thrown.

DataProvider

A DataProvider runs the same test with different data. It returns an Object[][]; each row is one run.

@DataProvider(name = "cities")
public Object[][] cityData() {
    return new Object[][] { {"Chennai"}, {"Pune"} };
}

@Test(dataProvider = "cities")
public void searchCity(String city) {
    System.out.println("Searching for " + city);
}

The test refers to the provider by its name, cities. This test runs two times, once for each row, and prints:

Searching for Chennai
Searching for Pune

testng.xml and Maven

A testng.xml file chooses what to run. Here only the smoke group of one class runs:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Smoke Suite">
  <test name="Smoke only">
    <groups><run><include name="smoke"/></run></groups>
    <classes><class name="demo.CoreTest"/></classes>
  </test>
</suite>

Tell the Maven Surefire plugin to use it by adding this inside the plugin configuration of pom.xml:

<suiteXmlFiles>
  <suiteXmlFile>testng.xml</suiteXmlFile>
</suiteXmlFiles>

Then mvn test runs only the two smoke tests, openHome and login, and prints a summary like this:

[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS

Try it yourself

• Write a BaseTest class and two tests that extend it. Print a message inside @BeforeMethod to see how often it runs.
• Put one test in group smoke and one in group regression. Run only smoke with testng.xml.
• Create a DataProvider with three rows and print each value.
• Use SoftAssert with one failing check, first without assertAll() and then with it.