Skip to content

beforeClass * <number of parameters in data provider> is shown in each test case result for test class with @Factory(dataProvider = "parameters") #896

Description

@OlegKuzovkov

Describe the bug
For the setup where we use a @factory(dataProvider = "parameters") for our test class constructor, to be able to execute the same set of tests with different parameter, we see all @BeforeClass/@afterclass executions in every test case result:
image

How TestNG works: it will execute @BeforeClass/@afterclass for each test class instance individually, which will be the number of items in the dataProvider.

To Reproduce
Steps to reproduce the behavior:

  1. Use a code snippet below:
package com.ctera.auto.tests;

import lombok.extern.log4j.Log4j2;
import org.testng.annotations.*;

import static io.qameta.allure.Allure.step;

@Log4j2
public abstract class TestClass {
    abstract String getParam1();
    abstract String getParam2();

    @BeforeSuite
    public void beforeSuite() throws Exception {
       log.info("before suite");
    }
    @BeforeClass
    public void beforeClass() throws Exception {
       log.info("before class");
    }

    @BeforeMethod
    public void beforeMethod() throws Exception {
       log.info("before method");
//        throw new Exception("Before Method failure");
    }

    @DataProvider
    public static Object[][] parameters() throws Exception {
        return new Object[][] {
                {"groups","domain"},
                {"users", "user"},
                {"users", "child"}
        };
    }

    @Test
    public void test1() {
        step("Step of the test", () -> {
            log.info("para1 " + getParam1());
            log.info("para2 " + getParam2());
        });
    }

    @AfterSuite
    public void afterSuite(){
        log.info("before suite");
    }
    @AfterClass
    public void afterClass(){
        log.info("after class");
    }
    @AfterMethod
    public void afterMethod(){
        log.info("after method");
    }
}
package com.ctera.auto.tests;

import io.qameta.allure.testng.TestInstanceParameter;
import lombok.Getter;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;

@Getter
public class TestAnotherClass1 extends TestClass {
    @TestInstanceParameter
    private final String param1;
    @TestInstanceParameter
    private final String param2;

    @Factory(dataProvider = "parameters")
    public TestAnotherClass1(String param1, String param2) {
        this.param1 = param1;
        this.param2 = param2;
    }

    @DataProvider
    public static Object[][] parameters() throws Exception {
        return new Object[][] {
                {"oleg","domain"},
                {"noe", "user"},
                {"imanuel", "child"}
        };
    }
}
  1. Run test TestAnotherClass1 using TestNG.
  2. Upload results to Allure TestOps
  3. Check each test case result:
    image

Expected behavior
Only @BeforeClass/@afterclass method executions that are relevant to test case should be included into a report, and not all of them for each test class instance execution.

Screenshots
Attached above.

Additional context
Add any other context about the problem here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions