Our Automation SDK in a CI Pipeline with Jenkins and Selenium

Introduction

Integrating an accessibility testing tool into an existing test automation suite can be challenging.
In this post, we will suggest a simpler approach that you can use to add Evinced’s accessibility engines into your tests with as little effort as possible.

The Evinced Selenium SDK is uniquely positioned for a plug and play approach because it does not interact with test code directly. This has distinct advantages. First, because all of the needed setup is in auxiliary classes or methods, the tests stay exactly as the developer intended. This also means there is nothing needed to setup or maintain tests. This greatly reduces the friction created with the introduction of new tools and dramatically reduces the time it takes to start seeing a return on investment in accessibility testing.

Adding Evinced accessibility scans to your existing tests

So let’s dig in! But first, please note that every development environment is different and customization will likely be needed to suit individual needs. Also, this is not the ONLY way to implement the Evinced Selenium SDK, but we do feel like the concept presented provides a great solution for large organizations that have numerous teams and complex needs.

To add evinced to our tests we simply need to add these 4 lines of code to our test framework.

// Start Evinced
driver = new EvincedWebDriver(new ChromeDriver(chromeOptions));
driver.evStart();

// Stop Evinced
Report report = driver.evStop();
EvincedReporter.writeEvResultsToFile(testName.getMethodName(), report, EvincedReporter.FileFormat.HTML);

Most modern test frameworks have the concept of before and after hooks/methods that we can utilize to start and stop evinced within our framework. Using JUnit it would look like this:

@Before
public void setUp() {
	driver = new EvincedWebDriver(new ChromeDriver(chromeOptions));
		driver.evStart();
}

@After
public void tearDown() {
	Report report = driver.evStop();
	EvincedReporter.writeEvResultsToFile(testName.getMethodName(), report, EvincedReporter.FileFormat.HTML);
	driver.quit();
}

At this point we could just simply add your tests to this script and everything would work perfectly fine. But let’s think about the developers. We want them focused on their task of developing great software and not worrying about Selenium drivers, adding the Evinced engine, generating reports etc. We would recommend moving the tests to their own class that extends a base class containing these Before and After methods. Here is what an example class would look like:

imports ...

public class BaseAccessibilityTest {

	public EvincedWebDriver driver;

	@Before
	public void setUp() {
		ChromeOptions chromeOptions = ChromeOptionsHelper.getHeadlessOptionsConfiguration();
		driver = new EvincedWebDriver(new ChromeDriver(chromeOptions));
			driver.evStart();
	}

	@After
	public void tearDown() {
		Report report = driver.evStop();
		EvincedReporter.writeEvResultsToFile(testName.getMethodName(), report, EvincedReporter.FileFormat.HTML);
		driver.quit();
	}
}

Then our test classes simply need to extend this base class:

imports ...

public class SampleAccessibilityTest extends BaseAccessibilityTest {
	@Test
	public void demoSiteNavigationTest() {
		driver.get("https://demo.evinced.com/");
		// interacting with the page - opening all dropdown
		WebElement firstDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(1) > div > div.dropdown.line"));
		firstDropdown.click();
		System.out.println("Clicked on first dropdown");
		WebElement secondDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(2) > div > div.dropdown.line"));
		secondDropdown.click();
		System.out.println("Clicked on second dropdown");
		driver.findElement(By.cssSelector(".react-date-picker")).click();
		System.out.println("Clicked on third dropdown");
	}

	@Test
	public void anotherTest() {
		//Test Code
	}
}

Developers can focus on their work in the test classes not even having to think about the addition of Evinced accessibility scans.

CI Integration

Now that our tests are ready to run, what if we want to run the Selenium tests without accessibility scans? How can we configure this to happen automatically as part of a CI/CD pipeline? These are not “nice to haves” when it come to implementation in an enterprise development environment, this is a minimum requirement.

So let’s create a mechanism that easily enables Evinced accessibility scans and just as easily disables them. Because the starting and stopping of the Evinced engine is so simple we can easily use an “if” statement with a flag that would indicate whether to add accessibility scans to the selected tests. One possible flag is to use an environmental variable populated on the CI node at runtime to indicate whether to add accessibility scans or not. For the purpose of this example we will use the variable ADD_A11y and set it to “true” or “false” within our base class.

public class BaseAccessibilityTest {
	public EvincedWebDriver driver;
	String a11yEnabled = System.getenv("ADD_A11Y");

	private boolean isAccessibilityEnabled() {
		return a11yEnabled != null && a11yEnabled.equals("true");
	}

	@Before
	public void setUp() {
		ChromeOptions chromeOptions = ChromeOptionsHelper.getHeadlessOptionsConfiguration();
		driver = new EvincedWebDriver(new ChromeDriver(chromeOptions));
		if (isAccessibilityEnabled()) {
			driver.evStart();
		}
	}

	@After
	public void tearDown() {
		if (isAccessibilityEnabled()) {
			Report report = driver.evStop();
			EvincedReporter.writeEvResultsToFile(testName.getMethodName(), report, EvincedReporter.FileFormat.HTML);
		}
		driver.quit();
	}
}

So here, if the ADD_A11Y variable is set to “true”, the Evinced engine will be started to continuously pinpoint accessibility issues throughout the tests, and if ADD_A11Y is set to “false” the tests will run normally.

Jenkins Example

This type of variable can easily be added to the environment section of your pipeline script or set within the Jenkins UI. Below is an example pipeline script:

pipeline {
    agent {
        label '!windows'
    }
    environment {
        ADD_A11Y = 'true'
        // More ENV variables
    }
    stages {
        stage('Test') {
            steps {
              // Test Build steps...
            }
        }
    }

We have now implemented the Evinced SDK in a way that can take full advantage of our existing investment in Selenium. By extending the BaseAccessibilityTest class, we can add accessibility scans to any test in our framework. We can also enable or disable the addition of accessibility tests on demand as part of our CI/CD process using a single flag.

Reports

At the conclusion of our tests, we get a consolidated and detailed Evinced HTML/JSON accessibility report containing all issues found, severity levels, descriptions, effect on end users, and actionable information on how to resolve the issues.

Our Automation SDK in a CI Pipeline with Jenkins and Selenium

Interested in seeing this in action? Find more information in our developer documentation and/or contact us to get started!