Using our Automation SDK for Selenium with Sauce Labs

Having worked closely with Selenium for the past 7 years I love the idea of getting as much value from your functional UI tests as possible. One way to do that is to add accessibility testing as well. This allows us to shift a11y testing left discovering issues earlier when they are faster, easier, and cheaper to fix. Add to this the power of the Sauce Labs Selenium grid with its instant scalability and actionable test insights and we have a solution that will help us dramatically improve end user experience for everyone.

The advantages of the Evinced Selenium SDK over other accessibility tools are two fold. First, there is no need alter individual test code!!!!! (go ahead and read that again). Anyone that has used other a11y Selenium integrations understands the pain of having to add an accessibility scan after every interaction with the page that causes a significant DOM shift. An end to end test might have 10+ scans! Not only does this cause continual maintenance and reliability issues, but every time you add a scan to a test it creates a report that must be managed. This means organizing dozens of reports and wasting time with a mess of duplicate issues. Evinced provides a single detailed report at the conclusion of your tests with all the details you need to understand the issues and fix them quickly.

Unlike other tools that rely on syntax analysis, our Automation SDK for Selenium uses computer vision to model the UX intent and actions to the actual implantation. This approach allows Evinced to find more critical accessibility issues that were once only found with manual testing.

In this post we will discuss how easy it is to take an existing Selenium test built to run on Sauce Labs and add the Evinced accessibility scanner to pinpoint accessibility issues along the way.

So, let’s take a look at our Sauce Labs example test:

import com.evinced.dto.results.Issue;
import com.evinced.dto.results.Report;
import com.evinced.utils.ChromeOptionsHelper;
import com.evinced.EvincedReporter;
import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.*;
import org.junit.rules.TestName;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.*;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.edge.EdgeOptions;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.MalformedURLException;
import java.net.URL;
import java.rmi.Remote;
import java.util.List;

public class SauceLabsTest {
    // Sauce Labs username and access key stored as environmental variables
    public String sauce_username = System.getenv("SAUCE_USERNAME");
    public String sauce_accesskey = System.getenv("SAUCE_ACCESS_KEY");
    // Test base url
    public static final String demoPage = "https://demo.evinced.com/";
    private WebDriver driver;
    
    @Before
    public void setUp() throws MalformedURLException {
        // sauceOtps pass the Sauce Labs specific paramaters 
        MutableCapabilities sauceOpts = new MutableCapabilities();
        sauceOpts.setCapability("username", sauce_username);
        sauceOpts.setCapability("accessKey", sauce_accessKey);
        sauceOpts.setCapability("name", "Evinced A11y Selenium SDK Demo");
        sauceOpts.setCapability("screenResolution", "2048x1536");
        ChromeOptions chromeOpts = new ChromeOptions();
        chromeOpts.setCapability("browserVersion", "latest");
        chromeOpts.setCapability("platformName", "macOS 11");
        chromeOpts.setCapability("sauce:options", sauceOpts);
        // Set the Sauce Labs endpoint as the remote URL location
        URL sauceURL = new URL("https://ondemand.us-west-1.saucelabs.com/wd/hub");
        driver = new RemoteWebDriver(sauceURL, chromeOpts);
    }
    
    @After
    public void tearDown() {
        driver.quit();
    }
    
    @Test
    public void evincedTrvlTest() {
        driver.get(demoPage);
        // Click "Your New Home" dropdown
        driver.findElement(By.cssSelector("div.filter-container > div:nth-child(1) > div > div.dropdown.line")).click();
        // Click "Where" dropdown
        driver.findElement(By.cssSelector("div.filter-container > div:nth-child(2) > div > div.dropdown.line")).click();
        // Click "When" date picker
        driver.findElement(By.cssSelector(".react-date-picker")).click();
        // Click on the "Search" Button
        driver.findElement(By.className("search-btn")).click();
        // Assert we have navigated to the results page
        Assert.assertEquals("Page two | Evinced, Demos site", driver.getTitle());
    }
}

The first place we want to take a look at is the @Before method. This is where we will want to initiate the EvincedWebDriver class by passing in the RemoteWebDriver instance sauceDriver that has created a Selenium session on the Sauce Labs cloud. Check out the Sauce Labs platform configurator for all the information needed to select a testing environment.

We can then start the Evinced engine using the evStart() method. The engine will continuously run in the background tracking all DOM changes and collecting a11y violations as the tests execute.

private EvincedWebDriver driver;

@Before
public void setUp() throws MalformedURLException {
    // sauceOtps pass the Sauce Labs specific paramaters 
    MutableCapabilities sauceOpts = new MutableCapabilities();
    sauceOpts.setCapability("username", sauce_username);
    sauceOpts.setCapability("accessKey", sauce_accessKey);
    sauceOpts.setCapability("name", "Evinced A11y Selenium SDK Demo");
    sauceOpts.setCapability("screenResolution", "2048x1536");
    ChromeOptions chromeOpts = new ChromeOptions();
    chromeOpts.setCapability("browserVersion", "latest");
    chromeOpts.setCapability("platformName", "macOS 11");
    chromeOpts.setCapability("sauce:options", sauceOpts);
    // Set the Sauce Labs endpoint as the remote URL location
    URL sauceURL = new URL("https://ondemand.us-west-1.saucelabs.com/wd/hub");
    sauceDriver = new RemoteWebDriver(sauceURL, chromeOpts);
    //Pass the sauceDriver instance to the EvincedWebDriver
    driver = new EvincedWebDriver(sauceDriver);
    // Start the Evinced engine continuously scanning for a11y issues
    driver.evStart();
}

The last thing we need to do is stop the engine and generate the accessibility reports. We can do that in the @After method. A Report object is created when we stop the Evinced engine using the evStop() method. We can then choose to export the accessibility report as a JSON or HTML report (or both!).

@After
public void tearDown() {
    // Stop the Evinced engine and generate the report object
    Report report = driver.evStop();
    // Export the report - JSON
    EvincedReporter.writeEvResultsToFile("Evinced-Sauce-A11y-JSONReport", report, EvincedReporter.FileFormat.JSON);
    // Export the report - HTML
    EvincedReporter.writeEvResultsToFile("Evinced-Sauce-A11y-HTMLReport", report, EvincedReporter.FileFormat.HTML);
    driver.quit();
}

We are now ready to execute our test!

Not only do we gain all of the additional insights provided by Sauce Labs such as videos, logs, screenshots, and HAR file to make debugging our functional test quicker and easier, but we also get a detailed Evinced HTML and/or JSON accessibility report containing all issues found, severity levels, descriptions, effect on end users, and actionable information on how to resolve the issues.

For more info, check out the Evinced Automation SDK and the docs for our Selenium implementation, along with the Sauce Labs pages.