Appium, Perfecto and Our Automation SDK

Appium is a popular open source mobile automation tool and has a significant advantage with its approach to automated mobile testing. Appium is cross platform meaning you can test both iOS and Android from the same framework. I think we all can agree that minimizing the number of context switches required in a given day is always better.

With the addition of the Evinced Appium SDK your tests can return even more value by pinpointing accessibility issues as well. The Evinced API goes beyond what the iOS and Android accessibility APIs offer to help you find more issues that could be impacting the accessibility of your application. In addition, Evinced uses advanced algorithms to create a single actionable report that is easy to digest. Having had the pleasure of wading through a sea of 20+ reports created based on individual scans, this is a feature I am very excited about.

The one thing missing from our even more powerful testing practice is the ability to test on a wide variety of devices and operating systems. This is crucially important for both the Android and iOS platforms.

For Android, breadth of coverage is extremely important due to the large segmentation in the Android market (Samsung, Google, LG, HTC, OnePlus, Huawei, etc). All these manufacturers take the default Android OS image from Google and make it their own. This creates the need for additional testing to make sure we give the best experience possible to all users. For iOS, you may have noticed that once you upgrade your iOS device it is impossible to roll it back to a previous version. This way, proper testing requires a library of iOS devices on all the versions a potential customer might be using to ensure a great experience.

Fortunately, Perfecto has a comprehensive cloud based continuous testing platform where we can execute our Appium tests with the Evinced Engine to pinpoint accessibility issues on nearly any device and operating system combination.

In this post, we will discuss how to take an existing Appium test running on a Perfecto device and add the Evinced engine to scan for accessibility issues.

Let’s take a look at our Java JUnit Appium test running on a Perfecto Device:

import io.appium.java_client.ios.IOSDriver;
import io.appium.java_client.ios.IOSElement;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.Platform;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import com.evinced.appium.sdk.core.EvincedAppiumSdk;

public class EvincedAppium {

    public static URL url;
    public static IOSDriver<IOSElement> driver;
    public static EvincedAppiumSdk a11yValidator;
    
    @BeforeClass
    public static void setupAppiumDriver() throws MalformedURLException, IOException {
        String securityToken = <Your Perfecto Security Token>;
        
        DesiredCapabilities capabilities = new DesiredCapabilities("", "", Platform.ANY);
          capabilities.setCapability("model", "iPhone.*");
          capabilities.setCapability("app", "<APP_LOCATION>"); // Set Perfecto Media repository path of App under test.
          capabilities.setCapability("bundleId", "com.bundle.id"); // Set your App's bundle Id here
          capabilities.setCapability("enableAppiumBehavior", true);
          capabilities.setCapability("autoLaunch", true); // Whether to install and launch the app automatically.
          capabilities.setCapability("openDeviceTimeout", 2); // Waits for 2 minutes before device connection timeout
          capabilities.setCapability("iOSResign",true);
          capabilities.setCapability("platformName", "iOS");
          capabilities.setCapability("platformVersion", "14.7.1");
          capabilities.setCapability("automationName", "XCUITest");
          capabilities.setCapability("securityToken", securityToken);
    
        driver = new IOSDriver<IOSElement>(new URL("https://partners.perfectomobile.com/nexperience/perfectomobile/wd/hub"), capabilities);
        driver.manage().timeouts().implicitlyWait(20, TimeUnit.SECONDS);
    }
    
    @AfterClass
    public static void tearDown() {
        driver.quit();
    }
    
    @Test
    public void testStationsScreenIsAccessible() {
        IOSElement firstStationTableCell = driver.findElement(MobileBy.className("StationTableViewCell"));
        firstStationTableCell.click();
        IOSElement playButton = driver.findElement(MobileBy.className("XCUIElementTypeButton"));
        playButton.click();
        Assert.assertTrue(!playButton.isDisplayed());
    }
}

The first place we want to take a look at is the @BeforeClass method. This is where we will want to initiate the EvincedAppiumSdk class by passing in the IOSDriver instance that has created an Appium session on the Perfecto device cloud. This will not impact any of the existing functionality of the driver, it will simply add the tools needed to add accessibility scans to our test. Check out this guide for all the information needed to select a cloud device. Next, add your Evinced service ID and API key for to authenticate the Appium SDK. These credentials can be found by logging into the Evinced Product Hub.

@BeforeClass
public static void setupAppiumDriver() throws MalformedURLException, IOException {
    String securityToken = <Your Perfecto Security Token>;
    
    DesiredCapabilities capabilities = new DesiredCapabilities("", "", Platform.ANY);
      capabilities.setCapability("model", "iPhone.*");
      capabilities.setCapability("app", "<APP_LOCATION>"); // Set Perfecto Media repository path of App under test.
      capabilities.setCapability("bundleId", "com.bundle.id"); // Set your App's bundle Id here
      capabilities.setCapability("enableAppiumBehavior", true);
      capabilities.setCapability("autoLaunch", true); // Whether to install and launch the app automatically.
      capabilities.setCapability("openDeviceTimeout", 2); // Waits for 2 minutes before device connection timeout
      capabilities.setCapability("iOSResign",true);
      capabilities.setCapability("platformName", "iOS");
      capabilities.setCapability("platformVersion", "14.7.1");
      capabilities.setCapability("automationName", "XCUITest");
      capabilities.setCapability("securityToken", securityToken);

    driver = new IOSDriver<IOSElement>(new URL("https://partners.perfectomobile.com/nexperience/perfectomobile/wd/hub"), capabilities);
    driver.manage().timeouts().implicitlyWait(20, TimeUnit.SECONDS);
    evincedAppiumSdk = new EvincedAppiumSdk(driver);
    evincedAppiumSdk.setupCredentials("<Evinced Service ID>","<Evinced API Key>");
}

Now that we have added the capabilities need to the driver, the next step is to identify in our test where adding accessibility scans will give use the maximum coverage of our application. This will likely be after a tap or a swipe that takes us to a new page or opens a menu. Taking a look at our test we can identify the following points:

@Test
public void testStationsScreenIsAccessible() {
    // The main application screen has loaded. Let's scan for a11y issues!
    IOSElement firstStationTableCell = driver.findElement(MobileBy.className("StationTableViewCell"));
    firstStationTableCell.click();
    // We have clicked on a cell that has taken us to a new page. Let's scan for a11y issues!
    IOSElement playButton = driver.findElement(MobileBy.className("XCUIElementTypeButton"));
    playButton.click();
    // We have now entered the player view. Let's scan for a11y issues!
    Assert.assertTrue(!playButton.isDisplayed());
}

Now that we have identified the points in our test that would be good to scan, lets add the code:

@Test
public void testStationsScreenIsAccessible() {
    // Let's scan for a11y issues!
    evincedAppiumSdk.analyze();
    IOSElement firstStationTableCell = driver.findElement(MobileBy.className("StationTableViewCell"));
    firstStationTableCell.click();
    // Let's scan for a11y issues!
    evincedAppiumSdk.analyze();
    IOSElement playButton = driver.findElement(MobileBy.className("XCUIElementTypeButton"));
    playButton.click();
    // Let's scan for a11y issues!
    evincedAppiumSdk.analyze();
    Assert.assertTrue(!playButton.isDisplayed());
}

In this test we have added the scans directly to test code for demonstration purposes. For actual implementation we would recommend adding the scans to existing page objects methods to make maintenance simple and easy.

The last thing we need to do is generate the report of all the scans. We can do that in the @AfterClass method.

@AfterClass
public static void tearDown() {
    List<Report> reports = evincedSdk.reportStored(true);
    driver.quit();
}

This single line of code will automatically compile all of the data from the scans and generate a set easy to digest HTML and JSON report files.

We are now ready to execute our test!

Not only can we run our test on any of the hundreds of devices and operating system versions Perfecto provides, we also gain the additional insights provided such as videos, logs, and screenshots to make debugging our functional test quicker and easier. This is extremely powerful combined with the actionable Evinced HTML and/or JSON accessibility report containing all issues found, severity levels, descriptions, effect on end users, and a link to actionable information on how to resolve the issues.

Check out the Evinced and Perfecto pages for more information on getting started!