Want to scale your accessibility efforts? Test automation is a great tool to have in your toolbox and can help you do just that, whether it’s for mobile or web applications.
For many accessibility managers, test automation might seem complex at first glance. It can feel intimidating, too technical, and for someone without an engineering background, it can seem too much like coding. And with so many options across platforms and frameworks, you might not know where to start.
Plus, you might be worried that whatever you pick will interrupt or even break the development workflow, shattering the chances for adoption.
But you don’t have to be an engineer to lead a successful test automation initiative.
You just need to understand what it is, why it matters, and how to partner with your development team to implement it in a way that supports your goals – and theirs.
This article will walk you through the basics of accessibility test automation, how it compares to manual testing, and where it fits in your current development workflow.
What is test automation, exactly?
Accessibility test automation is the practice of using scripts or tools integrated into the development process to search for accessibility issues, which scales and augments the efforts of human testers.
Even checks that still need expert input become more manageable when repetitive, low-complexity issues are caught automatically. Automation frees your team up to focus on high-impact, high-context accessibility work, the kind that truly benefits from human expertise.
These tests run behind the scenes, typically in your Continuous Integration/Continuous Development (“CI/CD”) pipeline, scanning for known issues and generating reports – sometimes even with suggested fixes.
Think of it as an accessibility spell-checker in that it doesn’t catch everything, but what it does catch is useful and important.
Why it matters
The reality is some engineering teams are shipping code more than 1,000 times a week. Test automation and automation in general are your best bet for keeping up with that pace.
The alternatives: manual and semi-manual testing
In the world of mobile applications, most accessibility teams today run manual or semi-manual accessibility tests.
Manual testing relies on subject matter experts and testers to interact with applications the way end users would, leveraging accessibility features found in mobile operating systems. With mobile applications for example, someone testing for screen-reader functionality would use Apple’s VoiceOver or Android’s TalkBack to test navigation and user experience of an app.
Manual testing can find unexpected bugs since humans are creative (hooray for that!) and might try something in the app that the development team didn’t plan or code for. Plus, human testers can give real-time feedback, and the setup is straightforward. All you need is a human, a device, and the app you’re testing.
The biggest downside is that manual testing tends to be too late in the process. It’s usually done after code is written or committed or, worst of all, in production. It’s also time-consuming, inconsistent, and hard to scale. If your app has a lot of features or frequent releases, manual testing can slow the dev cycle down. You’ll be faced, as Mark Penicook from Capital One has pointed out, with terrible choices: “Test less. Test later. Or grind your delivery to a halt.”
Semi-manual testing uses manual testing practices in combination with automated tools. For mobile applications, native automated tools include Accessibility Scanner (Android) and VoiceOver Accessibility Inspector (iOS). There are also third party automation tools that can be brought in to help scale some accessibility testing efforts, such as browser-based scanners. But ultimately this approach depends on a human to run the test, evaluate and investigate the results, and figure out what to do next.
Which means it’s faster than manual-only testing, but still reactive and hard to scale, especially for full, complex applications that release new features often.
Where accessibility test automation fits
The last thing you want, as an accessibility manager, is to ask your engineering team to change their workflow.
The good news is that you don’t have to.
Accessibility testing can be integrated directly into the developer workflow – specifically, into your CI/CD pipeline.
What that means in practice is that when developers commit code, the CI/CD system automatically runs a series of tests to make sure the code is ready to be merged into the main branch. Accessibility tests can be included right alongside those other tests.
Think of the CI/CD system as a factory conveyor belt and accessibility is just one of the inspection stops along the way.
If the tests detect accessibility issues, the pipeline can flag them immediately and even block the commit from being merged into the main codebase until the issues are fixed.
Checking for accessibility, done this way, requires no extra steps. No separate process. Just one more way to make sure your product works for everyone, without making more work for your team.
Tips to get engineers on board with test automation
One challenge to consider? Getting your engineering team to buy in, even if they already do test automation in general.
Remember, your engineering team has heard plenty of promises before for other kinds of software that aimed to make their job easier. And they’ve been burned when that software turned out to be faulty, overly complicated, or frustrating to use.
In our experience, four things you can do to get buy-in are:
- Run a workshop. In many cases engineers may have misconceptions about what “testing” entails. For example, they may think that all testing needs to be written before coding can start, or that testing takes significantly longer in development cycles. The best way to dispel these myths is to create a safe space for engineers to practice and to ask questions.
- Choose easy. Easy matters. A tool should be easy to integrate into a CI/CD pipeline, and easy for a developer to work with, of course. But it’s not enough for a developer to be told that there are X errors in their code. They need to know where those errors are, and what to do in order to fix them, in developer language they can understand.
- Partner with an executive. Like all teammates, developers and QA folks can be hesitant to add more to their workload. What we’ve seen is that you’ll need to enlist a respected engineering leader to help drive adoption, and someone who can see that catching bugs earlier doesn’t add work, it saves work.
- Talk in terms developers can understand. Present your test automation plan as something that is going to improve what developers are already concerned about. It’s not enough to tell them how important accessibility is, since everybody who asks an engineering team for something says it’s important! Instead, ground your appeal in some metric or task they are already worried about, like “These tools will help you fix this issue now, and in a quarter of the time it would take to fix it later. So you can spend less time on fixing bugs, your efficiency metrics will go up, and you can spend more time on Project X.”
Look, implementing change in an organization is never easy. But some changes are worth the trouble – and test automation is one of them, for sure. And you’re not alone. We’re here to help.