A New Approach to Accessibility at Capital One…with Evinced
Background
Capital One has made digital accessibility a priority for many years. Over time, we’ve invested in developing an industry-leading accessibility program to stay at the forefront of delivering best-in-class experiences to our existing and prospective customers. A key imperative has always been continuously advancing our goal of integrating accessibility into every aspect of our software development lifecycle.
Accessibility at Capital One is evaluated end-to-end during product ideation, design, and development, through automated and manual testing, and ongoing monitoring of our production environment. Ensuring all of our web and mobile assets are accessible and usable by all is a commitment across our organization and executive leadership, who are fully committed to this effort.
Problem
Compared to other areas of modern cloud infrastructure, such as cloud management, security, application development and data management, accessibility testing is too often an afterthought and left behind when it comes to cutting edge innovation. Most automated testing solutions use static, syntax analysis which provides adequate coverage in a world of static web properties. Today’s modern web and mobile content, however, leverage more dynamic, JavaScript heavy web applications which can leave existing testing solutions limited in monitoring accessibility/automated testing, often catching less than 10% of critical issues.
Most experts believe existing automated testing tools cover somewhere around 20-30% of Web Content Accessibility Guidelines (WCAG). With those solutions leaving so much more to address, a bulk of the heavy lifting for accessibility testing falls on someone (an accessibility team, if you’re lucky) to manually conduct.
As software release cycles become more frequent, manual review quickly struggles to scale and slows the overall release cycle. For example, during the implementation phase of a new feature, development teams previously interacted with Capital One’s Digital Accessibility Team (DAT) in a traditional consulting fashion. During the development phase, an accessibility subject matter expert worked with developers to make sure that the feature was accessible.
Although this process eventually resolved the outstanding issues, the back and forth, testing and retesting dialogue takes time. As release frequencies are increasing the DAT looked for solutions and technology to best ensure that Capital One’s products upheld the WCAG.
Solution
Evinced brings real innovation to this problem with a sophisticated technological approach, leveraging a combination of computer vision and machine learning models to detect a greater number of accessibility issues automatically. Capital One partnered with Evinced early, to guide their development with a particular focus on: helping developers release accessible code integrating multiple automated testing steps through the build and deployment lifecycle building products that can automatically scan for accessibility across a full web property (including through logins and internal repositories), and do this fast.
Our developers use the Evinced Dev Debugger to check their code for accessibility before a pull request. From there, some teams have begun using the Evinced Cypress Automation SDK to automatically test critical flows for accessibility in the CI flow.
When it comes to the DAT, we use the Evinced User Flow Analyzer to ensure COF products upheld the WCAG, and augment that with some manual testing. Finally, once code is pushed into production, the Evinced Site Scanner continuously monitors our production environment and reports our overall accessibility status, including new issues, old issues, MTTR of critical issues etc.
Results
We’ve seen Evinced discover as much as 10x more critical accessibility issues than we were previously finding through automated testing alone. An even greater number of issues are discovered when a site is more interactive, including keyboard and screen reader usability issues (<keyboard accessible> and <interactable role>) Automated testing on a large enterprise scale can be an extremely complex and time consuming effort. Evinced is speedy and reliable, with 40x faster execution, enabling us to cut our processing time in some cases from 4-5 days down to less than 3 hours (and is being further optimized)
What’s next
Imagine a world where we already know what most of the issues are before developers submit their code for review. With the ability to automate testing of code repositories at scale, on demand, and before release we can create a snapshot at any given moment of the accessibility issues in dev and qa environments all over the company.
We will be able to leverage events (webhooks) from prominent code repositories to kick off scans with little input from the corresponding dev team. This will give us almost real time data as code flows from developers’ fingers all the way into production. Evinced is a great partner, delivering real innovation and helping us further integrate accessibility in our software development lifecycle.