If you’re like us, you took the first rush of news coming in from recent developments in AI with a grain of salt.
When somebody offered an AI tool that was going to automagically “remediate” websites, what we heard was that a system trained on inaccessible code (e.g., the internet) and prone to hallucinations was somehow going to be put in charge of the henhouse.
This sounded like a terrible idea. And yet AI, and agentic coding, is extremely powerful – indeed, it’s an engineering marvel growing so quickly that many development teams are using it in sophisticated ways to write much of their code.
Is there a middle ground? A way to, ahem, harness the power of agentic coding for accessibility in a way that doesn’t guess its way into a mess? That will produce accurate fixes, and reliably so? In the engineering world, the relevant buzzword here is deterministic. And it’s essential for any tool, from toothbrushes to software.
We think we may well have an answer, and we’re packaging that capability into three products today. We really do think of these of products as a new day – a dividing line between the time when LLMs had limited usefulness, and the time when their opportunity, harnessed properly, is right in front of us.
The answer, for us, is to integrate our expertise into the agentic development loop in the form of tests. If we know how a component is supposed to behave, we can help the LLM write a test correctly, in real time, then evaluate the test results, in real time, and iterate with the LLM until the proposed code changes pass the test. And since this is agentic coding, it does this across many many pieces of code at a rate of speed that is unimaginable to a single developer. Think minutes per issue, not hours or days.
Let’s examine these new products, and how they make testing central to their effectiveness.
Autopilot
Autopilot is our tool that runs inside CI systems and alongside a customer’s LLM. It detects accessibility bugs in code that developers commit into the pipeline, as we have done for years with our Automation SDKs. What’s different is that Autopilot can actually fix those bugs, test them to make sure the fixes are correct, then write the changes up for the developer to approve inside their normal pull request cycle.
Note here that the developer is very much in the loop, but only for final approval at the end. Indeed, the developer may not necessarily even know that Autopilot is running until they see the request to approve the proposed code change.
That pull request, by the way, will contain the fixes and the reasoning for the fixes. In itself, that’s an amazing educational tool. The teachable moment, like in high school, is when you get the graded homework back.
Harness
Harness shares much of this approach, but it’s imagined as an interactive tool for developers to use before they submit code. An accessibility pair programmer, if you will.
Developers can ask Harness for help while building something for them – and have a build-test-fix loop operate automatically until that piece of code passes appropriate Evinced tests.
Or Developers can ask Harness to review something already built, provide fixes, and test those fixes.
Developers can use it to analyze and fix code where a web page (or mobile app) is in a certain state, too: for example, if a popup window has opened.
Resolve
In many ways, Resolve is the ultimate expression of our test-fix-test philosophy.
Because Resolve operates on a list of pre-existing issues that you feed it. That list can come from a manual audit, or even a report from one of our analyzer or scanner tools.
As a first step, Resolve works to relieve developers of a tedious task by reproducing the issues – that is, to verify that they still exist in production. This verification is incredibly nontrivial, technically, but it’s powerful, not something an LLM on its own is good at, and it works for a startlingly high percentage of reported issues.
Once an issue is verified, it’s fixed, and then verified as fixed. The developer reviews the fixed issues and takes the necessary next steps.
Getting started
More about all these products to come, and we are implementing them with select customers as we speak. If they sound like a fit for your team, feel free to reach out.
You won’t regret it. And we’re not guessing at that, either.