Automated Accessibility Testing Tools: How Much Do Scans Catch?
Dec 1, 2020
This blog was created before the release of WCAG 2.2. For information on the most up-to-date WCAG standards, visit our WCAG Compliance page.
Automated accessibility testing tools are an essential component of web accessibility testing, but can they be used as a standalone tool to assess whether your website meets the Web Content Accessibility Guidelines (WCAG)?
Automated scans can flag approximately 30% of WCAG success criteria, but to get a full and accurate assessment of your website’s conformance with WCAG 2.1 AA, both automated accessibility testing and manual testing is necessary.
Why automated accessibility testing tools aren’t enough
Automated accessibility testing tools are cost-effective, easy to use, and can catch a multitude of issues across websites of all sizes in less than a minute. This efficiency is extremely helpful to your organization’s team.
With an automated scan, you can get a good idea of where your website’s accessibility stands and some important issues to fix. However, it’s important to understand the limitations of automated accessibility testing.
For WCAG 2.1 AA tests, automated scans can only check for 20-25% of WCAG success criteria. This means 75-80% of issues will not be detected.
Scans fall short because they’re bound by yes-or-no conditions. To fully assess accessibility, there must be a dynamic mind capable of processing whether certain WCAG success criteria are met.
To date, no artificial intelligence or AI has been able to replicate human minds in testing accessibility.
Experts are able to review the findings flagged by automated testing tools, and — more importantly — go much deeper to test critical areas and capabilities on your site that automated tools aren’t capable of assessing.
Some of these areas include:
Navigating a website without a mouse is a necessary capability for countless users with disabilities. A manual test is the only way to determine if your digital properties are optimized for keyboard-only navigation, which will allow users to browse your website, access page menus, interact with links, etc., using only keyboard commands.
Titles, headings, and alt text all must be appropriately descriptive. Scans can tell you whether a title or alt text is present. Scans can flag heading structure as potentially out of order. But scans cannot assess whether these elements are appropriate for the page.
Today’s websites are complex, and automated accessibility testing tools can’t interact with them the same way a real user would. For example, consider testing your website’s checkout experience. An automated accessibility testing tool cannot add items to a cart and complete a purchase, leaving one of the most critical components of your website untested. Conversely, a manual tester would be able to simulate the entire purchase process and identify any accessibility issues occurring throughout.
Automated accessibility testing tools are also prone to missing issues in conditions that they check for. This amounts to a false negative because someone can be led to believe there are no accessibility issues where some exist.
For example, if an image has an empty alt attribute — indicating it’s a decorative image — this will pass an automated test because decorative images are acceptable. However, an alt text value may actually be needed if the image is meant to convey information.
This is why an automated scan and manual testing is essential when assessing the accessibility of your website.
Your software + people solution
With accessibility lawsuits soaring, it’s best practice to make sure your scan errors are down to 0, then have independent testing conducted and remediate your website per the resulting recommendations.
Our experts can help you perform an audit on your website to ensure WCAG conformance.
Connect with us today to learn more about our automated and manual testing capabilities and extensive range of digital accessibility services designed to help keep you compliant with ADA, AODA, Section 508, and other global regulations.