A smartphone, tablet, and laptop showing an image of a person in a wheelchair and the letters ADA

ADA and the Internet: ADA Settlements – Testing and Bug Fixing

Written by: Beth Crutchfield

This is the sixth installment in the ADA and the Internet Series.

Automated Web Accessibility Testing

Purchasing and deploying an automated web accessibility testing tool is a key part of any web accessibility program. As part of the settlement, most clients will need to secure and use an automated testing tool that covers web, mobile and electronic documents.

Automated testing is another basic requirement we see in most agreements. While this can be done internally, most clients outsource this and have SSB run automated tests. There is no material cost associated with this approach as we can allocate a few hours per report under a structured service agreement.

User Testing

Engaging a user group of people with disabilities throughout the testing process is a good idea, however, the problem is getting the timing right in modern development environments. Fast moving scrum teams, for example, aren’t going to have a lot of value for user feedback that occurs six months later. Thus, the goal is to get users actively involved in the development process at points where they can have a positive impact on it.

One compelling reason for user testing: often a site is not 100% technically compliant but is effectively usable by people with disabilities. Being able to demonstrate usability can be profoundly helpful in bridging the gaps where the code may still need work but the website or app is functional for people with disabilities.


Defining—or at least contemplating—a method for scoring compliance during settlements can be a helpful activity. There should be a reasonable method for gauging the compliance level of the system, taking into account the number of total potential issues, not just total issues.

For example, if we found one image missing alternative text out of 10,000, there are two ways to score that website.

Claim one: One image out of 10,000 is missing alternative text. That’s pretty good. Compliance is around 99.9%.

Claim two: We found an image missing alternative text, therefore the site is not compliant. Compliance is 0%.

While the latter claim is clearly absurd, we have heard it in meetings before. Thus, we want to set some expectations and discuss scoring early in the settlement process.

General v. Perfect Compliance

In line with the discussion on scoring, we want to set some expectations around general or material compliance versus perfect compliance. In line with the example above, let’s say testing finds a single image on the site.

Claim one: One image out of 10,000 is missing alternative text, thus we would deem that to be materially compliant with that portion of the WCAG.

Claim two: We found a single image that is missing alternative text, therefore the site is not compliant with the WCAG as a whole.

As with the Scoring discussion, clearly the latter is absurd, and clearly we have had that discussion before as well.

Modification of Bug Fix Priority Policies

Many settlements have standard language that requires accessibility bugs to be handled with the same priority as other bugs. This is a best practice for ensuring accessibility gets mapped into the organization’s specific bug control approach.

Stay Tuned…

Watch for the final installment in this series, ADA Settlements – Customer Service