Comparison of Online Accessibility Evaluation Tools: An Analysis of Tool Effectiveness

The use of online tools is a common practice for evaluating the accessibility of a website, identifying problems, and providing useful feedback on how to fix detected issues. For ease of accessibility validation, many tools have been developed and implemented successfully over the years. Yet, the results of these tools show differences in terms of coverage, correctness, and reporting-related issues. In this study, we compared online accessibility evaluation tools to understand to what extent they differ in detecting accessibility problems in websites. A total of 41 government websites of different countries were tested for violations of accessibility guidelines using six evaluation tools. We observed that each tool generated different evaluation data for the same websites. As some of the tools are complementary to each other, meaning the highest coverage and completeness can be possible with the right combination of evaluation tools. Therefore, we suggest different tools should be utilized to provide consistency and obtain reliable data from online evaluation tools, thereby improving tool effectiveness.