Lower Costs and better Quality for Accessibility Tests through Component Checking
The costs for accessibility tests are often quite high. Even if this initially seems positive from the service provider's perspective, it can have a negative impact on both sides. The costs increase for the customer. This can result in them deciding against a test altogether or simply hiring another service provider who offers dumping prices. To clarify the wording: website stands for the entire web Application with all pages, a web page is one page of the website.
At first it seems logical that as many pages as possible should be checked in order to find as many problems as possible. Page-based testing is still common practice. The customer or service provider selects certain pages to be checked. Costs are charged per page to be checked. This also has disadvantages for the quality of the test.
Problems with page-based testing
Errors are often due to coincidences. For example, it can happen that the editor has used a component incorrectly. It can be that a heading was used instead of paragraph formatting. This is an accessibility error that is not due to the component, but to its one-off incorrect use. If this error only happens once or is always caused by the same editor, there is little point in documenting it. But since it is de facto an error, testers must document it, even if it is not actually relevant to the application.
A sense-making testing procedure is not about finding individual errors. Test reports are often simply an accumulation of such one-off small errors, one example being missing Markup for Change of language. Rather, the aim should be to find systemic problems, i.e. problems that are structurally related to the components used. Errors made by editors are not irrelevant, but they are more of a QA problem. You don't need consultants for €200 an hour to find and document them. Lazy consultants like these page-based tests because you get a lot of findings with relatively little effort and can therefore justify your calculations even better.
Another problem arises when a complete click path is to be tested. An eLearning offering can consist of many subpages. As a rule, however, only a few components are used again and again: questionnaires, media players, certain learning formats such as cards that have to be turned over. There are usually only a few content and interactive components. From an economic point of view, it does not make sense for the customer to test the entire learning path or user journey in a webshop.
Testing Components instead of subpages
Modern websites, like apps, are often based on components that are combined by the online editors. The classic HTML website, which is basically just filled with content, still exists, but in my observation is a discontinued model on complex websites.
When testing selected subpages, it can happen that you test rather unimportant components and the important or complex components slip through because they are not present on the selected pages.
It also makes no sense to test pages that are based on the same template. It is very common that you get two or three pages that contain exactly the same components in different configurations or are based on the same page type. You will always find something, but it is rarely relevant. This does not have to be malicious: As a consultant, you cannot know how the customer's systems work. Even if pages look the same on the surface, they can still be based on different templates or components.
It can also happen, however, that components look the same but are programmed differently. The customer can find this out together with their developer. That can be difficult for an external Consultant.
Fake pages as a solution
How can you test based on components? The customer is asked to put together one or more subpages with all the existing components. Navigations are to be viewed as separate components, which usually do not need to be checked on every page, but in their various states.
It is correct that checking these individual fake pages can be more expensive than checking actually published websites. However, since fewer subpages need to be checked overall, the costs for the entire check are usually lower. In my opinion, the following components could be useful for the check.
- Homepage
- Content page with all text-image components and table types
- Form page with all form elements
- Navigation as a separate component in the various states such as opened, closed and so on
This measure will not always lead to a reduction in the costs of the test. After all, a website may have a lot of components or components that are rarely or not at all used may be tested. The customer should avoid this. From my point of view, however, it is clear that this approach can identify more relevant problems and improve the quality of the test.
Possible problems
There is a realistic risk that the customer will make a lot of effort to make such fake pages accessible. After all, they want to present themselves in a particularly good light.
As a rule, however, you can dissuade them from such ambitions. The point of a test is to find possible problems. It is not the customer who is being assessed, but their product. The accessibility consultant should be seen like a doctor. You don't start brushing your teeth a week before you go to the dentist.