How to work with Accessibility Reports as Customer

This article is about how to work with digital accessibility audit reports from a customer's perspective.

Why are audit reports so extensive?

First of all, you should generally expect these audit reports to be very extensive.

The fact that accessibility reports are so long is due to the large amount of meta-information. There is a detailed introduction documenting the baseline conditions, and every single test step is listed precisely. However, the documentation of errors takes up the most space – primarily due to descriptions and screenshots. Especially when multiple screenshots are included per issue (i.e., per identified problem), it inflates the report.

I have seen companies provide these reports as PowerPoint presentations. Personally, I consider this to be completely unnecessary. Particularly in the public sector, there are unfortunately still people who print out such reports – and often multiple times, so that everyone has a copy. With PowerPoint, this is a pure waste of paper and resources. In a PDF file, it is usually a bit more compact, but often still more extensive than necessary. If you have the choice – and you should always demand this – request a format that allows you to continue working without converting or manual copy/paste. This could be Markdown, HTML, or JSON. In other words, a format that can be processed automatically as much as possible and transferred directly into ticket systems (such as Jira). It will still remain extensive due to the screenshots and descriptions, and it naturally depends on how many barriers were found. This is frequently underestimated but is simply part of the process.

The challenge with technical jargon

The second major challenge is language. We accessibility experts always try to phrase problem descriptions in a way that is generally understandable. Our target audience on the customer side is not just developers, but also management or project leads. We know that the readers are usually not accessibility specialists.

On the other hand, we have to keep it brief. If we start explaining deeper details like exact ARIA semantics in the report, there would be no end to it. That would result in too much text, which often does not help the local teams at all. Here we run into time and resource constraints – and the readers' patience is not unlimited either.

Make sure to check whether the descriptions are understandable. It is best to do random spot checks and verify whether the development teams can actually work with them. Of course, not every term will be immediately clear. We have to use technical terms like ARIA, machine-readability, or semantics in the reports. We do this so that developers can research the problems precisely. If we use inaccurate terms here, conducting your own research on the web becomes extremely difficult.

In this context, it is often more useful to use the English technical terms (especially regarding ARIA). Much of this is poorly or inconsistently documented in German. For example, there are several German translations of the WCAG (the official translation, the version from the BITV, the version for the EN standard, etc.) that do not always agree. With English terms, you will find significantly more and far more precise information when researching.

Therefore, we must assume that the customer side does their share of the work. If terms like "ARIA" or "live regions" are unclear, you have to research them yourself. There is no way around this topic in accessibility. Furthermore, an audit report cannot demonstrate how to practically implement a live region or similar elements; otherwise, it would be a tutorial.

Consulting ensures success

Therefore, I strongly recommend: Do not just book the pure test, but get the appropriate consulting right along with it. It is best to commission the consulting from the same organization that conducted the test, as experience shows this runs much more smoothly.

The worst thing you can do is to have a test conducted and then leave the report sitting in a drawer – true to the motto: "We did the test, so our duty is fulfilled." That is nonsense; in that case, you might as well not bother at all.

The second biggest mistake is trying to implement the audit report within your own team without the corresponding expertise. It is great if you have this expertise in-house. If not, the easiest way is to get support – either from the auditing body or from a neutral person for consulting.

I am not saying this to generate more business, but because practice shows: Without accessibility expertise, implementation is extremely tedious, difficult, and prone to errors. Furthermore, the auditing body can naturally assume no responsibility for the final result if there was no prior technical alignment.

Checklist: What should you look for in an audit report?

When you receive an audit report, the following core information (metadata) should always be included:

Tested pages/views: Which parts of the product were tested?
Criteria catalog: Which standard was used as a basis? (Usually the European standard EN 301 549 or the WCAG 2.2).
Conformance level: For WCAG, it should always state which level was tested (A, AA, or AAA).
Baseline data: The test date and the name of the auditing company. The date is important because things might have changed in the product during or after the test.

The core question is: Is the documentation of the test steps truly complete? A well-documented defect must always contain the following elements:

The specific WCAG or EN test step.
The precise description of the problem.
A suggested solution or the expected behavior (target state).

Therefore, it must always be clear from the report: What is currently going wrong and how should it actually look? A suggested solution is not always possible, as accessibility experts are often not deeply involved in the development details.

Transparency regarding testing tools

Another important aspect is the question: Which testing tools were used? Some criteria are tested purely visually, such as a meaningful order of content. This can often only be assessed based on human judgment. Many other test steps, such as contrasts, are highly objective by contrast. There are special tools for this, such as contrast checkers. Which exact tool you use for this does not really matter, as it should always yield the same result.

On the other hand, there are areas like ARIA where it is extremely important to document which specific tool was used. This is the only way to replicate the errors accurately later on. This is crucial for remediation: If you cannot replicate an error, it is difficult to fix it. Therefore, make sure that the descriptions are designed in a way that allows the implementing team to recreate the problems and that the tools used are documented.

This is particularly important when using screen readers or for test step 11.7 (User preferences) from EN 301 549. In this area, there is often disagreement about how exactly the test should be conducted. Here, you need to know which settings the auditor used to simulate custom views. This is the only way you can replicate the behavior on your end and correct it in line with the audit report.

What tasks fall upon the client?

The completeness check: As just described, you should verify whether all important metadata and understandable descriptions are present.
Translation into your own technical language: You need to translate the entire bundle of data into your internal language. Every company has its own terminology and names for specific components. You should perform this translation work before you pour the defects into a ticket system.
Creating component-oriented tickets: The WCAG is heavily checklist-oriented and unfortunately not structured by components. You work your way from top to bottom through the list (from success criterion 1.1.1 to the last one, usually 4.1.3) and test the entire product against it. However, modern software teams today work almost exclusively with components. This means for you: You have to map the individual test steps to the respective components of your application. A concrete example: Contrast issues are usually all collected into a single test step in the report. However, completely different components (e.g., the header, a button, and the footer) are often affected by this. You therefore need to split these defects up. This is a bit tedious, but it can hardly be avoided – unless you involve consulting so deeply that they know your internal components and handle the mapping directly. As an outsider, you simply do not know this otherwise.
Assigning tickets and creating user stories: After creation, the tickets must be assigned to the appropriate teams. Depending on the team structure, those involved can also pick out the points relevant to them themselves. In addition, appropriate user stories should be written based on this – usually by the developers or the product owners – to make the requirements tangible for practical application.

Prioritization of tasks

A comprehensive audit report can quickly contain dozens or hundreds of defects. It is completely clear that you cannot tackle this all in one go, and often not even in a single year. Therefore, you must prioritize the tasks sensibly. There are various approaches to this:

Critical blockers first: Which barriers completely prevent users from operating the system and must be resolved immediately?
Take advantage of quick wins: These are errors that can be fixed quickly and with little effort. This motivates the team because you see quick successes before drilling the thickest planks.
Utilize synergies: If a component is currently being worked on anyway, you should incorporate the accessibility requirements directly into that work.
Phasing out outdated components: Sometimes tickets take care of themselves. If a faulty component is scheduled to be removed soon anyway, you might be able to bring the replacement forward, saving you the trouble of fixing the bug in the old version.
Postponing high-effort tickets: Complex problems with high effort that are not critical can be deliberately scheduled for a later date.

Important note on conformance: Errors vs. Recommendations

This is often misunderstood on the customer side, so I will emphasize it once again very clearly: Everything marked as an error in the report must strictly be fixed.

The law requires conformance. Legally speaking, an application is only accessible or conformant once all errors have been eliminated. Auditing bodies often prioritize errors into "high", "medium", or "low" to provide you with guidance. However, even a "low priority" error remains an error and must be resolved.

You need to distinguish between two categories:

Errors: Must be fixed in any case (whether today or tomorrow is a matter of your prioritization).
Recommendations (Best Practices): Are exactly that – recommendations. They optimize usability but are not mandatory for legal conformance.

Make use of the debriefing

As a rule, auditing bodies offer a consultation hour or a debriefing meeting after handing over the report. I can only highly recommend: Take advantage of this opportunity!

Do not wait too long to do so, however. Accessibility experts test a great deal. At my peak, I tested 30 to 40 websites and applications per month. While we document as precisely as possible so that we can get back into the mindset ourselves, after a few months, nobody remembers off the top of their head what the specific problems were in a particular project. After two to three weeks, the memory is still fresh; after a year, it is gone.

So look at the report quickly and ask your follow-up questions in a timely manner. Of course, there are limits to this – an endless consultation is usually not included in a pure testing budget. But clarifying ambiguities or imprecise descriptions in the report is part of the process. It is legitimate to ask questions if something has not been described precisely enough or if you see an assessment differently.

If it's important to you, always ask whether problems can be assigned to special components. Accessibility reports are almost always grouped according to WCAG or EN requirements: all issues related to contrast are assigned to the contrast requirements, not to the component. From our perspective, this makes sense because 1. we work through the requirements this way, and 2. as outsiders, we don't know what a component is from the customer's point of view.