Writing effective Accessibility Bug Reports
This article provides an overview of what is important when writing error reports.
Some possibilities are optional and depend on your technical and accessibility experience. For example, if you don't have the knowledge of how to fix a particular bug, omit that item from your bug report.
- Information about the test environment
- Writing bug reports
- Reference to WCAG criterion or best practice
- Prioritization/Severity Assessment
- -Documenting errors
- Read more
Information about the test environment
Write down the basic information about the system so that errors can be traced more easily. This includes:
- Operating system and version, possibly also sub-version
- browser and version used
- possibly assistive technology and version
- used test tools
- if applicable, smartphone used
During the test, make sure that the browser is as “clean” as possible. This means that no cookie blockers, script blockers, ad blockers or similar should be activated. It may make sense to create your own browser profile just for testing. This is possible in Firefox, for example. No extensions should be installed in this profile unless they are relevant for testing.
With some editorial systems such as WordPress or Drupal, it makes sense to log out before testing or to use an anonymous tab for testing. Logged-in users often see content differently. The browser cache should also be cleared if necessary, otherwise you won't be able to see some things like the cookie notifications.
Also write down the date of the test. Some pages are updated daily. In addition, the reports may be read months after the test has been done.
Writing bug reports
Bug reports should always be written from the perspective of whoever is supposed to fix the bug. This person may have little to no understanding of accessibility.
The following points should be included in every bug report:
- At what point does the error occur.
- When does the error occur? What steps must the developer take to reproduce the error.
- What is the desired outcome? What must be different or happen so that the error no longer occurs.
Please note: There are websites that are generated completely dynamically. This includes, for example, search result pages in online shops or notifications of incorrectly filled out forms. Such bugs can only be reproduced if developer knows how you went about reproducing the bug on your own.
It may be that the developer has little or no experience with accessibility. And no time to deal with it in detail. Therefore, it should always be made clear what exactly is going wrong and what the right/desired behavior would be.
Match each problem to the appropriate WCAG criterion. The WCAG refers to documents that are important for developers, such as the Techniques (How to Meet) and "Understand...", which can help those responsible to better understand the problem and to fix it if necessary.
If possible, provide recommendations on how to fix a bug or solve a problem. In any case, show which behavior is desired or corresponds to accessibility.
Reference to WCAG criterion or best practice
In case of errors, always refer to either the appropriate WCAG criterion or to best practice documentation. WCAG criteria weigh more heavily, since a fail here means that the entire WCAG was not passed and the website is therefore not standard-compliant.
Best practices are most interesting when they are easy to apply and proven to have a positive impact on user experience.
It is also interesting to know who is responsible for correcting an error. There are usually two groups of persons: developers who take care of the maintenance of the website and editors who are responsible for maintaining the content of the website. As a rule of thumb, all elements that form the framework of the website, i.e. navigation, footer, banner and so on, fall under the technology. Only the actual content area is managed by the editors. This applies to a limited extent to interactive elements such as forms, players or other dynamic elements in the content area. They often fall into the technology corner.
You can create your own column in the error report in which you can present the presumed responsibility. Editors can usually fix errors faster than developers.
It is also important that you specify the severity of the problem.
The severity of an issue depends on how badly the issue affects users. A problem is serious when it is
- has a strong impact on many users
- a user group completely prevented access
A serious problem for several user groups are forms without labels. They cannot be used or cannot be used comfortably by the blind, visually impaired or persons with motor disabilities.
A serious problem for a certain user group are information graphics without image descriptions. Blind persons then do not know what is there, which means that they miss out on information.
Serious problems that are likely to be difficult to fix should also be prioritized. More lead time or woman power is needed here to fix it.
On the other hand, problems that are not severe but are complex to fix can be prioritized down.
Problems with accessibility should also be clearly documented graphically . One easy way is to make good screenshots. You can also add highlights or comments there, for example. Choose the clippings carefully so the developers know exactly where the problem is occurring.
For more complex websites/interactions it can also be useful to record videos or several screenshots of a history of the error chain. You should also make sure that all interactions are clearly visible, for example the mouse cursor should be visible.
There are some programs and browser extensions for this purpose.