The Problems of Artificial Intelligence for digital Accessibility

In this article we will think about artificial intelligence – not from the perspective of its potential, but rather with a view to the risks that are already emerging or have already manifested themselves, particularly in the context of digital accessibility. In previous posts, I've discussed the opportunities of AI in detail. It seems all the more important, therefore, to now also examine the problematic aspects. First, I will outline the key challenges and then discuss possible solutions.

Summary

This post examines the risks of AI in the area of ​​digital accessibility. While automatic code generation offers democratic potential (e.g., through tools like GitHub Copilot), it is currently unable to reliably create fully accessible applications. AI can provide valuable support to developers, but it does not replace expert review.

Key problems include:

• Error-prone or non-compliant code (e.g., regarding WCAG),
• Training data that predominantly contains inaccessible content and reproduces existing deficiencies,
• Hallucinations in summaries and simplifications,
• Bias towards people with disabilities nsufficiently accessible AI tools themselves,
• Poor-quality automatic image descriptions, subtitles, or sign language avatars.

It is particularly critical that affected target groups often cannot check content themselves and are therefore heavily reliant on accurate AI output.

Possible solutions include:

• AI as an assistant with multi-stage quality assurance instead of full automation,
• Curated and quality-assured training data
• Clearly defined accessible code patterns
• Regulatory frameworks such as the EU AI Act
• Above all, the systematic, early, and compensated involvement of people with disabilities in development and quality assurance

Key message: AI can promote inclusion, but without targeted quality assurance and genuine participation, it creates new risks of exclusion. Accessibility is not an add-on feature, but a fundamental requirement.

Automated Code Generation by AI

One topic currently being discussed particularly is the automated code generation by AI systems. This field is both fascinating and controversial. While some are already predicting the end of traditional software development, others warn of serious quality and security problems when program code is generated automatically on a massive scale. As is so often the case, the truth lies in a nuanced perspective.

Significantly more code is being generated automatically today – often by people without formal programming training who use AI to create simple solutions. This also applies to blind and visually impaired people. Here, a real opportunity for the democratization of software development opens up: Even minor adjustments or simple macros currently cost several hundred to a thousand euros, as professional developers must be compensated accordingly for their time. Through so-called "vibe coding"—that is, the dialog-based generation of code using AI tools—even non-experts can now develop their own solutions without relying on external service providers. In this sense, the technology has emancipatory potential.

At the same time, the risks should not be underestimated. Automatically generated code can contain security vulnerabilities, be inefficient or error-prone, and is often inadequately documented. These are serious shortcomings for professional software development.

In the context of digital accessibility, another aspect comes into play: There are already too few developers with in-depth expertise in accessible implementation. This bottleneck means that many digital products do not meet accessibility requirements, or only do so inadequately. The hope now is that AI-supported code generation could at least partially remedy this situation. However, an important distinction must be made. A distinction must be made between two scenarios:

Complete generation of entire applications with the aim of accessibility

Currently, in practice, it is not possible to instruct a tool to create a complete, accessible app. The requirements for semantically correct HTML, ARIA implementations, keyboard navigation, screen reader compatibility, and contrast ratios are complex and context-dependent. While AI systems generate formally plausible code, they do not guarantee standards-compliant implementation, for example, according to WCAG or EN 301 549.2. Targeted Support for Experienced Developers.

Using AI as an assistance system seems significantly more sensible: AI can generate boilerplate code, take over repetitive tasks, or provide information about potential barriers. However, the prerequisite remains that the responsible person can professionally assess the quality and accessibility.

Coding assistants such as GitHub Copilot, which can be integrated into development environments like Visual Studio Code and offer context-sensitive code suggestions during programming, present a promising opportunity. I see considerably more potential here than in the complete, automated generation of entire applications. An experienced developer can professionally evaluate the suggestions of such an assistant without being deeply immersed in accessible development.

In an ideal case, a multi-stage quality assurance system is created:

1. An AI tool generates or supplements code.
2. Another tool—ideally from a different provider—automatically checks this code for semantic and logical correctness.
3. In addition, traditional testing tools are used, such as static analyses or accessibility checkers.
4. Finally, a manual validation is performed by a qualified person.

Such a setup can significantly accelerate development work. It's even conceivable that a highly specialized expert in accessible development isn't required; solid programming skills are sufficient if AI-supported assistance systems provide structured support for accessibility aspects. Today, the problem isn't so much training a developer in accessibility. However, the knowledge isn't lost if the person only occasionally programs with accessibility in mind. They can't build up experience continuously, and it's not worthwhile for them to constantly stay up-to-date.

A key problem lies in the generation process itself. Many AI-supported code tools don't work incrementally in the classic sense, but rather regenerate large parts of the application when changes are made. Every new prompt can lead to existing structures being overwritten or unintentionally altered. Improvements in one area are then often accompanied by deteriorations in another. The probabilistic work of Large Langauge Models is not the best way to generate accessible code for a whole Application.

Accessibility, however, is not an add-on, but a systemic property of an application. It requires consistent architectural decisions, consistent semantic structuring, and stable interaction logic. A destructive generation approach undermines these prerequisites.

Another fundamental risk lies in the training data of the models. For web applications, the underlying HTML, CSS, and JavaScript code is publicly accessible and therefore easily usable as training material. However, most existing websites are not implemented with accessibility in mind. If a model is trained predominantly with faulty or non-standard code, it will statistically reproduce these deficiencies. Even explicit instructions such as "Generate accessible code according to WCAG" often fail to result in consistent implementation in practice. Semantic depth and normative requirements are either incompletely considered or misinterpreted. Experience from the field—for example, when using AI functions in design or development tools—shows that corresponding prompts do not guarantee truly accessible results. The intention is understood linguistically, but not implemented technically in a standards-compliant manner.

Against this backdrop, it seems unrealistic in the foreseeable future that AI systems will independently generate complete, standards-compliant, and robust accessible applications.

A structural risk arises particularly when AI systems begin using their own output as a training basis. We are already observing that more and more websites are being generated using AI—often without any consideration of accessibility. If this very code then serves as training material for new models, a self-referential cycle is created.

1. Inaccessible code is generated.
2. This code becomes publicly available.
3. It is incorporated into new training datasets.
4. The model reproduces and amplifies existing deficiencies.

This phenomenon is not limited to code. It is also being discussed that AI-generated content is increasingly being used as training data for text-based models. The danger lies in the fact that errors, simplifications, or ambiguities in content can accumulate—while maintaining high linguistic quality. This is precisely the strength, but also the risk, of these systems: They produce seemingly plausible results, even if the underlying content is lacking.

In the context of code, this means: formally correct structures that are, however, semantically incorrect, insecure, or not implemented in an accessible way. Models lack an intrinsic awareness of quality; they do not recognize that an ARIA attribute is logically inconsistent or that an interaction structure is inaccessible to screen readers—unless this has been explicitly and validly defined in the training material.

To break this cycle is not trivial. Possible approaches include:

Whether and to what extent work is being done on this is difficult to assess from the outside. However, one thing is clear: the problem is real and structural.

What I do consider as a possible solution though, is the automated accessibility testing of applications. So far, axe-core and similar systems are entirely rule-based and produce many false positives. However, some providers are working on integrating AI into the process, such as Accessibility Cloud. The ability to work more with the visual interface reduces common false positives such as those caused by contrast or keyboard focus.

Accessibility of AI Tools

Another often underestimated aspect concerns the accessibility of the AI ​​tools themselves. Many of these tools appear relatively easy to use in their basic interface. However, as soon as more complex functions are used—such as advanced configuration options, debugging interfaces, or integrations into development environments—barriers emerge:

For people with disabilities, this can effectively lead to exclusion—especially when their professional lives increasingly require the use of such tools. This creates a paradox: technologies that could theoretically promote inclusion generate new risks of exclusion.

This problem is often underestimated because the tools appear "simple" at first glance. However, the complexity becomes apparent in the depth of their functionality.

Problems with Summarizing and Comprehensible Translations

Another point that remains relevant in practice concerns so-called hallucinations—that is, factually incorrect but plausibly presented output. They often arise during AI-generated text optimization, summaries, or conversion into understandable formats using AI.

For my newsletter, I had several links summarized. Some of them were texts I wrote myself. The summaries contained a lot of nonsense. They included content that sounded relevant but wasn't in the original text.

Something similar happened to me with Gemini. I copied the entire article into the input field and then asked for a summary of that exact text. Despite this, the summary contained statements that weren't in the original. This content was clearly incorrect.

This is difficult. Many people use such tools to simplify texts. For example, they read an article from a government agency and ask for a more understandable summary. If the tool then adds incorrect information, a serious problem arises.

These tools are supposed to help people who have difficulty reading or understanding complex texts. This target group, in particular, often can't verify the information themselves. If they could read and understand the original text, they wouldn't need a summary. Therefore, it's especially critical if the summary contains errors.

I've observed this behavior with several tools, both Gemini and ChatGPT. It was less pronounced with ChatGPT, but I also didn't test it extensively.

These experiences have shaken my trust in such systems. For many years, there has been talk of so-called hallucinations. If this problem remains unresolved, the question arises whether it can be solved at all. I cannot make a definitive technical assessment. I don't know the architecture of such systems well enough. However, my impression is that it still doesn't work reliably.

Bias in Data

A major problem is the data basis of AI systems. The topic of disability is only inadequately represented there.

Disability is a complex issue. Even with physical disabilities, there are many different forms. Nevertheless, these differences are hardly reflected in image databases and text collections. The lived reality of disabled people is often not sufficiently considered.

This has concrete consequences. In automated application processes, disabled people can be disadvantaged. A system might, for example, evaluate photos or resumes. If someone doesn't conform to the expected norm in appearance or in their resume, the person can be automatically rejected. This happens even if they are professionally qualified.

There can also be risks in road traffic. Autonomous or semi-autonomous vehicles must correctly interpret human behavior. If a system misinterprets the behavior of a blind or visually impaired person, dangerous situations can arise.

For example, the vehicle might assume that a person stops because they see or hear the car. In reality, the person might not perceive the vehicle and continue walking. This could lead to an accident.

Such scenarios are particularly relevant in countries where autonomous vehicles are already in use, such as the USA or China. In Germany, fully autonomous systems are currently only permitted in very limited circumstances on public roads.

Another problem concerns information processing. Blind people often require different information than sighted people when interpreting images or graphics.