Why assistive technologies should be tested objectively

The Internet has made it easier to access information about assistive devices. The number and variety of aids has increased significantly. And that's even if you don't include the various apps that can serve similar purposes as aids. Let's look at a few examples.

To explain: Dedicated aids are devices that are designated as aids, are often covered by health insurance and usually serve a very specific purpose. A distinction must be made between everyday devices that are used like aids, even if they were not originally intended for that purpose: smartphones, tablet PCs, voice-controlled assistance systems and so on. I'm only mentioning aids that I know at least a little about, so I have to exclude wheelchairs and the like. This also only includes dedicated tools.

Aids for the blind

There is currently a small boom in obstacle detection systems for blind people: shoes, belts, equipment for the blind cane and so on. It is practically impossible to get neutral information beyond the manufacturer's marketing blahblah, which of course is of next to no use. Mobility trainers who teach orientation techniques to blind people often know the equipment and can report a lot from their own experience, but rarely know all the available equipment, their strengths and weaknesses in detail and in comparison to each other.

Another example are braille displays: During my last visit to the SightCity trade fair, I had a small braille display from Humanware in my hand: a real plastic bomber, it seemed poorly made even for my low standards. These small Braille displays are often stored in backpacks or jacket pockets and, even when they are in a protective bag, often have to withstand more than large Braille displays that are almost never moved. The lower weight of plastic was probably the deciding factor here.

And then the OrCam, a tool that was incredibly hyped. In fact, they can't do anything that a standard smartphone with a few apps can't do: recognize colors, texts, faces... Their only advantage is that they can be attached to the temples of glasses, leaving your hands free for blind people super important. I would be seriously interested to see whether it actually performs better compared to, say, the SeeingAI app for the iPhone. My guess is no. The reason is quite simple, even an older iPhone probably had better technology installed and SeeingAI is the prestige project of the billion-dollar company Microsoft. Of course, we don't know how much AI there is in it, but it's certainly more than the OrCam will ever have. But as I said, an objective comparison would reveal the strengths and weaknesses. Currently, a structured comparison with the - in my opinion significantly better - Envision AI would be interesting.

It would also be exciting to ask what apps for color recognition are actually good for, especially recognition compared to full-blown color recognition devices, which quickly cost as much as a smartphone but are less practical. This makes it easy to test objectively, but no one does it.

By the way, testing does not mean simply using the devices. You have to have a little knowledge of the subject matter, for example about the correct alignment of devices for obstacle detection, the strengths and weaknesses of different technologies such as infrared or ultrasound, and a sensible methodology.

Hearing aids

We find the same tragedy exacerbated with hearing aids. There is the additional problem here that there is a confusion of brands and acousticians: the acoustician chain X only sells hearing aids from brand Y, sometimes simply branded, but sometimes not. In the worst case scenario, if you take this chain, you may only have hearing aids to choose from, none of which are ideal for your hearing problem.

In addition, the health insurance company only pays the minimum amount, but the acousticians are happy to charge for extras - often unnecessary ones: mini hearing aids, additional devices for special purposes, even for the Pipifax Bluetooth connectivity to the smartphone are charged separately.

In addition, the apps often offer additional setting options for the hearing aids. So it is by no means irrelevant which device you choose; the choice of acoustician can determine your future hearing quality.

Now try to get objective information about the individual devices, their possible services, functions and additional costs. This is definitely a tedious and thankless task.

Why personal experience tests are not the answer

I think we agree that salespeople are generally bad advisors. Nowhere does the principle of capitalism fail as badly as in the medical aid market. The marketing at this point is excessive, my favorite example of this is OrCam, which for a while presented itself as a kind of new messiah. But where else would the information come from?

Now there are some people in the blind community who are testing aids. However, such results are always very subjective or distorted. The German INCOBS tests are legendary among blind technology fans because of their miserable quality. Someone who obviously had no idea about Android and Talkback should test how accessible Android was for blind people using an outdated device and an outdated version of all the programs.

And of course no private person has the financial and time resources to extensively test and evaluate all the devices in a category and publish the results, at least if they have a normal job. The best way I could imagine here is a kind of crowdsourcing, in which many individuals evaluate their aids according to certain defined criteria. But this poses the risk of manipulation.

No, the best option would be an institution that does not make money from the sale of a particular aid. It would have to develop and carry out an assessment procedure that is as objective as possible, ideally with test subjects who have been affected themselves. You would have to test the aids in practice for as long as possible. Basically, the health insurance companies should have an interest in such an institution because in Germany they pay for pretty much all the aids, only a small part is covered by other providers such as the integration offices.

Unfortunately, such an institution is not in sight. I would prefer the c't or I fix it, because Braille displays for blind people in particular are often designed in such a way that they cannot be easily repaired. The companies want to collect in several ways: for sales, for maintenance, for replacing the batteries and probably for disposal if the good piece gives up the ghost after 3.5 years. What I like about c't is that it compares products, but doesn't create a generally meaningless ranking list. This makes even less sense in connection with aids than elsewhere, because aids often have to fulfill specific purposes. The 10 best obstacle detection devices would therefore make no sense if there are even 10 devices of this type in Germany.

Read more on assistive technology