Sense Subsitution for better Accessibility
Anyone who has ever tried to help a person who was born completely blind explaining colors knows how difficult that is. The blind can grasp shapes haptically, the deaf can feel the rhythms of the music as vibrations. Intellectually, the blind can grasp that a certain shade of red stands for danger or green represents freshness, but colors and music do not work intellectually, but on the emotional level and these emotions are difficult to grasp if the corresponding sense is missing or disturbed.
Article Content
Sounds instead of Language
While language can be an adequate substitute for images, it is not always so. For example, I never got used to audio description for movies. This may also be due to the fact that most films that are equipped with it are not my cup of tea. My problem is that the audio description always lies over the film like a foreign body and is not inserted organically. It is often pretended that AD blind persons have a similar perception of the film as sighted persons, but that is wrong.
A promising approach to approach this problem in a way other than purely verbal-descriptive is to replace one sense with another that works but can have a similar effect. For example, a simple, if inadequate, substitute for music or recorded voice is the envelope, which represents a sound as a diagram. Anyone who has ever edited interviews knows the characteristic form of the "er", which looks something like the letter h. Professionals can see even more patterns in the envelope of a voice recording, making editing easier.
Sounds are an almost natural substitute for color. High tones and violins for bright colors, low tones and bass for dark colors.
Sounds become spaces
While we're on the subject of sounds, we can also use them to describe spaces. I am elsewhere already on the click sonar technology, in which blind persons use clicking noises and their echo to obtain a kind of three-dimensional image of a town. If we translate a room into music, we could say, for example, that the music gets quieter the further the room is or an object is away, and louder the smaller the room or the closer the object is.
Music can be translated into sign language for the deaf. How this works in practice, I can't say for lack of personal experience. I could also imagine translating musical instruments as colors. I already mentioned above how colors could be translated into sounds. Therefore, the opposite way is also conceivable, music is translated into colors and supplemented with sign language. Together with the movement of the dancers and the gestures, there is an optical equivalent to the music. An American project that wants to make music accessible to the deaf is the Deaf Performing Artists Network.
sensory replacement devices
The classic cane has held up well, and new electronic orientation aids such as ultrasound systems have only come onto the market in recent years. These systems provide haptic or acoustic feedback if, for example, an object is in the way. With more complex feedback, it would theoretically also be possible to record the shape and thus perhaps even the type of object. If this works with click sonar, it should also work with ultrasound. Well-known devices from this area are for example K-Sonar or iGlasses Ultrasonic Mobility Aid. I've never used either device so I can't say anything about their suitability. If you want to build it yourself, Sonar Eye is an open variant of these systems.
Perhaps an even more promising approach is sensory substitution devices. An Israeli researcher is working on a system that outputs the color, position and shape of an object as tones and tone sequences.
It doesn't sound that spectacular at first, but in fact users can not only recognize individual objects, which is great for a blind person. With a few hours of practice they can even recognize facial expressions. We are still at the beginning of this development, so it is difficult to say how much compensation is possible, but the approaches certainly sound promising.