Published on
Updated
Reading 2 min.
Masahiro Fujimoto, a Japanese eSports player, is blind and often needs a friend to help him get around Tokyo. But he hopes artificial intelligence, seen as a promising tool for people with disabilities, can help him get around on his own.
This player of the fighting game “Street Fighter” tested the latest version of the ChatGPT conversational agent by going to the stadium during a recent Para eSports meeting.
“I can’t go to an event like this without having someone to rely on.“, the 26-year-old man explained to AFP.If I can use technology like ChatGPT to design my own assistance, that would be great.“.
This year, US company OpenAI launched GPT-4o, which has voice, text and visual commands in multiple languages.
The generative tool, like others such as Google’s Gemini, is part of a growing field that experts say could make education, jobs and everyday services more accessible.
Following the tactile paving of Japanese sidewalks, “Mashiro,” his online pseudonym, uses his cane decorated with a little monkey to find his way from the train station.
As he walks, he talks to GPT-4o as if he were a friend, receiving responses through one earpiece, leaving his other ear free to listen to surrounding noises.
AI’s “huge” potential
After asking for simple directions, he adds: “Actually, I am blind. Could you give me some more details for the blind?”
“Of course“, the AI replies.”You may notice an increase in crowd noise and activity as you get closer“.
The journey, which takes 20 minutes for sighted people, takes four times longer for Mashiro, who has had to turn around several times.
Caught in a sudden downpour, he asks for help from the friend who is accompanying him to complete the journey.
“Arrived !“, finally exclaims Mashiro, blind since birth and who only uses sound to fight his opponents in “Street Fighter 6”.
AI can better meet specific needs than standard assistive products and technologies, says Youngjun Cho, associate professor of computer science at University College London (UCL).
“Its potential is enormous“, says Mr Cho, who also works at UCL’s Global Disability Innovation Hub.I think it can empower many people and encourage independence.“.
Trying to match the human eye
People with hearing loss can, for example, use AI speech-to-text transcription, while chatbots can help format a resume for someone with learning disabilities.
Some tools for the visually impaired, such as Seeing AI, Envision AI, and TapTapSee, describe images taken by a phone’s camera.
And Danish app Be My Eyes is collaborating with OpenAI to develop a “digital visual assistant.”
Masahide Ishiki, a Japanese expert on disability and digital accessibility, warns, however, that it can be “tricky” to detect errors in ChatGPT, which “responds so naturally“.
“The next goal (of generative AI) is to improve the accuracy of real-time visual recognition, in order to achieve capabilities close to those of the human eye.“, says Mr. Ishiki, himself blind.
Marc Goblot of the Tech for Disability group also warned that AI is being trained on “very general public datasets“who are not”not representative of the whole spectrum of people’s perceptions and especially at the margins“.
Mashiro believes that ChatGPT’s limited recognition of Japanese words and locations made his AI-assisted journey more difficult.
Although the experience was “a lot of fun,” it would have been easier if ChatGPT had been connected to a mapping tool, thinks the player, who traveled across Europe last year using Google Maps and the help of his entourage.