Artificial intelligence will enable non-invasive reading of thoughts, converting them to text – Human – Science and technology

Artificial intelligence could help people who have lost the ability to speak. Scientists at the University of Texas have developed a decoder that will enable the so-called non-invasive reading of thoughts.

An artificial intelligence device can translate thoughts into text using brain imaging.

The decoder was able to reconstruct speech with incredible accuracy using only functional magnetic resonance imaging (fMRI). The participants of the study listened to the story, or even just silently imagined it. Until now, similar devices required the introduction of a surgical implant, but the new method is non-invasive and brings hope that new methods will be created to restore speech to people who have lost the ability to speak, for example due to stroke or amyotrophic lateral sclerosis (ALS).

Read also Living alone is the same health risk as smoking, study says

Neurologist Alexander Huth, who led the study, noted that he was somewhat shocked by the accuracy of the device. “We were quite shocked that it worked so well. I’ve been working on it for 15 years… So it was exciting when it finally started working,” he said.

This achievement overcomes a fundamental limitation of fMRI, which is that while the technique can capture brain activity at a specific location with incredibly high resolution, it has a time delay that makes it impossible to track brain activity in real time. The delay arises because fMRI maps neuronal activity indirectly in response to changes in blood flow.

“It’s a noisy, slow proxy for neural activity,” Huth said. This limitation makes the ability to interpret brain activity following natural speech difficult, as it provides a “.mess of information” spread over seconds. However, the development of so-called large language models, i.e. those on which OpenAI’s ChatGPT chatbot is based, for example, opens a new path.

Read also ChatGPT provides therapy, can lead to a revolution in mental health

Large language models are able to numerically represent the semantic meaning of speech, which has allowed scientists to track which patterns of neuronal activity correspond to strings of words with a particular meaning, rather than trying to read the activity word by word.

The procedure was difficult, each of the three volunteers had to spend 16 hours in the magnetic resonance machine and listen to podcasts. The decoder learned to assign meaning to brain activity using the large language model GPT-1, the predecessor of the ChatGPT model. Later, the same participants were recorded listening to a different story or imagining they were telling a story, and a decoder was used to generate the text based on brain activity alone. In about half of the cases, the text matched, sometimes very precisely, the intended meaning of the original words.

Read also Scientists have improved magnetic resonance imaging. The brain scan is thus 64 million times sharper

“Our system works at the level of thought, semantics, meaning,” Huth said, noting that the device captures not the exact words but the essence of the message. For example, when a participant heard the sentence “I don’t have a driver’s license yet,” the decoder interpreted it as “She hasn’t even started learning to drive yet.” Another time, conversely, the sentence “I didn’t know whether to scream, cry, or run away. Instead, I said, ‘Leave me alone!'” he translated as “She started screaming and crying, and then she just said, ‘I told you to leave me alone.'”

Study participants also watched short videos and the device then described them based on their brain activity. In some cases, the machine got it wrong, it has problems especially with personal pronouns. The decoder was also personalized so it had unintelligible output when used on another person. Participants could also trick the system by imagining animals or another story.