Scientists Taught an Algorithm to Translate Thoughts Into Speech

In “What Women Want,” protagonist Nick Marshall gets electrocuted in a hairdryer-in-the-bathtub mishap. Suddenly, he can hear women’s thoughts as if they were saying them aloud. It’s not exactly a classic, but this film from 2000 has staying power — not just because it was recently remade, but because it’s getting closer to reality. Recently, for the first time ever, neuroengineers at Columbia University made human thought into intelligible, audible speech.

Siri the Mind Reader?

The thought-to-speech breakthrough, detailed in a recent paper from researchers at Columbia University and Hofstra/Northwell School of Medicine, relies on the same technology that powers our robot butlers Siri and Alexa: the vocoder. A vocoder is just a computer algorithm that can synthesize human speech, and in this case, the research team taught such an algorithm to recognize and translate brainwave patterns in a person’s auditory cortex into speech. The auditory cortex is the area of the brain that’s activated when we speak and listen, and — importantly! — when we imagine speaking and listening. In other words, when we think.00:0000:00

Well, actually — pause for a second. The definition of thought is an open philosophical question. It could be considered everything a conscious brain does. In the context of this study, though, it’s “inner speech,” or language we’ve actively called to mind. Conscious brains do many other things, too: They can also, for instance, picture a sunflower, remember a smell, or perform mental math. But words constitute a substantial chunk of our waking thoughts — at least 20 percent of them, by psychologist Charles Fernyhough’s estimate — and verbal thoughts were the focus here.

To teach the vocoder to translate verbal thoughts, researchers relied on brain scans conducted during routine brain surgeries on epileptic patients. They tracked these patients’ auditory cortex activity as they listened to sentences read aloud by a variety of people. Over time, the scans taught the vocoder to match neural activity to simple speech. So far, it can “read” the numbers 0 through 9 from brain activity alone; next, researchers plan to experiment with other words and sentences.

That’s only part of the breakthrough here, though. It’s also groundbreaking that the vocoder’s robo-voice can translate brainwaves reliably and intelligibly. “We found that people could understand and repeat the sounds [of the vocoder] about 75 percent of the time, which is well above and beyond any previous attempts [at translating thought],” said one of the study’s co-authors.

A Voice for the Voiceless

Sure, it’s not as intelligible as a human voice yet, but this new technology still has huge implications — and not just for watchdogs fact-checking “What Women Want.” This technology could one day offer a new way to connect with the world for anyone who can’t speak, whether it’s due to a stroke, amyotrophic lateral sclerosis (ALS), or some other illness or injury.

The research remains in the early stages, but researchers hope that one day, this vocoder technology can be built into an implant — something akin to the chest implant some epilepsy patients wear to reduce the frequency and severity of their seizures. Just like a voice, this implant would allow patients to communicate just by thinking. No pressing of buttons, no writing of notes, not even a need for an eye-tracking interface like Stephen Hawking used.

What about privacy? A device that broadcast your every thought would streamline communication while creating a million other embarrassing problems. Thankfully, the implant researchers envision would only translate thoughts on command. Still, as convenient and groundbreaking as the device would be, it does open up the possibility of hacking into another human’s thoughts — which sounds at the same time alarming, epic, and, if Twitter is any indication of what thought looks like, pretty dull.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this:
search previous next tag category expand menu location phone mail time cart zoom edit close