"Silent" Speech Device May Aid Divers, Firefighters, Cell Phone Users

Stefan Lovgren
for National Geographic News

December 27, 2006

It's technology that lets you speak your mind—literally.

NASA scientists are developing a speech recognition system that can understand and relay words that haven't been said out loud.

The system uses electrodes attached to the throat to detect biological signals that occur as a person reads or talks to him- or herself.

The signals can then be converted into text or synthesized speech.

"Subvocal speech recognition is basically the understanding of words without the requirement for sound," said Chuck Jorgensen, a scientist at the NASA Ames Research Center in Moffett Field, California, whose team is developing the system.

"We're looking at the neuromuscular patterns being sent through the nervous system and inferring from those patterns what words would have been said out loud had a person actually permitted himself to produce the acoustics."

The technology could one day be used for communication in high-noise environments or in "silent" cell phone calls.

Like "Watching" Music

Even when a person reads or mouths words silently, electrical signals are sent from the brain to the tongue and vocal chords.

Technology is under development to read similar signals directly from the brain, allowing a person wearing a special cap to, for example, operate a computer with their thoughts.

(Related news: "Man Moves Objects With His Mind Using New Sensor" [July 12, 2006].)

But the NASA team's device, Jorgensen said, focuses instead on the muscles receiving the brain's signals.

"Our problem is analogous to having a deaf person sit in an orchestra audience watching someone play an instrument like a flute and, by looking only at the finger position [of the flute player], trying to figure out what music is being played," Jorgensen said.

"In our case, the finger positions are actually the signals that we're getting from muscles that control the speech articulators."

In preliminary experiments, the scientists used button-size sensors placed under the chin and on the throat to gather nerve signals.

The signals were sent to a processor and then to special software programmed to recognize words such as "stop," "go," "left," and "right" and the digits "zero" through "nine."

Initial word-recognition results were 92 percent accurate. But as of now the system can only recognize 25 to 30 words.

In the future the scientists would like to focus on detecting phonemes—units of sound that are the building blocks of words.

"When you can do that, the number of words [the software can recognize] becomes irrelevant," Jorgensen said.

Can You Hear Me Now?

According to Jorgensen, the idea for his research was initially spurred by rude diners.

"I was sitting in a restaurant with people talking on their cell phones and I was saying, If only their [conversations] were silent," he said.

Reducing background chatter from phone calls is only the tip of the iceberg when it comes to applications for subvocal speech recognition, Jorgensen added.

The technology could be used by astronauts, whose voices can become distorted inside a spacesuit, or by divers, who would be able to talk underwater even while wearing a mouthpiece.

Such devices could also allow firefighters working inside a noisy burning building to communicate with the fire captain stationed outside.

"The system is immune to [background] noise," Jorgensen said.

A separate team of scientists based at Carnegie Mellon University in Pittsburgh, Pennsylvania, is using subvocal technology to develop a wearable translator.

Their device uses electrodes attached to muscles on the face and neck to pick up voice patterns and convert the signals into text.

That text is then translated to a different language and synthetically spoken by the device (related interactive feature: Egyptian hieroglyphs translator).

Tanja Schultz, a researcher with the project, says the effect is like the real-life equivalent of watching a television show that has been dubbed into a foreign language.

It would really allow us to tear down language borders, Schultz told the Tartan, Carnegie Mellon's student newspaper.

In the future, the NASA researchers say, their device may be able to make out not only which words people are saying but also how they're being said.

"We call that speech enrichment," Jorgensen said.

"A lot of stuff comes down this nervous system channel besides a particular word," he said.

"We may be able to detect if you just had a cup of coffee or if you're feeling happy or sad."

Free Email News Updates
Best Online Newsletter, 2006 Codie Awards
Sign up for our Inside National Geographic newsletter. Every two weeks we'll send you our top stories and pictures (see sample).