He presented the prototype at the American Association for the Advancement of Science's annual meeting in St. Louis, Missouri.
Vilhjálmsson's research is financed by the Pentagon's Defense Advanced Research Projects Agency and the Office of Naval Research.
The system was first tested in 2003 by cadets studying Arabic at the U.S. Military Academy at West Point, New York.
It has since been incorporated into the officers' curriculum at the Expeditionary Warfare School at Camp Pendleton in California.
Before users start the game, they learn how to make introductions in the foreign language.
"The most basic skill is to introduce yourself to a local person using the right phrases, the right level of respect for a given situationtell them your name, ask for their name, and so on," Vilhjálmsson said.
"You then need to use your newly acquired skills in the actual game missions."
In each mission, students must complete an overall task focused on civil affairs and reconstruction efforts. In the Afghan version, the task is to rebuild a clinic in a remote village.
Each story is broken down into different scenes, which the user must navigate successfully to progress to the next level.
A speech recognition system allows the speaker to communicate with characters on the screen. Using the computer mouse, students can also select their own gestures.
"One option might be to take off your sunglasses, allowing you to make eye contact," Vilhjálmsson said. "What you say and how you behave then affect the other game characters."
Entering the Village
In the first scene of the Pashto program, users must make initial contact with the local villagers. But they first encounter a group of children. The correct response is not to ignore them but to ask how they are doing.
"If you do a good job with the children, you end the scene by saying goodbye to them and making contact with two adults who have been watching you," Vilhjálmsson said. "They then lead you to the following scene."
Nonverbal communication is as important as what is spoken, experts point out.
"People naturally tend to rely heavily on nonverbal communication when they are learning a foreign language," said Lewis Johnson, the director of the USC's Information Sciences Institute.
Users can gauge how well they are doing by the reactions of the other characters.
"When having a conversation face to face, people rely on spontaneous nonverbal clues," Vilhjálmsson said.
"Puzzled or offended expressions are much more intuitive clues than a printed message saying, 'Your words not understood.'"
But nonverbal gestures can also be a source of confusion, since people in different cultures use different gestures.
The game's gesture training includes common Arabic mannerisms that a Westerner might misinterpretfor example, Arabs may roll their eyes to mean "no"and American gestures that an Arab might misinterpret.
"By exposing learners to realistic face-to-face situations and by training them to be culturally sensitive, we prepare them to become effective social players as well as speakers in the new language," Vilhjálmsson said.
Free Email News Updates
Sign up for our Inside National Geographic newsletter. Every two weeks we'll send you our top stories and pictures (see sample).
SOURCES AND RELATED WEB SITES