Video Games Help U.S. Soldiers Learn Arab Language, Culture

Stefan Lovgren
for National Geographic News
Updated February 21, 2006
Researchers have developed an interactive computer system that uses artificial intelligence and gaming techniques to teach Arabic to U.S. soldiers.

Soldier-students equipped with microphones navigate through an Arabic-speaking environment on a computer screen. If they successfully phrase questions and understand the answers, they can move on to the next level of the game.

But this is more than just a language lab.

The system emphasizes nonverbal behavior. Users are taught to adopt local customs such as putting their right hand over their heart when meeting someone for the first time.

The characters that users face in the game, meanwhile, are animated by artificial intelligence. They may nod in approval or cross their arms with skeptical hostility in response to the users' actions.

It's all part of an effort to create the most believable experience that soldiers can expect to face in a foreign environment, says Hannes Högni Vilhjálmsson, the designer of the computer engine behind the games, which is called Social Puppet.

"Language without any context is hard to learn," said Vilhjálmsson, a research scientist with the Information Sciences Institute at the University of Southern California in Los Angeles.

"But if you put it into the context of face-to-face communication, allowing for gestures and other non-verbal behavior, it becomes easier [to learn]."

All the games in the Tactical Language and Culture series use the Social Puppet software and are designed to help train soldiers.

The first game, called Levantine, taught soldiers the Levantine dialects of Arabic spoken in Lebanon, Syria, and Palestine.

The second program, Tactical Iraqi, has been used by hundreds of soldiers now stationed in Iraq.

Today Vilhjálmsson presented a research prototype for his latest training game, Tactical Pashto, for teaching the Pashto language to soldiers headed to Afghanistan (See an interactive map of Afghanistan.)

He presented the prototype at the American Association for the Advancement of Science's annual meeting in St. Louis, Missouri.

Civil Affairs

Vilhjálmsson's research is financed by the Pentagon's Defense Advanced Research Projects Agency and the Office of Naval Research.

The system was first tested in 2003 by cadets studying Arabic at the U.S. Military Academy at West Point, New York.

It has since been incorporated into the officers' curriculum at the Expeditionary Warfare School at Camp Pendleton in California.

Before users start the game, they learn how to make introductions in the foreign language.

"The most basic skill is to introduce yourself to a local person using the right phrases, the right level of respect for a given situation—tell them your name, ask for their name, and so on," Vilhjálmsson said.

"You then need to use your newly acquired skills in the actual game missions."

In each mission, students must complete an overall task focused on civil affairs and reconstruction efforts. In the Afghan version, the task is to rebuild a clinic in a remote village.

Each story is broken down into different scenes, which the user must navigate successfully to progress to the next level.

A speech recognition system allows the speaker to communicate with characters on the screen. Using the computer mouse, students can also select their own gestures.

"One option might be to take off your sunglasses, allowing you to make eye contact," Vilhjálmsson said. "What you say and how you behave then affect the other game characters."

Entering the Village

In the first scene of the Pashto program, users must make initial contact with the local villagers. But they first encounter a group of children. The correct response is not to ignore them but to ask how they are doing.

"If you do a good job with the children, you end the scene by saying goodbye to them and making contact with two adults who have been watching you," Vilhjálmsson said. "They then lead you to the following scene."

Nonverbal communication is as important as what is spoken, experts point out.

"People naturally tend to rely heavily on nonverbal communication when they are learning a foreign language," said Lewis Johnson, the director of the USC's Information Sciences Institute.

Users can gauge how well they are doing by the reactions of the other characters.

"When having a conversation face to face, people rely on spontaneous nonverbal clues," Vilhjálmsson said.

"Puzzled or offended expressions are much more intuitive clues than a printed message saying, 'Your words not understood.'"

But nonverbal gestures can also be a source of confusion, since people in different cultures use different gestures.

The game's gesture training includes common Arabic mannerisms that a Westerner might misinterpret—for example, Arabs may roll their eyes to mean "no"—and American gestures that an Arab might misinterpret.

"By exposing learners to realistic face-to-face situations and by training them to be culturally sensitive, we prepare them to become effective social players as well as speakers in the new language," Vilhjálmsson said.

Free Email News Updates
Sign up for our Inside National Geographic newsletter. Every two weeks we'll send you our top stories and pictures (see sample).


© 1996-2008 National Geographic Society. All rights reserved.