|
“On August 4, 2001, Mohamed al Kahtani was denied U.S. entry
at Orlando International Airport. On September 11, 2001, while three planes
carried five hijackers, United Flight 93 carried only four. These four were
overpowered by passengers, sparing a potential target in Washington. Federal
investigators believe that al Kahtani was meant to be Flight 93's fifth
hijacker.
Why did Immigration and Naturalization Service inspector Jose Melendez-Perez
turn away al Kahtani on August 4? The inspector said there was something
"chilling" about Kahtani. Melendez-Perez was quoted in the 9/11
Commission Report as saying, "My first question to the subject [through
the interpreter] was why he was not in possession of a return airline ticket.
The subject became visibly upset... in an arrogant and threatening manner,
which included pointing his finger at my face." Thus, though the two
didn't share a spoken language, Melendez-Perez used nonverbal cues to help
make his decision.
Melendez-Perez, like many professionals, relied on his
intuition to understand nonverbal behavior. Can we formalize our
understanding of nonverbal behavior, so that it can be used by the many,
instead of the intuitive few? This is the goal of the MITRE-Sponsored
Research (MSR) project "Understanding (Arabic) Nonverbal Behavior."
Our objective is to make information from nonverbal behavior interpretable
and usable for a broad range of work tasks, including field-based activities,
video analysis interpreting behavior and intent, airport screening, and
immigration, customs, and border patrols.”
Whether Obvious or Subtle, Nonverbals Are Useful
What kinds of nonverbal behavior might be of use? We can
consider three types, along a continuum from obvious to subtle.
One category of easily recognizable gestures, called
"emblems," has codified culture-specific meanings. Examples include
the American "thumbs-up" and "OK" signs (which are both
offensive in some Arabic cultures). Knowledge of such pre-defined signals is
useful to anyone working in multicultural settings.
Other types of nonverbal behavior are less consciously
recognized, but easily identified once pointed out. For example, use of
interpersonal space (proxemics) varies in distance and meaning among
cultures. Many Arabs prefer a closer distance when talking to others than do
Americans. This leads some Arabs to feel Americans are aloof because they
stand too far away, while Americans may feel Arabs are pushy because they
stand too close. Knowledge of these behaviors is beneficial to anyone
engaging in face-to-face interactions with others (such as soldiers or
airport screeners).
Finally, some nonverbals are revealed only by careful
analysis. For instance, some analysts believe that reviewing video for
clusters of behaviors (such as shifting of arms and legs to and from
"rest positions") may signal evasion. Knowledge of subtle cues, how
they are used, and what they mean in different contexts can be beneficial in
both real-time screening as well as forensic video analysts.
Two Objectives
While the MSR's overall goal is to enable and facilitate
greater recognition and interpretation of nonverbal information (beyond
intuition), the project's title—Understanding (Arabic) Nonverbal
Behavior—hints at its two supporting objectives. The first objective is to
provide new methods and enabling technology for analyzing nonverbal behavior
in general. The second objective is to use those methods and technology to
understand nonverbal behavior in specific cultures, starting with the Iraqi
Arabic culture.
What do we mean by "enabling technology"? We are
first refining a methodology for nonverbal analysis. For example, in
analyzing videos of humans in conversation, what kinds of phenomena should be
analyzed or annotated? Gaze? Posture shifts? Eyeblinks? It's impossible to
focus on everything, and choices must be made from a complex array of body
movement. Second, we're investigating tools for such analysis. A sample
analysis tool is ANVIL (Annotation of Video and Language). This software
contains a digital video player at the top, with time-aligned annotation
tracks below; as the video is played forwards or backwards, the annotation
tracks scroll right or left, and vice versa.
In addition to tools for analysis, we are creating further
tools for sharing analyses between research groups, and, finally, devising an
accessible knowledge base for storing, sharing, and retrieving results of
nonverbal analysis. These latter two activities focus on promoting
intracommunity information exchange and are especially important, as relevant
academic data are largely confined to islands of specialized research. There
is currently no easy way to bridge between research groups or between the
laboratory and our sponsors' real-world needs. We plan to leverage MITRE's
information-sharing success in another field: neuroimaging. MITRE is already
helping the brain-mapping community share brain-scan images and metadata; we
envision a similar infrastructure for nonverbal data (videos) and metadata.
The research project's second supporting objective is to use
the above enabling technology to analyze and understand nonverbal behavior
from a particular culture, starting with Iraqi Arabs, whose nonverbals have
rarely been studied. Volunteer native Iraqis have been filmed in a variety of
scenarios, including free conversation and interactive storytelling. The
tools described above will be used to analyze the videos, and the findings
will be entered into the knowledge base for dissemination across research and
other potential communities of interest (COIs).
Real-world Relevance
How will these findings help MITRE's sponsors? There are
several possibilities. The online knowledge base of nonverbal behaviors and
their meanings will be useful to those who need insight into interactions
with Iraqis. Articles and white papers describing these behaviors, and the
methods used to analyze and interpret them, will be published for the use of
researchers and other COIs. These findings will be particularly useful for
training materials designed for government personnel who interact with other
cultures. For example, many government personnel going to Iraq are equipped
with an illustrated laminated pocket card that contains basic tips and
phrases to aid intercultural communication. We also envision collaborating
with immersive simulated training environments to provide better bodymovement
realism in the synthetic actors with which soldiers practice communicating.
Each of these training media is already making use of easily recognizable
emblematic gestures. We are collaborating with the video-gaming group in
MITRE's Command and Control Center to ensure our nonverbal findings can be
integrated into immersive training environments.
Impact
Our sponsors go to great expense to understand a variety of
human communication channels. Yet the ubiquitous channel of nonverbal
behavior is under-exploited. Our goal is to increase the safety and
effectiveness of U.S. personnel by providing a clearly-documented
understanding of Iraqi Arabic nonverbal behavior that will also serve as a
prototype for how to provide comparable insights into other cultures. We will
also create the first instance of enabling technology infrastructure to allow
this knowledge to be used more broadly outside research laboratory settings.
MITRE's sponsors are realizing the need for cultural knowledge
such as this and have begun coming to us for help. Few entities in the
research community have the understanding of technology, social sciences, and
sponsor requirements to tackle the significant challenges here. Thus, MITRE
is well positioned to move this field forward and empower our sponsors with
important and underutilized data on human communication.
|
