Are First Impressions Useful? Do They Keep Us Safe?

“On August 4, 2001, Mohamed al Kahtani was denied U.S. entry at Orlando International Airport. On September 11, 2001, while three planes carried five hijackers, United Flight 93 carried only four. These four were overpowered by passengers, sparing a potential target in Washington. Federal investigators believe that al Kahtani was meant to be Flight 93's fifth hijacker. Why did Immigration and Naturalization Service inspector Jose Melendez-Perez turn away al Kahtani on August 4? The inspector said there was something "chilling" about Kahtani. Melendez-Perez was quoted in the 9/11 Commission Report as saying, "My first question to the subject [through the interpreter] was why he was not in possession of a return airline ticket. The subject became visibly upset... in an arrogant and threatening manner, which included pointing his finger at my face." Thus, though the two didn't share a spoken language, Melendez-Perez used nonverbal cues to help make his decision. Melendez-Perez, like many professionals, relied on his intuition to understand nonverbal behavior. Can we formalize our understanding of nonverbal behavior, so that it can be used by the many, instead of the intuitive few? This is the goal of the MITRE-Sponsored Research (MSR) project "Understanding (Arabic) Nonverbal Behavior." Our objective is to make information from nonverbal behavior interpretable and usable for a broad range of work tasks, including field-based activities, video analysis interpreting behavior and intent, airport screening, and immigration, customs, and border patrols.” See more of this article at http://www.mitre.org/news/the_edge/summer_08/friedland_loehr.html Whether Obvious or Subtle, Nonverbals Are Useful What kinds of nonverbal behavior might be of use? We can consider three types, along a continuum from obvious to subtle.   One category of easily recognizable gestures, called "emblems," has codified culture-specific meanings. Examples include the American "thumbs-up" and "OK" signs (which are both offensive in some Arabic cultures). Knowledge of such pre-defined signals is useful to anyone working in multicultural settings. Other types of nonverbal behavior are less consciously recognized, but easily identified once pointed out. For example, use of interpersonal space (proxemics) varies in distance and meaning among cultures. Many Arabs prefer a closer distance when talking to others than do Americans. This leads some Arabs to feel Americans are aloof because they stand too far away, while Americans may feel Arabs are pushy because they stand too close. Knowledge of these behaviors is beneficial to anyone engaging in face-to-face interactions with others (such as soldiers or airport screeners). Finally, some nonverbals are revealed only by careful analysis. For instance, some analysts believe that reviewing video for clusters of behaviors (such as shifting of arms and legs to and from "rest positions") may signal evasion. Knowledge of subtle cues, how they are used, and what they mean in different contexts can be beneficial in both real-time screening as well as forensic video analysts. Two Objectives While the MSR's overall goal is to enable and facilitate greater recognition and interpretation of nonverbal information (beyond intuition), the project's title—Understanding (Arabic) Nonverbal Behavior—hints at its two supporting objectives. The first objective is to provide new methods and enabling technology for analyzing nonverbal behavior in general. The second objective is to use those methods and technology to understand nonverbal behavior in specific cultures, starting with the Iraqi Arabic culture. What do we mean by "enabling technology"? We are first refining a methodology for nonverbal analysis. For example, in analyzing videos of humans in conversation, what kinds of phenomena should be analyzed or annotated? Gaze? Posture shifts? Eyeblinks? It's impossible to focus on everything, and choices must be made from a complex array of body movement. Second, we're investigating tools for such analysis. A sample analysis tool is ANVIL (Annotation of Video and Language). This software contains a digital video player at the top, with time-aligned annotation tracks below; as the video is played forwards or backwards, the annotation tracks scroll right or left, and vice versa. In addition to tools for analysis, we are creating further tools for sharing analyses between research groups, and, finally, devising an accessible knowledge base for storing, sharing, and retrieving results of nonverbal analysis. These latter two activities focus on promoting intracommunity information exchange and are especially important, as relevant academic data are largely confined to islands of specialized research. There is currently no easy way to bridge between research groups or between the laboratory and our sponsors' real-world needs. We plan to leverage MITRE's information-sharing success in another field: neuroimaging. MITRE is already helping the brain-mapping community share brain-scan images and metadata; we envision a similar infrastructure for nonverbal data (videos) and metadata. The research project's second supporting objective is to use the above enabling technology to analyze and understand nonverbal behavior from a particular culture, starting with Iraqi Arabs, whose nonverbals have rarely been studied. Volunteer native Iraqis have been filmed in a variety of scenarios, including free conversation and interactive storytelling. The tools described above will be used to analyze the videos, and the findings will be entered into the knowledge base for dissemination across research and other potential communities of interest (COIs). Real-world Relevance How will these findings help MITRE's sponsors? There are several possibilities. The online knowledge base of nonverbal behaviors and their meanings will be useful to those who need insight into interactions with Iraqis. Articles and white papers describing these behaviors, and the methods used to analyze and interpret them, will be published for the use of researchers and other COIs. These findings will be particularly useful for training materials designed for government personnel who interact with other cultures. For example, many government personnel going to Iraq are equipped with an illustrated laminated pocket card that contains basic tips and phrases to aid intercultural communication. We also envision collaborating with immersive simulated training environments to provide better bodymovement realism in the synthetic actors with which soldiers practice communicating. Each of these training media is already making use of easily recognizable emblematic gestures. We are collaborating with the video-gaming group in MITRE's Command and Control Center to ensure our nonverbal findings can be integrated into immersive training environments. Impact Our sponsors go to great expense to understand a variety of human communication channels. Yet the ubiquitous channel of nonverbal behavior is under-exploited. Our goal is to increase the safety and effectiveness of U.S. personnel by providing a clearly-documented understanding of Iraqi Arabic nonverbal behavior that will also serve as a prototype for how to provide comparable insights into other cultures. We will also create the first instance of enabling technology infrastructure to allow this knowledge to be used more broadly outside research laboratory settings. MITRE's sponsors are realizing the need for cultural knowledge such as this and have begun coming to us for help. Few entities in the research community have the understanding of technology, social sciences, and sponsor requirements to tackle the significant challenges here. Thus, MITRE is well positioned to move this field forward and empower our sponsors with important and underutilized data on human communication.

Patti Wood, MA, Certified Speaking Professional - The Body Language Expert. For more body language insights go to her website at http://PattiWood.net. Also check out the body language quiz on her YouTube Channel at http://youtube.com/user/bodylanguageexpert.