Abductive Inference of Behavior Models

1
Abductive Inference of Behavior Models

• Dana Nau
• Director, Laboratory for Computational Cultural
  Dynamics (LCCD)
• University of Maryland, College Park, MD
• http//www.umiacs.umd.edu/research/LCCD

2
Introduction

• Computer technology is producing huge changes in
  how we can reason about groups in diverse
  cultures
• Gather data about different cultural groups
• Learn intensity of opinions on various topics
• Build/extract models of behavior
• Refine those behaviors through shared,
  multi-person, learning experiences
• Inherently cross-disciplinary
• Behavioral and social sciences
• political science, psychology, journalism,
  anthropology, sociology
• Technological fields
• computer science, game theory, operations
  research
• LCCD is a cross-disciplinary laboratory to
  facilitate these developments
• Faculty from Computer Science, Political Science,
  Psychology, Criminology, Linguistics, Public
  Policy, Business, Systems Engineering

3
Abductive Inference of Behavior Models

• Want to build models of the behavior of groups
  (political organizations, terror groups,
  corporations, etc.) quickly and accurately
• Abductive inference problem
• Find a behavior model M such that M ? the
  groups observed behavior
• Motivation use such models in order to
• Predict the most probable responses ofa given
  group in a situation (real or hypothetical)
• Identify what actions we can take in order to
  maximize theprobability of eliciting a desired
  response from a group
• Want to do this in near-real time
• Groups we have modeled
• Various tribes on the Pakistan/Afghanistan
  borderlands
• Players in the Pakistan/Afghanistan drug economy
• Hezbollah
• Fatah Revolutionary Council Abu Nidal
  Organization

4
Process
Unclassified sources
Focus ofmy talk
3. Build Behavioral Models
4. Explore Repercussions Find best COAs
1. Extract Timely, relevant data
2. AssessIntensity ofOpinions
V. Subrahmanian, M. ALbanese, M. V. Martinez, D.
Nau, D. Reforgiato, G. I. Simari, A. Sliva, O.
Udrea, and J. Wilkenfeld.CARA A
cultural-reasoning architecture. IEEE Intelligent
Systems, March/April 2007.
Classified sources
5
Step 1 Gather relevant and timely data
Global Terror DB
Environment Library
Stochastic Opponent Model Extractor
Minorities at Risk DB
Virtual Exploration Environment
Counterterror data
Realtime Extracted DBs
Open-source data News Blogs Newsgroups Social
network Sites Classified Data
T-REXInfo Extractor
SOMAModel Library
Character Library

• Use existing databases (classified open)
• Mine information on some group of interest
• structure, activities, history, funding,
  relationships, etc.

OASYSOpinion Extractor
6
T-Rex
Five thousand Hazara Afghans were massacred by
the Taleban in Mazar-e-Sharif (Source
http//www.hraicjk.org/archives2.html)

• Uses existing databases (classified open)
• Mines information on some group of interest
• Structure, activities, history,
  funding,relationships, etc.
• Automatically extracts RDF triples
  fromEnglish-language text
• About 40K-45K documents per day
• STORY (an early version of T-Rex)got honorable
  mention for a2005 Computerworld Horizon Award
• Work with Wilkenfelds Center for the Study
  ofTerrorism and Responses to Terrorism (START)
• Gather information for theirMinorities at Risk
  databases
• Validation/tuning from theirexisting databases
  on theBasques and Kikuyus

7
Step 2 Assess intensity of opinions
Global Terror DB
Environment Library
Stochastic Opponent Model Extractor
Minorities at Risk DB
Virtual Exploration Environment
Counterterror data
Realtime Extracted DBs
Open-source data News Blogs Newsgroups Social
network Sites Classified Data
T-REXInfo Extractor
SOMAModel Library
Character Library

• Assess a groups intensity of opinion on topics
  that
• might influence their actions
• How strongly does Afghan press feel about
  Karzai?
• Can we quantify this accurately?

OASYSOpinion Extractor
8
OASYS Opinion Extractor

• Analyzes about 12K news articles per day
• Currently handles eight languages
• English, Italian, Spanish, French,Arabic,
  Korean, Chinese, Russian
• Graphs intensity of opinion on a given topic
• Input sources, time frame, topic
• Different curves
• country and/or news source
• High correlation with human users
• Pearson correlation 0.47 withhuman evaluators
• Beats all competitors we know of
• Winner 2006 ComputerWorld Horizon Award
• Press Computerworld magazine, Aug. 21
• Panorama magazine, Sep. 21
• RAI-3 TV Italy, newscast Nov. 6

9
Technology Transfer

• Data on 7-8 tribes on the Pakistan/Afghanistan
  borderlands
• Sent to the Armys 10th Mountain Division in
  early 2006 prior to their deployment there
• Working with US Army/AMSAA
• Apply our technology to assess impact of
  intelligence collection operations on
  inter-tribal relationships
• Working with the US Naval Research Lab
• Advanced visualization of STORY/T-REX related data

10
Step 3 Build Behavior Models
Global Terror DB
Environment Library
Stochastic Opponent Model Extractor
Minorities at Risk DB
Virtual Exploration Environment
Counterterror data
Realtime Extracted DBs
T-REXInfo Extractor
SOMAModel Library
Character Library
OASYSOpinion Extractor
11
Step 3 Build Behavior Models
Probability intervals, so that we can model cases
where conditions arent independent

• SOMA rules
• Extension of probabilisticlogic programs
• Group g takes set of actions A withprobability
  in the interval p1, p2when condition C holds
• Abductive inference problem
• Build a SOMA model(set of SOMA rules)that
  explainsobserved behavior
• Ill briefly discuss two examples
• Noisy IPD
• Hezbollah

G. Simari, A. Sliva, V. Subrahmanian, and D. Nau.
A stochastic language for modeling opponent
agents. In International Joint Conference on
Autonomous Agents and Multiagent Systems (AAMAS),
2006 S. Khuller, V. Martinez, D. Nau, G. Simari,
A. Sliva, and V. Subrahmanian. Finding most
probable worlds of probabilistic logic programs.
In International Conference on Scalable
Uncertainty Management (SUM 2007), Oct. 2007
12
Iterated Prisoners Dilemma (IPD)

• Axelrod (1984), The Evolution of Cooperation
• Two players, finite numberof iterations of the
  Prisoners Dilemma
• Widely used to study emergence ofcooperative
  behavior among agents
• No optimal strategy
• Performance depends on thestrategies of all of
  the players
• The best strategy in Axelrods tournaments
• Tit-for-Tat (TFT)
• On 1st move, cooperate. On nth move,repeat the
  other players (n1)-th move
• Could establish and maintain advantageouscooperat
  ions with many other players
• Could prevent malicious players fromtaking
  advantage of it

Payoff matrix
If I defect now, he might punish me by defecting
next time
13
IPD with Noise
C
C

• Noise can model accidents and misinterpretations
• Theres a nonzero probability (e.g., 10)that a
  noise gremlin will changesome of the actions
• Cooperate (C) willbecome Defect (D),and vice
  versa
• Tit-for-Tat and other strategiesfail to maintain
  cooperation

C
C
Noise
C
C
He defected, so Ill defect next time
C
C
D
D
C
He defected, so Ill defect next time
C
D
He defected, so Ill defect next time
D
C
C
D
He defected, so Ill defect next time


14
Opponent Modeling

• We cant know which actions wereaffected by
  noise
• But if we have a good opponentmodel, we can make
  good guesses
• Our DBS program
• Observe other players behavior
• Keep track of how oftenvarious behaviors
  occurunder various circumstances
• Build a rule-based model p ofthe other players
  strategy
• Special case of SOMA rules
• Noise detection
• If opponents actions disagree with p, assume
  its noise
• If opponents actions disagree with p too many
  times
• Assume the opponents strategy has changed
• Recompute p based on the opponents recent
  behavior

T.-C. Au and D. Nau. Accident or intention That
is the question (in the iterated prisoners
dilemma). In International Joint Conference on
Autonomous Agents and Multiagent Systems (AAMAS),
2006. T.-C. Au and D. Nau. Is it accidental or
intentional? a symbolic approach to the noisy
iterated prisoners dilemma. In G. Kendall,
editor, The Iterated Prisoners Dilemma
Competition Celebrating the 20th Anniversary.
World Scientific, 2007, to appear.
15
Planning DBSs Actions

• Game tree search against the opponent model p
• Problem game trees grow exponentially with
  search depth
• Key assumption p accurately models the other
  players future behavior
• Then we can use dynamic programming
• Makes the search polynomial in the search depth
• Can easily search to depth 60
• This generates fairly good moves

Current iteration
(C,C) (C,D) (D,C) (D,D)
Next iteration
Iteration after next




16
The 20th-AnniversaryIterated Prisoners Dilemma
Competition

• http//www.prisoners-dilemma.com
• Category 2 IPD with noise
• 165 programs participated
• DBS dominated the top 10 places
• Two programs beat DBS
• Both used a master-and-slaves strategy that
  came dangerously close to cheating

17
Master and Slaves
My strategy? Iorder my goons togive me all
their money

• Each participant could submit up to 20 programs
• Some submitted programs that worked as teams
• 1 master, 19 slaves
• When slaves play with master, they cooperate
  andthe master defects, so the master gets all
  the points
• When slaves play with anyone not in their team,
  they defect
• Analysis
• Average score of each master-slaves team was much
  lower than DBSzs
• If BWIN and IMM01 each had 10 slaves, DBS would
  have placed 1st
• If BWIN and IMM01 had no slaves, they would have
  done badly
• Unlike BWIN and IMM01, DBS had no slaves
• None of the DBS programs even knew the others
  were there
• DBS worked by establishing cooperation with many
  other agents
• DBS could do this despite the noise, because it
  could filter out the noise

I order mygoons tobeat upthe others
18
Example 2 Hezbollah

• Suppose Hezbollah is not engaged in non-suicide
  attacks
• Let x be the probability that Hezbollah will
  engage in suicide attacks
• Let y be the probability that they will engage in
  suicide attacks when education and propaganda is
  NOT part of their strategy
• Can you guess x and y?
• Just ballpark estimates?
• Which is higher?

19
A (preliminary) Hezbollah rule

• Transnational Targets outside country boundaries
  chosen are based on ethnicity
• If 1. It is based within the country it lives in
  
• 2. Severity of conflict with highest level of
  inter-org conflict involves substantial numbers
  of people
• 3. Electoral politics is a minor strategy

Probability 0.875
20
Another (preliminary) Hezbollah rule

• Transnational Targets outside country boundaries
  chosen are based on ethnicity
• If 1. It is based within the country it lives in
  
• NOT
• (2. Severity of conflict with highest level of
  inter-org conflict involves substantial numbers
  of people
• 3. Electoral politics is a minor strategy )

Probability 0.133
21
Behavioral Rule Extraction

• Number of rules extracted automatically for
  Hezbollah for the following actions
• ARMATTACK 949
• BOMB 62
• DSECGOV 1011 (org. targets domestic state lives
  and security organizations)
• KIDNAP  2830
• TLETHCIV  9999 (org. chooses transnational
  targets based on ethnicity)
• Over 14000 rules extracted in total

22
Step 4 Explore Repercussions, Find Best COAs

• Immersive Virtual Reality

23
Step 4 Explore Repercussions, Find Best COAs

• A 3-d virtual environment
• Landscape resembles the real landscape of the
  part of the world being modeled
• Characters resemble the people in the part of the
  world being modeled
• Aggregate behavior of the characters conforms to
  the SOMA models of the characters involved
• US decision makers can try to
• Decide what repercussions potential US policies
  might have
• How best to play out the actions of multiple
  groups in a region

GOAL Experimental tool to help US DoD/DHS
decision makers anticipate the potential
cultural/religious/social impact of possible
actions
24
Challenges

• Groups can act in a potentially HUGE number of
  ways
• Need scalable methods to use the behavioral
  models of an opponent group in order to
  understand what they may or may not do
• Example
• Suppose there are 1000 actions that we are
  interested in modeling for a group
• There are 21000 ? 10330 ways in which the
  opponent can act
• So far, we can deal with around 1027 possible
  ways in a reasonable amount of time
• Number of atoms on earthabout 1050
• Number of particles in theuniverse about 1087

KEY CHALLENGE increase the number of possible
courses of action we can consider, by several
orders of magnitude
25
ICCCD

• First International Conference on Computational
  Cultural Dynamics
• University of Maryland, August 27-28
• http//www.umiacs.umd.edu/conferences/icccd2007
• Supported in part by
• AFOSR
• AAAI
• LCCD
• UMIACS

26
Contact Information

• CREDITS
• Key team members
• V.S.Subrahmanian (lead)
• Max Albanese
• Tsz-Chiu Au
• Vanina Martinez
• Mary Michael
• Dana Nau
• Diego Reforgiato
• Gerardo Simari
• Amy Sliva
• Jon Wilkenfeld
• Affiliates
• F. Benamara
• B. Dorr

• Laboratory for Computational Cultural Dynamics
• http//www.umiacs.umd.edu/research/LCCD
• Dana S. Nau
• Tel [phone]
• Email nau_at_cs.umd.edu
• Web www.cs.umd.edu/nau
• V.S. Subrahmanian
• Tel [phone]
• Email vs_at_umiacs.umd.edu
• Web www.cs.umd.edu/vs