Challenges Facing CounterTerrorism Analysis

1
Challenges Facing Counter-Terrorism Analysis

• Presented by
• Eric Reffett

2
Understanding the Analyst Culture
The New Threat Spectrum
Traditional Emerging
Force-on-Force Threat
Un-Centric Threat
World Trade Center
Enduring Freedom
Iraqi Freedom
Soviet Union
Target

• Nation states, overt and known

• Transnational, covert and unknown

Indications Warnings

• Red organization and process based

• Individuals and commercial transactions

Strategy

• Assured response

• Pre-emptive interdiction

Planning

• Deliberate

• Adaptive

Intelligence

• Descriptive (operational value)

• Predictive (temporal value)

C3 Operations

• Sequential processes

• Cooperative processes (virtual/collaborative)

Timeline
OODALoop
Deliberate Planning Years/Months
Crisis Action Planning Days/Hours
Hours/Seconds
3
Media Coverage of SNAs Application to
Counter-Terrorism
Can Network Theory Thwart Terrorists? New York
Times Magazine, March 12, 2006
Al Qaedas Web of Terror ABC News, March 10,
2006
this capacity to network and spread propaganda
represents a clear security risk, the most
dangerous and stealthiest use of the Internet by
al Qaeda is for communication, training and
planning purposes.
In the increasingly popular language of network
theory, individuals are "nodes," and
relationships and interactions form the "links"
binding them together by mapping those
connections, network scientists try to expose
patterns that might not otherwise be apparent.
From the Bali bombing in 2002 to the London
attacks last July, every major terrorist
operation undertaken by Osama bin Laden's
organization since 9/11 involved extensive and
clandestine use of the Internet.
Big Brother 101 Anti-terrorism measures
Popular Science, August, 2006
NSA has massive database of Americans' phone
calls USA Today, May 11, 2006
The government is collecting "external" data on
domestic phone calls but is not intercepting
"internals," a term for the actual content of the
communication, according to a U.S. intelligence
official familiar with the program
Whos the most important player in a group?
Whos merely peripheral? Data crunchers find out
by plotting people as nodes on computerized
graphs, forming web-like networks. The links
between nodes are then weighed and analyzed using
matrix algebra and other tools.
The data are used for "social network analysis,"
the official said, meaning to study how terrorist
networks contact each other and how they are tied
together.
4
Data Presents a Significant Challenge
Data Challenges

• Much of the data resides in unstructured formats
• Reports
• Cables, Message Traffic
• Detainee Interviews
• Natural language processing is getting better,
  but the technology is not sufficient
• Requires eyes on the data
• Very demanding and tedious work
• Classification issues still impede access

And then theres the data itself.
5
Bad Or Missing Data Is a Fact of Life
Collected Network

• Our adversaries actively try to keep us from
  finding data or feed us incorrect data
• Typical search techniques start with a few known
  nodes and snowball out from there
• The search generally ends when the data begins to
  wrap in around itself

6
Network Data Can Be Difficult to InterpretAll
Ties Are Not Created Equal
Collected Network

• In our data gathering, we uncover two different
  kinds of ties
• To determine network operations, one must
  distinguish between functional ties and dormant
  (latent) ties

Functional Ties

• Subset of ties that represent the current
  actionable links and connections

Dormant (Latent) Ties

• Historical or incidental associations that
  indicate that two people may be connected or have
  the potential to associate
• Serve as indicators of potential connections, not
  active connections

Functional (Red) Dormant (Blue) Ties
7
Network Data Can Be Difficult to Interpret We
Dont Know What We Dont Know
Collected Network View A
True Network
Collected Network View B
The whole network may look significantly
different than what is presented by the data
8
Context Is Critical to Applying Network Analysis
Tools

• Individual human behavior is complex the
  behavior of humans interacting in groups is
  dauntingly complex
• Nonlinear and multidimensional
• Dynamic and path dependent

• The complexity of groups means that tools output
  and behavioral models will always be
  error-prone
• Applying SNA in any context requires
  subject-matter expertise

• Modeling quantitative data analysis in a vacuum
  will either yield stale macro-level trends, or
  inaccurate predictions of micro-level detail
• Utilizing subject-matter expertise on countries,
  cultures and even on the organizations themselves
  provides the contextual queues to make sense of
  the patterns and trends in the data

9
Context Is Critical to Applying Network Analysis
Tools

• SNA tools and models can be very powerful
• Identify patterns in network data
• Point to features in the network structure
• Identify areas that are of interest and project
  future changes to networks under a variety of
  hypothetical conditions
• However, none of these tools alone can identify
  vulnerabilities, nor can they forecast future
  developments with confidence
• Modeling and quantitative analysis results also
  require translation to be actionable context
  matters for understanding the appropriate actions
• SNA models and heuristics do not distinguish
  between drug dealers and hot dog vendors
  context matters for action!

10
Designing tools without first addressing the
underlying methodology and providing the
appropriate education is counterproductive

• New tools need to be integrated into the
  analysts accepted methodologies
• Analysts are EXTREMELY busy and dont have time
  or want to learn an exotic skill they may or may
  not use frequently
• Powerful tools litter the desktops of analysts
  and get little use
• Expertise is built by the analysts taking an
  interest and learning it on their own, dont even
  ask to send an analyst to a three month course
• Tool experts are out there, but once they get
  reassigned, the skill is generally gone
• SNA experts often speak a different language than
  the analysts - cloaked in academic jargon
• Many analysts just want a tool to do the
  network analysis (Microsoft SNA)
• If it takes longer than learning Word, my
  analysts wont use it

Much of what we do in the SNA community is in
art
11
Tool Development Process
Developing Successful Tools
Developing Inadequate Tools
the way it should be done
the way it is done
Recognize Problem

• In the current threat environment, military
  leaders and policy makers face problems that may
  require tool development
• However, often tools are created and benchmarked
  before a true understanding of the problem is
  reached leading to tools that are disconnected
  from the problems they were meant to solve, and
  benchmarks results that are either meaningless or
  provide a false sense of accomplishment
• Instead, careful analysis of the problem must be
  done before tool development begins, including
  formalizing a methodology for confronting the
  problem and validating that methodology

Develop Solution Methodology
Create Tool
Validate Methodology
Benchmark Tool
Create Tool
Engage Problem
Benchmark Tool