Waging War Against the New Cyberwarrior

1
Waging War Against the New Cyberwarrior

• Tom Longstaff
• tal_at_cert.org
• CERT Coordination Center
• Software Engineering InstituteCarnegie Mellon
  UniversityPittsburgh, PA 15213
• Sponsored by the U.S. Department of Defense

2
Incidents Reported to CERT/CC
2001 52,658 2002 82,094
3
Vulnerabilities Reported
2001 2,437 2002 4,129
4
Cyber Strategy

• Cyber-war is not just simple hacking
• Sociology of warriors vs. hackers
• Morale
• Organization
• Vigilance vs. assumed invulnerability
• Motivation of warriors vs. hackers
• Accountability vs. anarchy
• Delayed vs. immediate gratification
• Internal vs. external gratification
• Preparation of warriors vs. hackers
• Training
• Intelligence / strategy

5
Incident Trends
6
Intruder Technology

• Intruders use currently available technology to
  develop new technology

7
Information Collection, Analysis and Sharing for
Situational Awareness
8
Overview

• Challenge statement
• Too much data too little information not
  shared
• Operational Need
• CERT Vision/Goals
• Our Approach
• Project Maturity
• Wrap up

9
Data Challenge

• System Network Administrators overwhelmed
• Data overload
• Important data often not collected
• Local/parochial focus
• Poor Network Situational Awareness
• Network Security Information is not shared
• Unconnected Islands of Information
• Ineffective, non-standard security tools and
  processes
• Non-technical reasons (organizational and
  liability)
• Unwilling to yield autonomy to gain better
  information
• Attackers share information more efficiently

10
Our Vision

• An operationally flexible system providing
• Clear avenues for exchanging relevant data
• Improved local monitoring
• Improved cueing methods
• Cross organization analytical capabilities
• Improved indications and warning
• Cross organization situational awareness

11
Our Goal

• Collect structured, sanitized, and representative
  situational awareness data in a standardized
  format to
• Recognize and respond faster (prior to damage)
• Permit collection of focused information on
  activity and trends
• Alert operators for proactive response
• Provide tools for sites to manage incident
  information

12
Bi-directional Solution

• Top-down
• Collection, organization, and analysis of data
  from wide, shallow sensors
• Bottom-up
• Federation of data from narrow, deep sensors
• Alerts from IDSs and Firewalls
• Raw data from sniffers recorders

13
Top-Down Approach

• Similar to the DEW line early indication that
  an attack may be coming facilitated by sensing
  the entire network
• Analysis for IW
• Hacking involves reverse engineering the
  attacker must probe, examine and determine the
  right approach
• Frequently precursors to attacks are buried in
  the noise
• Improve our ability to detect attacker behavior
  in the pre-attack stages
• Preventive Analysis
• Detect configuration errors
• DEW - Distant Early Warning

14
Top-Down
Internet
OC3
Edge Router
Netflow Collector
100Mb
T1
Firewall/Router
Real time collection analysis and alert tools
Intranet
15
Top-Down

• Collect coarse data
• No payload data
• Headers Only Source, Destination IP and ports
  protocol times traffic volumes (e.g. packets
  and bytes)
• Both inbound and outbound
• Collect wide data
• gt95 network coverage
• Multiple networks
• Collect a lot of data
• Requires a data center with large computational
  and storage capacity to facilitate historical
  analysis
• Scalable collection and analysis
• Outbound data indicates planted code or insiders

16
Top-Down - Wide Shallow Sensors

• Netflow
• Originally defined by CISCO but increasingly
  becoming standard
• See what the router sees
• Records of flows created at the router
• Assist in routing and in reporting network
  traffic statistics
• Consists of flow records aggregated from packets
• Sent to a collector and aggregated into different
  information records for varied analysis.

17
Inbound Slammer Traffic
18
Slammer Precursor Detection
UDP Port 1434 - Precursor
160000
140000
120000
100000
Flows
80000
Series1
60000
40000
20000
0
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
0
1
2
3
4
Hour 1/2400 1/2504
19
Slammer Precursor Analysis

• Focused on hours 6, 7, 8, 13, 14
• Identified 3 primary sources, all from a known
  adversary
• All 3 used a fixed pattern
• Identified responders 2 out of 4 subsequently
  compromised.

20
Detecting Scans

• Detect scans against client network hosts
• Higher intensity scans
• Low and slow scans
• Coordinated (distributed) scanning

21
Low-Packet Filtering
22
Stealth Tool Detection

• We are studying extremely slow (1 packet a day
  scanner) traffic on the Internet.
• As an initial trial, we identified sources
  sending between 1 and 3 packets of TCP (non-Web)
  traffic per day into the clients networks. We
  applied this to the period September 1-11,
  finding that 0.00001 of the traffic matched this
  pattern.
• Further analysis yielded a fingerprint for one
  tool. The tools profile appears to match Compaq
  Insight Manager XE on the client network.

23
Bottom-Up Approach

• Using data from Commercial Off the Shelf (COTS)
  security solutions already deployed
• e.g., Intrusion Detection Systems, firewalls,
  system logs, Snort, RealSecure, PIX, IPTables,
  syslog
• Custom-developed technology (AirCERT), currently
  not present in commercial products, to integrate,
  convert, analyze, and share the data
• Combination enables analysis of security event
  data from across administrative domains
• Different entities
• Different scales
• Subsidiary
• Corporation
• Sector

24
Bottom-Up
To other subnets.
Firewall/Router
Intranet
Web Server
Mail Server
IDS System
Sensor (Packet Capture)
25
Bottom-Up

• Collect data from by security devices (firewalls
  and intrusion detection devices)
• All or part of a packet
• Testimonials (e.g., IDS alerts), and associated
  contextual data
• Collect widely varied data
• Maximize network diversity (e.g., edge vs.
  transit many administrative domains)
• Maximize sensor diversity (e.g., IDS, firewall)
• Configurable volume of data
• Determined by local site and collaborators
• Scalable collection and analysis

26
Bottom-Up

• Implementation
• Flexible, open-source, standards-based reference
  implementation of an Internet-scalable threat
  assessment system
• Capability consists of components for
• Data Collection
• Data Sharing

27
Implementation
Internet
OC3
Edge Router
100Mb
Netflow Collector
T1
Firewall/Router
Intranet
Web Server
Mail Server
IDS System
Sensor (Packet Capture)
28
What Do You Do With This Data?

• Predictive numerical and statistical analysis
• Calculate long-term trends
• Profile traffic map servers, create baselines
• Continual monitoring for attack precursors
• Traffic Analysis
• Routing Anomalies and flaws
• Packet/Byte Characteristics
• Weak general results can drive strong focused
  analysis
• Analysis from Top-Down can drive Bottom-Up, and
  vice-versa

29
What Else Do You Do With This Data?

• Manage and analyze event data at all points in
  reporting hierarchy to detect and identify
• Compromise with cross-site data
• Coordinated, distributed attacks
• Slow and stealthy scans
• Network attack fronts
• Multi-site trends
• Distinguish between local and global activity
• Targeted scans
• Vulnerability probes

30
Integrating Top-Down Bottom-Up Analysis

• Augment data collection and configuration at the
  leaves
• Supplement or verify existing local security
  analyses and processes
• Employing cues gained from analysis at the
  root, focus analysis on data previously deemed
  benign or ignored
• Verify suggestive top-down and cross-site
  analysis by the selective analysis of data
  collected at the leaves

31
ACID Architecture
ACID can only analyze what is in the
Alert Database

ACID
32
Views of Data (grouping)

• ACID has no implicit analysis functionality --
  only presents the data by
• Event (Signature)
• Classification
• IP Address
• Port
• Flow
• Time
• Sensor
• Charts grouped by time, IP, classification and
  ports
• User defined queries

33
Event (Signature) viewUnique Alert

• Identifies the different type of attacks
• from Main, click on number next to Unique Alert

Number of Src/Dst IP
Number of Sensors
First/Last Occurrence
Signature
Reference
Classification
Total Number of Occurrences
34
Classification view

• Identifies the different event classifications
• From Main, click on the number next to
  categories

First/Last Occurrence
Number of Src/Dst IP
Number of Events
Number of Sensors
Classification
Total Number of Occurrences
35
Address view

• Identifies mostly frequently attacked machines
• Identifies network blocks of frequent attackers
• From Main, click on number after IP

Number of times seen in opposite direction
Number of Unique Events
Fully Qualified Domain Name
Number of Sensors
IP Address
Total Number of all Events
36
Port view

• Identifies most commonly targeted services
• From Main, click on number after Port

First/Last Occurrence
Port
Number of Sensors
Number of Unique Events
Number of Src/Dst IP
Total Number of all Events
37
Flow view

• Identifies suspicious events by flow activity
• From Main, click on number after Unique IP
  LInks

Protocol
FQDN and IP of Destination
FQDN and IP of Source
Total Number of all Events
Number of Unique Events
Unique Destination Ports
38
Sensor view

• Aggregate statistics on sensor
• From Main, click on number next to of Sensors

First/Last Occurrence
Number of Unique Events
Sensor Name
Total Number of all Events
Number of Src/Dst IP
Sensor ID
39
Temporal view Alert Listing

• Identifies event chronology
• Returned by any Searches or Alert Listing
  Snapshots

Layer-4 IP encapsulated protocol
Src/Dst IP and Port
Timestamp
Event (Signature)
Query Seq. Number, Sensor ID, Event ID
40
Temporal view (2)Graph Alert Detection Time

• Graphs number of alerts aggregating on hour, day,
  or month
• Visually represents peak attack periods
• From Main, click on Graph Alert Detection Time

Number of Events occurring in the time interval
Time Interval
41
Drill-Down Individual Alert

• Click on the ID in any Alert Listing

42
Drill-Down IP Address

• Provides statistics on an individual IP address
• Links to external registries and tools to gather
  information about the address
• Click on the IP address in any Alert Listing

43
User Interface Main
44
User Interface Navigation
Currently Selected Criteria
ACID Browser Backbutton
Browsing Buttons
Alert Actions
Checkbox to select alert
45
Analysis Example Most Frequently Targeted TCP
Services
46
Project Maturity

• Top-Down
• Highly efficient data partitioning and packing
  format
• Does not rely on a relational database
• Packs 90Gb per day into less than 30Gb
• Generic analysis tools written to perform ad-hoc
  analysis
• Processes a days worth of data in under 10
  minutes
• Rapid analytical tool development API
• Operational deployment at sponsor site
• Bottom-Up
• Prototype collection infrastructure developed and
  tested
• Active involvement in IETF security standards
  activity
• Pilot testing in progress

47
Project Maturity Continuing Efforts

• Involve more pilot sites
• Improve analytical capabilities
• Improve automated configuration
• Continue standards development efforts
• Increase collection diversity by supporting
  additional COTS
• Persuade vendors to adopt standards
• Planned Extensions to Netflow Analysis
• Enhanced with additional data based on payload
  but packed into the existing form-factor
• Aggregation into session records
• Matching aggregated session records into
  transaction records

48
Summary

• Transformational approach to data collection,
  sharing, analysis and response for Computer
  Network Defense
• Provides timely, focused information to operators
  providing cues for immediate action
• Provides tools for local, tailored analysis
• Provides local, enterprise and Internet
  Situational Awareness information
• Levels the playing field

49
Modeling and Simulation

• How do we drink from this fire hose?
• Goal is to use the volume of information to gain
  a predictive power over our adversaries

50
Emergent Algorithms
New Ideas Survivability is an emergent
property of a system Emergent algorithms are
distributed computations that fulfill mission
requirements in the absence of central control
and global visibility Local actions
Near-neighbor interactions gt Complex global
properties
Recognize Resist
Attack
Recover
Adapt
Current Research Design an emergent algorithm
simulation environment and language (Easel)
to Simulate and visualize the effects of
specific cyber-attacks, accidents and failures
Create a test-bed for mission-critical systems
Impact A new methodology for the design of
highly survivable systems and architectures
Ability to produce desired global effects through
cooperative local actions distributed throughout
a system (self-stabilizing)
51
The nature of complex, unbounded systems

• Easel is a new computer language designed to
  simulate complex, unbounded systems. Such systems
  exhibit the following properties
• Large numbers of autonomous components
• Incomplete and imprecise information
• Limited local knowledge
• No central control
• Bounded number of neighbors
• Competing objectives
• Such systems are more survivable because of
• adaptability
• graceful degradation
• no critical points of failure
• awareness of the local environment

52
Six explorations in survivability

• cascade failure in organizations
• failure propagation through an organizational
  network
• network topology generation
• survivability is a function of topology
• simple network message routing
• illustration of a very simple routing algorithm
• network attackers and defenders
• attackers compromise and defenders patch
• epidemic dynamics
• local contact leads to global infection
• seismic collapse of a building
• elastic response of linked beams to seismic
  shaking

53
Where can Easel help?

• Provide independent verification that
• complex system designs have no serious
• survivability flaws
• Analyze scenarios with respect to impact of
• design assumptions
• human error
• incomplete or imprecise information
• common mode failures
• single point of failure leading to cascading
  failure
• organized malicious attacks

54
Dealing with the Threat - Fusion Analysis Efforts

• Data Collection
• AirCERT
• Open source correlation
• Individual Event Analysis
• Statistical Analysis
• Modeling and Simulation

55
Whats Next?

• Our coordination of information must be
  commensurate with the enemys ability to use this
  information against us
• We must create a new world of checks and balances
  to match the appropriate use of information in
  the pursuit of malfeasants
• The key to this revolution is local
  administration of information while maintaining
  global coordination

56
Changes in Intrusion Profile

• 1988
• exploiting passwords
• exploiting known vulnerabilities

• Today
• exploiting passwords
• exploiting known vulnerabilities
• exploiting protocol flaws
• examining source and binary files for new
  security flaws
• abusing anonymous FTP, web servers, email
• installing sniffer programs
• IP source address spoofing
• denial of service attacks
• widespread, automated scanning of the Internet
• deep vuls in SNMP, SSL, WEP,

The definition of vulnerability on the Internet
is approaching that of the DoD in trusted systems
57
Scanning for Victims

• Today
• Wide scale scanners collect information on
  100,000s of hosts around the Internet
• Sniffers now use the same technology as
  intrusion detection tools
• Number and complexity of trust relationships in
  real systems make victim selection easier

58
Scanning for Victims

• Tomorrow
• Use of data reduction tools and more
  query-oriented search capability will allow reuse
  of scan data
• Inexpensive disk and computation time will
  encourage the use of cryptography and persistent
  storage of scan data
• Scan data becomes a commodity like marketing
  information

59
The Future of Probes

• Were very likely to see more
• widespread brute-force scanning with little
  regard for being detected
• stealthy probes like SYN and FIN that require
  packet logging to detect
• attempts to hide the origin of the probes through
  spoofing and decoys
• automated vulnerability exploits that probe and
  compromise in a single step

60
Typical Intruder Attack
Intruder scans remote sites to identify targets,
then attacks vulnerable or misconfigured hosts
Yesterday
61
Distributed Coordinated Attack
Intruder scans remote sites to identify targets,
then attacks vulnerable or misconfigured hosts
Today
62
Distributed Coordinated Attack

• Uses 100s to 1000s of clients (10,000s)
• Is triggered by a victim and time command
• Command channels include IRC, SNMP, ICMP
• May include dynamic upgrade and be spread by
  worms
• Will simultaneously attack the victim from all
  clients
• Today used in DoS attacks only

63
Issues for Responding to DoS Attacks

• Filtering/detecting this attack is problematic!
• The intruders intent is not always clear in
  denial of service attacks. The intruder might be
• using the DoS attack to hide a real attack
• misusing resources to attack someone else
• attempting to frame someone else for the attack
• disabling a trusted host as part of an intrusion
• Attacks also frequently involve
• IRC abuse
• intruders attacking each other
• retaliation for securing systems

64
The Future is Automation

• Put these together and what do you get?
• tools to scan for multiple vulnerabilities
• architecture identification tools
• widely available exploits
• pre-packaged Trojan horse backdoor programs
• delivery and recon through active content
• Bad news!
• Together, these publicly available tools could be
  modified to launch wide-spread scans and
  compromise systems automatically.

65
Warning Signs of Today

• We
• Tolerate unexpected program behavior
• Place little value on software quality
• Assemble parts with no clear idea what each part
  does nor who created it
• Spread highly capable and functional components
  through the hands of the unenlightened

66
Tom Longstaffs Predictions for the Next Decade
(well, at least the next 3 years)

• Network crime on the rise
• Many countries and NGOs preparing information
  warfare weapons
• Insiders and planted vulnerabilities control the
  battlespace
• Information warfare will be combined with
  traditional tactics (e.g., Iraq)