Cortex Program Status Brief

1
Cortex Program Status Brief

• Christopher Geib
• December 14th 2005

2
Talk Outline

• Quick Program Overview.
• Technical approach.
• Accomplishments since last PI Meeting.
• Red Team Results.
• Metrics.
• Next Steps/Follow-on directions.

3
Self Regeneration that is Mission Sensitive

• Systems must be designed that are capable of
  response and regeneration after deliberate attack
  and catastrophic failures.
• However, response and regeneration must be
  sensitive to
• the mission that is being executed,
• What tools and services are critical for the
  current mission goals?
• What tools and services are not critical for the
  current mission goals?
• and the lessons learned from the previous
  failure.
• What features of the protocol were exploited in
  the attack?
• Have features of the domain changed?
• Are there new kinds of connections that should be
  blocked?
• Are some kinds of attacks more common than we
  thought?

4
Program Research Objectives

• Prove the viability of automatically synthesizing
  a meta-level controller for model-based, system
  response and regeneration.
• Such regeneration control systems must have two
  critical capabilities
• Response planning that is sensitive to the system
  mission,
• How much of the systems resources should be
  committed or held in reserve?
• Planning conditional responses to known threats.
• Trading off commitments for mission critical
  objectives.
• and learning to prevent similar failures in the
  future.
• Patch the services that were exploited
• Modify the mission model to capture changes to
  the world.

5
Architecture and Technical Approach
6
Program Level Demonstration Objectives

• Control self-regeneration of mission critical
  service.
• Scenario of use Air Tasking Order generation.
• Protect a MySQL DB, guaranteeing availability.

Domain Model
Planner
Current Lead Taster DB
Input Query Stream
Once per phase
Replicator
RTS
Proxy (Dexter)
.
Learner
7
Planning Approach Generalized Semi-Markov
Decision Processes

• Most existing decision-theoretic planning systems
  are based on the Markov Decision Processes and
  have difficulty handling multiple, asynchronous
  events.
• GSMDP provides the most natural framework for
  this purpose.
• CIRCA (Cooperative Intelligent Real-Time Control
  Architecture) is the first GSMDP planner.
• Uses the decision-theoretic principle of
  maximizing expected utility (e.g. to trade off
  performance against safety).
• Uses rich stochastic models of world,
  transitions, actions, and time to construct best
  defense plans.
• Does not hand-build, but automatically
  synthesizes plans and can thus adapt to defend
  against combinations/mutations of existing
  exploits based on the mission model.

8
Learning Approach

• Active experimentation based on identified axis
  of vulnerability.
• Query content, Query length (single/multiple
  terms)
• Resource consumption patterns
• Probing (e.g. password guessing)
• Session-wide (multiple queries)
• Active experimentation should avoid false
  positives.

Attack Query
Learner
Blocking rules and attack models
Build Model of Normal Traffic
Use model to identify suspicious elements
Historical(Normal)Queries
Model of Normal
Experiment
9
Risks and Mitigations

• Search space for the controller is large.
• Mitigation Our work is focused on heuristics
  that are solving these problems without covering
  the whole space.
• Mitigation Plans are built offline before system
  commission.
• Mitigation Possible to build plans that provide
  safe reduced functionality states to move to
  while regenerating.
• Identification of a covering set of axes of
  vulnerability and experimentation methods.
• Mitigation Fixed protocols have limited degrees
  of freedom making it easier to enumerate.
• Mitigation Axis only has to be identified once .
• Binary poisons
• Mitigation Dont eat the second half of the
  poison.
• Mitigation Use program diversity tools to push
  some code level violations that would be binary
  poisons into our space.

10
Recent Progress
11
Infrastructure Progress

• New two stage mission model defined.
• Removed Apache and rebuilt to define 2 phase
  mission.
• Building demonstration in progress.
• Extended set of attacks.
• 2 Old attacks.
• Lion password buffer overflow.
• My_Dumper casting vulnerability for signed
  integers.
• 2 New attacks identified and implemented.
• Hoagie MySQL_shutdown() doesnt check
  authority.
• Eleven 11 simultaneous connections to MySQL
  causes crash.
• Independence of MySQL version verified by using
  earlier version.
• Our development done with MySQL 3.23.49.
• Tested with earlier version MySQL 3.23.8.

12
Learning Progress

• Classes of exploits
• Generalization across axis of vulnerability
• Anecdotal evidence for this.
• System learned My_Dumper when exposed to Hoagie.
• Addition of learned exploits to attack ontology,
  and domain threat model.
• No identified false positives.
• Red Team Goals.

• Cant sense or block (eleven)
• Can sense, will block (Hoagie)
• Can sense, block, and generalize (Lion, MyDumper)
  

13
Red Team Results

• Sandia Red Team verified learning capabilities.
• Effectively blocked Red Teams versions of our
  attacks, even when presented in close proximity.
• Generalized attacks as well.
• Identified (and patched) some weaknesses in RTS
  and Proxy.
• RTS Designed for highly engineered real time
  systems that are not subject to variability from
  human inputs.
• System suffered significant false positive
  problem.
• In some cases blocking all significant access
  beyond connection to the MySQL server.
• Why? Murphys Law we provided the wrong code
  development branch to the red team for testing.

14
Retesting on Our Own.

• Using the Red Teams versions of the exploits we
  reran their tests. Got no false positives.
• (show movie)
• Testing multiple identical instances of attacks
• Inter-attack timing lt 1 second results in no
  connection.
• Inter-attack timing gt 1 second attack is blocked.
• Havent had time to look at some possibilities
• varying the attack ahead of Cortex.
• bias the model of normal traffic.
• Very interested in having the Red Team back out.

15
Load Testing with Background Traffic

• Working on testing against SQLBench, a standard
  benchmark data set.
• Command set Covers space of SQL commands.
• Would be nice to conclusively show no false
  positives.
• Current problem is stock proxy (DEXTER) doesnt
  handle it.
• Testing against our own traffic generators
• Command set Inserts, Selects.
• Does learn, and will block later instances.
• We do loose a few queries. Proxy doesnt buffer.
• Learning happening on the same machine (not
  required)
• Working on looking at the learning load

16
Effect on System Runtime
17
Program Level Metrics

• Correctly diagnosing root cause 10 of attacks.
• System correctly diagnoses 50 of collected
  attacks.
• System will diagnose 100 of the attacks covered
  by an identified axis of vulnerability.
• Question becomes one of coverage of the axes of
  vulnerability relative to the set of attacks.
• Correctly responding to 5 of diagnosed attacks
• CORTEX plans controllers that respond to all
  (100) of the possible attacks enumerated within
  the mission model.
• Learning blocking rules for 75 of the observed
  attacks.
• Learning generalized blocking rules for 50 of
  the observed attacks (those covered by identified
  axis).

18
Project Metrics

• Time to synthesize optimal controller (measured)
• Experiments have cut the time to find the optimal
  plan for this problem by gt90.
• Accuracy of the rules learned (measured)
• Limited scope testing against our traffic
  generators 0 false positives.
• Working on standard SQLBench set for
  confirmation.
• Overhead added to normal processing (measured)
• Tests against our traffic generators
• 3 x with no hostile traffic
• 5 x with hostile traffic and limited learning
  resources.
• Working on standard SQLBench set for confirmation

19
Expected Major Achievements

• Prove the viability of automatically synthesizing
  a meta-level controller for model-based system
  response and regeneration.
• Limited impact on users
• Over all improved system throughput from
  increased system mission sensitive availability.
• Demonstrate system scalability in two mission
  phases.
• Prove the viability of automatically learning
  previously unseen exploits based on identified
  axis of vulnerability.
• Demonstrate system planning for at least two
  different mission phases with significantly
  different requirements and responses.

20
Future Directions
21
Measuring Capacity for Self-Regeneration

• Compare the protected system to unprotected (or
  differently protected) systems
• For the same set of realistic tasks or relevant
  application benchmark application.
• Quantify trade-off between system throughput and
  increased regenerative capacity
• for a given hardware platform.
• Baseline performance before attacks
• Employ as many types of attacks as available...
• Compare to performance during and after attacks.

22
Metrics for Phase II
Unprotected
Maximum degradation
CORTEX
Overhead
Recovery latency
Baseline
Transactions Per Minute
Pre-Attack
Post-Attack
Attack
Illustrative curvesnot real data
23
Metrics for Phase II

• Baseline scale is application specific.
• In database application transactions/unit time.
• Overhead.
• Ratio of protected to unprotected performance.
• Recovery Latency.
• Time required to restore the system to a
  threshold fraction of baseline performance after
  an active attack has ceased (per attack tested).
• Maximum Degradation.
• The ratio of the worst performance to nominal
  performance (per attack tested).

24
Possible Dramatic Performance Increases

• Recoding of Proxy and possibly RTS.
• Proxy is currently single threaded Perl.
• Retool/Rewrite while retaining extensibility for
  detection and control.
• RTS not designed for vagaries of human input.
• Correct distribution of system components.
• Improved Learning.
• Improved Planning.

25
Future Work Learning

• Benefits
• Defense against wider suite of more subtle
  attacks.
• Enable faster system deployment.
• Lower cost system maintenance.

• Research Issues
• More Comprehensive Sensing.
• Must be able to detect subtle forms of attack.
• Attacks that are not immediately (or ever?)
  fatal.
• Automated learning of Axis of Vulnerability.
• Rely on having a set of sensors that covers the
  space.
• Identify the sensor(s) that are critical.
• Determining the experimentation methodology.
• Automated restructuring of the domain model.

26
Future Work Planning

• Benefits
• Automated response planning for larger domains.
• Speed for replanning for RTSy.
• More accurate response to intelligent adversaries

• Research Issues
• Automated Abstraction and Problem Space
  Decomposition
• Finding fully connected (or maybe weakly
  connected) subspaces that can allow the system to
  solve subproblems.
• Some existing work in the DTP.
• Replanning at runtime.
• Automated use of specialized planning search
  heuristics.
• Utility based
• Resource utilization.
• Sampling search
• Counter Planning for intelligent adversaries
• Conditionalizing probability of opposition
  actions based on our actions.

27
CORTEX Mission-Aware Closed-Loop Cyber
Assessment and Response
Attacks, intrusions
NEW IDEAS
Security Tradeoff Planner
Computing services

• System Reference Model drives intrusion
  assessment, diagnosis, and response.
• Automatically search for response policies that
  optimize tradeoff of security against mission
  ops.
• Taste-tester server redundancy supports
  robustness and learning from new attacks.

Networks, Computers
Controller Synthesis Module
Scyllarus Intrusion Assessment
Active Security Controller Executive
CIRCADIA
IMPACT
SCHEDULE

• High confidence intrusion assessment and
  diagnosis.
• Pre-planned automatic responses to contain and
  recover from faults and attacks.
• Automatic tradeoffs of security vs. service
  level accessibility.
• Learns to recognize and defeat novel attacks.

Demos
Mission Aware Learning and Response Demo
Planning Demo
Thin slice demo
Learning Demo
Jan 05
Jul 05
Dec 05
May 05
28
End