A Sense of Self for Unix Processes

1
A Sense of Self for Unix Processes

• Stephanie Forrest, Steven A. Hofmeyr, Anil
  Somayaji,
• IEEE SP 1996

2
Introduction

• Artificial Immune System
• Develop computer security methods that are based
  on the way natural immune systems distinguish
  self from other
• Self is treated synonymously with normal behavior
• Short sequences of system calls in running
  processes generate a stable signature for normal
  behavior, clear separation between different
  kinds of programs, high probability of being
  perturbed when abnormal activities occur

3
Related Work

• Anomaly detection in other methods
• Requires audit trail of actions for each user
• Slowly adaptive, Abrupt changes in behavior are
  flagged as irregular and identified as intrusions
• Alternative approach by Fink, Levitt and Ko
• Focus on determining normal behavior for
  privileged processes
• Define normal behavior using a program
  specification language, in which the allowed
  operations of a process (system calls and
  parameters) are formally specified
• Much simpler representation of normal behavior
• Rely on examples of normal runs rather than
  formal specification of a programs expected
  behavior

4
Defining Self

• Monitor only privileged processes
• System damage is caused by running programs that
  execute system calls
• More dangerous, limited range of behavior
• Difficult to detect an intruder masquerading as
  another user
• Local ordering of system calls remarkably
  consistent, simple definition of self, or normal
  behavior

5
Details - database construction
6
Details - mismatches
7
Details miss rate

• Record the number of missmatches as a percentage
  of the total possible number of mismatches
• L 8, k 3, maximum database size of 18,
  missmatches 4, 22 miss rate

Maximum number of pairwise mismatches for a
sequence of length L with a lookahead of k
8
Experiments

• Usefulness of Self definition
• What size database do we need to capture normal
  behavior?
• What percentage of possible system call sequences
  is covered by the database of normal system
  call sequences?
• Does our definition of normal behavior
  distinguish between different kinds of programs?
• Does our definition of normal detect anomalous
  behavior?
• sendmail, lpr, Sun SPARCstation, unpatched
  version of SunOS 4.1.1 and 4.1.4, strace package

9
Building a normal database

• Suite of 112 artificially constructed messages
• Produced a combined trace length of over 1.5
  million system calls
• For a window size of 6, produced a database with
  1500 entries

10
Procedures

• 1. Enumerate potential sources of variation for
  normal sendmail operation
• 2. Generate example mail messages that cause
  sendmail to exhibit these variations
• 3. Build a normal database from the sequences
  produced by step 2
• 4. Continue generating normal mail messages,
  recording all mismatches and adding them to the
  normal database as they occur

11
New patterns over time
12
Distinguishing between processes
13
Anomalous behavior

• Traces of successful sendmail intrusions
• Traces of sendmail intrusion attempts that failed
• Traces of error condition
• Sunsendmailcp, syslog attack script, decode alias
  attack, lprcp

14
Results
15
Discussion

• Normal database buildup
• Standard set of artificial messages
• Local usage pattern
• Two predicates by this approach
• The sequence of system calls executed by a
  program is locally consistent during normal
  operation
• Code of most program is static, system calls
  occur at fixed places within the code
• Conditionals and function calls will change the
  relative orderings of the invoked system call but
  not necessarily add variation to short-range
  correlations
• Some unusual short sequences of system calls will
  be executed when a security hole in a program is
  exploited
• Race condition, case of an intruder using another
  users account

16
Conclusions

• Definition is compact w.r.t the space of possible
  sequences
• Definition clearly distinguishes between
  different kinds of processes
• Definition is perturbed by several different
  classes of anomalous, or foreign, behavior,
  allowing these anomalies to be detected