COSC4607 - Computer Security

1
COSC4607 - Computer Security
Dr. Haibin Zhu Assistant Professor Department of
CS and Math, Nipissing University Room A124A,
Ext. 4434 Email haibinz_at_nipissingu.ca URL
http//www.nipissingu.ca/faculty/haibinz Office
Hour Mon. Thurs. 230pm-430pm or by
appointment
2
Learning Outcomes

• On completion of this course you will be able to
• Understand a number of basic design principles in
  computer security.
• Demonstrate an understanding of the importance of
  security models with reference to the security of
  computer systems.
• Describe the features and security mechanisms
  which are generally used to implement security
  policies.
• Provide examples of the implementation of such
  features and mechanisms within a number of
  particular operating systems.
• Display a breadth of knowledge of the security
  vulnerabilities affecting computer systems.
• Demonstrate an understanding of the main issues
  relating to Web security in the context of
  computer systems.

3
Schedule

1. Introduction to Computer Security
2. Access Control
3. Security Models
4. Security Mechanisms
5. Linux / Unix Security
6. Windows 2000 Security
7. Computer Virus
8. Software Security
9. Security of Distributed Systems
10. Network Security
11. Cryptography

4
Resources
Notes Lecturers Books The Internet and
YOU and YOUR FELLOW STUDENTS
5
Books

• Textbooks
• D. Gollmann, Computer Security, John Wiley
  Sons, 1999.
• References
• C.P. Pfleeger, Security in Computing,
  Prentice-Hall, 1997 (2nd Ed).
• R. Anderson, Security Engineering, Wiley, 2001.
• B. Schneier, Secrets and Lies, Wiley, 2000.
• John Viega Gary McGraw Building Secure
  Software, Addison Wesley, 2001

6
Course Information
7
Questions ?
8
Review Operating Systems (1)

• It is useful to have some basic understanding of
  what an Operating System is and what it does.
• An Operating System is an application that sits
  between users and applications and the computer
  hardware. Essentially, it provides a moderately
  friendly interface to the hardware.
• Because computers think binary then they are
  extremely user-unfriendly! The Operating System
  removes some of the pain!

9
Review Operating Systems (2)
10
Review Operating Systems (3)
A more common version of the previous diagram is
given below
11
Review Operating Systems (4)

• The System Services layer is the real interface
  to the Operating System. It consists of many
  functions designed to translate an application
  request into something the Operating System can
  handle.
• POSIX is becoming the standard for System
  Services, mainly driven by the need to make it
  easier to port an application from one Operating
  System to another.
• So, what does an Operating System actually do?
• The primary functions of an Operating System are
  given on the next slide.
• Note that an Operating System must perform all
  its tasks efficiently and economically. Resource
  use by the system, in terms of CPU time, memory
  and disk usage must all be acceptable for the
  users.

12
Review Operating Systems (5)

• Starting and stopping programs and sharing the
  CPU between them
• Managing memory
• Memory allocation keeping track of memory usage
• Input and Output
• Device drivers concurrent handling of I/O
  devices
• File management
• Protection (preventing different programs from
  interfering with each other firewalling)
• Networking (seamless communication with other
  devices)
• Error handling (detection, recovery, warning)

13
Review Operating Systems (6)
14
Security Problems

• Security and reliability
• Buffer overflows
• Arrays and integers
• Canonical representations
• Race conditions
• Precautions defences
• Dangers of abstraction

15
Security Reliability

• On a PC, you are in control of the software
  components sending inputs to each other
• On the Internet, hostile parties can provide
  input
• To make software more reliable, it is tested
  against typical usage patterns it does not
  matter how many bugs there are, it matters how
  often they are triggered
• To make software more secure, it has to be tested
  against untypical usage patterns (but there are
  typical attack patterns)

16
Secure Software

• Software is secure if it can handle intentionally
  malformed input the attacker picks (the
  probability distribution of) the inputs
• Secure software protect the integrity of the
  runtime system
• Secure software ? software with security features
• Networking software is a popular target
• intended to receive external input
• involves low level manipulations of buffers

17
Preliminaries

• In code written in a typical programming language
  values are stored in variables, arrays, etc.
• To execute a program, memory sections (buffers)
  have to be allocated to variables, etc.
• In programming languages like C or C the
  programmer allocates and de-allocates memory
• Type-safe languages like Java guarantee that
  memory management is error-free

18
Buffer overflows

• If the value assigned to a variable exceeds the
  size of the buffer allocated to this variable,
  memory locations not allocated to this variable
  are overwritten
• If the memory location overwritten had been
  allocated to some other variable, the value of
  that other variable can be changed
• Depending on circumstances, an attacker could
  change the value of a protected variable A by
  assigning a deliberately malformed value to some
  other variable B

19
Buffer overflows

• Unintentional buffer overflows can make software
  crash
• Intentional buffer overflows are a concern if an
  attacker can modify security relevant data
• Attractive targets are return address (specifies
  the next piece of code to be executed) and
  security settings
• In safe languages such errors cannot occur

20
Stack heap

• Stack contains return address, local variables
  and function arguments it is quite predictable
  where a particular buffer will be placed on the
  stack
• Heap dynamically allocated memory it is more
  difficult but by no means impossible to predict
  where a particular buffer will be placed on the
  heap

21
Stack overflows

• Find a buffer on the runtime stack of a
  privileged program that can overflow the return
  address
• Overwrite the return address with the start
  address of the code you want to execute
• Your code is now privileged too

22
Precautions

• Be sparing with privileges if the code attacked
  runs with few privileges, the damage is limited
• If feasible, use a type-safe language
• In C and C avoid dangerous instructions like
  gets() and use instructions like fgets() where
  the length of the argument has to be specified
  extensive lists of good and bad instructions
  exist
• Non-executable stack stack cannot be used to
  store attack code

23
Defences

• Canaries (random) check values placed before the
  return address (StackGuard)
• Before returning, check that the canary still has
  the correct value

return address
my_address
write to A value1 value2 my_address to A
check value
value2 ? check value
canary
value1
buffer for variable A
attack detected
24
Heap overflows

• More difficult to determine how to overwrite a
  specific buffer
• More difficult to determine which other buffers
  will be overwritten in the process if you are an
  attacker, you may not want to crash the system
  before you have taken over
• Attacks that do not succeed all the time are a
  threat
• Heap overflow attacks have started to occur

25
Integers

• Mathematics integers form an infinite set
• Programming languages signed 4-byte integers,
  unsigned 4-byte integers, long integers,
• Truncation (Unix example) input UID as signed
  integer, value ? 0?, truncate to unsigned short
  integer 0x10000 ? 0x0000 (root!)
• Different interpretation of signed and unsigned
  integers (signed) -1 0xFFFF 216-1 (unsigned)

26
Integers

• Addition 0xFF00 0x0100 0x0000 base
  offset lt base
• Of course, the runtime may raise an exception
  when an overflow occurs on addition
• Ashcraft Engler IEEE SP 2002 Many
  programmers appear to view integers as having
  arbitrary precision, rather than being
  fixed-sized quantities operated on with modulo
  arithmetic.

27
Arrays

• Buffer overflow the length of an array is not
  checked when elements are written to the buffer
• Wrap-around to lower addresses when arithmetic
  operations do not have the result expected by the
  programmer

28
Scripting

• In scripting languages, executables can be passed
  as arguments
• Escape characters indicate that an argument is an
  executable
• Unescaping (making input non-executable) can
  protect code from malicious input
• Filters for escape characters have to know the
  escape characters and the character set in use
  CA-2000-2

29
Canonical Representations

• File names, URLs, IP addresses, can be written
  in more than one way
• Directory traversal c\x\data
  c\y\z\..\..\x\data c\y\z\2e2e\2e2e\x\data
• Dotless IP a.b.c.d ? a?224 b?216 c?28 d
• Symbolic link file name pointing to another file
• Canonicalization computes the standard
  representation
• When access right depends on location, you better
  get the location right do not rely on the names
  received as input

30
Race conditions

• Multiple computations access shared data in a way
  that their results depend on the sequence of
  accesses
• Multiple processes accessing the same variable
• Multiple threads in multi-threaded processes (as
  in Java servlets)
• An attacker can try to change a value after it
  has been checked but before it is being used
• TOCTTOU (time-to-check-to-time-of-use) is a
  well-known security issue

31
Example CTSS (1960s)

• Once the message of the day was the password file
• Every user had a unique home directory when a
  user invoked the editor, a scratch file with
  fixed name SCRATCH was created in this directory
• Several users working as system manager
• system manager one starts to edit message of the
  day SCRATCH ? MESS
• system manager two starts to edit the password
  file SCRATCH ? PWD
• system manager two stores the edited file
• MESS ? PWD

32
Defences Code inspection

• Code inspection is tedious we need automation
• K. Ashcroft D. Engler Using Programmer-Written
  Compiler Extensions to Catch Security Holes, IEEE
  SP 2002
• Meta-compilation for C source code expert
  system incorporating rules for known issues
  untrustworthy sources ? sanitizing checks ? trust
  sinks raises alarm if untrustworthy input gets
  to sink without proper checks
• Code analysis to learn new design rules Where is
  the sink that belongs to the check we see?
• Applied to Linux and OpenBSD kernels

33
Defences Black-box testing

• Black-box testing when source code is not
  available
• You do not need the source code to observe how
  memory is used or to test whether inputs are
  properly checked
• Oulu University Secure Programming Group PROTOS
  project (Juha Röning, http//www.ee.oulu.fi/resear
  ch/ouspg/)
• Syntax testing of protocols based on formal
  interface specification, valid cases, anomalies
• Applied to SNMP implementations

34
Defences Type-safety

• Type safety guarantees absence of untrapped
  errors
• Cardelli Practitioners who invented type safety
  often meant just memory integrity, while
  theoreticians always meant execution integrity,
  and its the latter that seems more relevant now.
• A language does not have to be typed to be safe
  LISP
• Safety guaranteed by static checks and by runtime
  checks

35
Type-safety

• Marketing ploy We are type safe, therefore we
  are secure
• Type safety is difficult to prove completely
• Proofs are conducted in an abstract model,
  problems may hide in the actual implementation
  (e.g. SUN Security Bulletin 00218)
• PROTOS Also software in Java were shown to have
  buffer overflows in native code sections
• Type safety is a useful property to have for
  security but type safety does not imply security

36
Keeping up-to-date

• Sources of information CERT advisories, BugTraq
  at www.securityfocus.com, security bulletins from
  software vendors
• Hacking tools have attack scripts that
  automatically search for and exploit known type
  of vulnerabilities
• Analysis tools following the same ideas will
  cover most real attacks
• Patching vulnerable systems in not easy you have
  to get the patches to the users and avoid
  introducing new vulnerabilities through the
  patches

37
Intrusion patterns
W. Arbaugh, B. Fithen, J. McHugh Windows of
Vulnerability A Case Study Analysis, IEEE
Computer, 12/2000
38
The wider picture

• We could treat all these problems individually
  and look for specific solutions, often limited to
  a given programming language or runtime system
  (penetrate-and-patch at a meta-level)
• Overall, a general pattern familiar programming
  abstractions
• variable, array, integer, data program, address
  (resource locator), atomic transaction,
• are implemented in a way that can break the
  abstraction

39
Summary of Security Problems

• Many of the problems listed may look trivial
• There is no silver bullet
• Code-inspection better at catching known
  problems, may raise false alarms
• Black-box testing better at catching known
  problems
• Type safety guarantees from an abstract
  (partial) model need not carry over to the real
  system
• Experience in high-level programming languages
  may be a disadvantage when writing low level
  network routines

40
Introduction to Computer Security

• Security objectives
• Security strategies
• Distributed systems security computer
  communications security
• Aspects of computer security
• Fundamental design principles for
• computer security

41
Security objectives

• Confidentiality prevent unauthorized disclosure
  of information
• Integrity prevent unauthorized modification of
• Information
• Availability prevent unauthorized withholding of
  information or resources
• Other aspects accountability, authenticity

42
Confidentiality

• Historically, security and secrecy were closely
  related sometimes, security and confidentiality
  are used as synonyms
• Prevent unauthorized disclosure of information
  (prevent unauthorized reading)
• ?Privacy protection of personal data
• ?Secrecy protection of data belonging to an
  organization

43
Integrity

• ITSEC Definition The property that prevents
  unauthorized modification of information (prevent
  unauthorized writing)
• Orange Book (US Trusted Computer Systems
  Evaluation Criteria)
• Data Integrity - The state that exists when
  computerized data is the same as that in the
  source document and has not been exposed to
  accidental or malicious alteration or destruction
  (integrity synonymous for external consistency)
• In communications detection and correction of
  intentional and accidental modifications of
  transmitted data

44
Availability

• IS 7498-2 ( Basic Reference Model for Open
  Systems Interconnection (OSI) Part 2 Security
  Architecture) The property of being accessible
  and usable upon demand by an authorized entity
• Denial of Service (DoS) The prevention of
  authorized access of resources or the delaying of
  time-critical operations
• Distributed denial of service (DDoS) is receiving
  a lot of attention and systems are now designed
  to be more resilient against these attacks

45
Accountability

• Audit information must be selectively kept and
  protected so that actions affecting security can
  be traced to the responsible party.

46
Dependability

• The property of a computer system such that
  reliance can justifiably be placed on the service
  it delivers. Here, the service delivered by a
  system is its behavior as it is perceived by its
  users a user is another system which interacts
  with the former.

47
A Remark on Terminology

• Definition Computer security deals with the
  prevention and detection of unauthorized actions
  by users of a computer system.
• There is no single definition of security
• When reading a document, be careful not to
  confuse your own notion of security with that
  used in the document
• A lot of time is being spent - and wasted
  trying to define an unambiguous notation for
  security
• Resources
• http//www.radium.ncsc.mil/tpep/process/faq.html
• http//www.cesg.gov.uk/assurance/iacs/itsec/index.
  htm
• ftp//ftp.cse-cst.gc.ca/pub/criteria/CTCPEC

48
Security strategies

• Prevention take measures that prevent your
  assets from being damaged
• Detection take measures so that you can detect
  when, how, and by whom an asset has been damaged
• Reaction take measures so that you can recover
  your assets or to recover from a damage to your
  assets

49
Example 1 Private Property

• Prevention locks at doors, window bars, walls
  round the property
• Detection stolen items are missing, burglar
  alarms, closed circuit (cable) TV
• Reaction call the police, replace stolen items,
  make an insurance claim
• Footnote Parallels to the physical world can
  illustrate aspects of computer security but they
  can also be misleading

50
Example 2 E-Commerce

• Prevention encrypt your orders, rely on the
  merchant to perform checks on the caller, dont
  use the Internet (?)
• Detection an unauthorized transaction appears on
  your credit card statement
• Reaction complain, request new card number, etc.

51
Security Policies

• Organizational security policy Laws, rules, and
  practices that regulate how an organization
  manages and protects resources to achieve its
  security policy objectives.
• Automated security policy Restrictions and
  properties that specify how a computing system
  prevents violations of the organizational
  security policy.
• D. F. Sterne On the Buzzword Security Policy,
  1991 IEEE Symposium on Research in Security and
  Privacy, pages 219-230

52
Distributed systems security

• Distributed systems computer systems connected
  by a network Communications (network) security
  deals with security aspects of the communications
  links
• Computer security deals with security aspects
  related to the end systems today, this is the
  difficult part
• Application security relies on both to provide
  services securely to end users

53
Other Computer Security Aspects

• Access control (authorization) prevention and
  detection of unauthorized actions by users of a
  computer system
• How to design access control systems
• How to support application security policies
• Secure software software that can cope with
  malicious inputs (important paradigm shift from
  the PC world to the Internet)

54
Principles of Computer Security

• The Dimensions of Computer Security

55
1st - Where to focus security

• The focus may be on data operations users
  e.g. integrity requirements may refer to rules on
• Format and content of data items (internal
  consistency) account balance is an integer
• Operations that may be performed on a data item
  credit, debit, transfer,
• Users who are allowed to access a data item (
  authorized access) account holder and bank clerk
  have access to account

56
2nd -Where to place security controls?
57
3rd-Complexity or Assurance?

• The location of a security mechanism on the
  man-machine scale is often related to its
  complexity
• Generic mechanisms are simple, applications
  clamor for feature-rich security functions
• Do you prefer simplicity - and higher assurance -
  to a feature-rich security environment?

58
The Fundamental Dilemma ofComputer Security

• Security-unaware users have specific security
  requirements but no security expertise.
• Simple generic mechanisms may not match specific
  security requirements. To choose the right
  features from a rich menu, you have to be a
  security expert.
• Security unaware users are in a no-win situation

59
Security Evaluation

• To check whether a system delivers the security
  services promised, one has to state the function
  of the security system and the required degree of
  assurance (trust) in its security
• To gain high assurance, the security system has
  to be examined in close detail
• There is a trade-off between complexity and
  assurance. The higher an assurance level you aim
  for, the simpler your system ought to be.
• Feature-rich security and high assurance do not
  match easily

60
4th-Central or Decentralized Control?

• Within the domain of a security policy, the same
  rules should be enforced.
• With a single entity in charge of security, it is
  easy to achieve uniformity but this central
  entity may become a performance bottleneck.
  Distributed solutions may be more efficient but
  added care has to be taken to guarantee that
  different components enforce a consistent policy.
• Should the tasks of defining and enforcing
  security be given to a central entity or should
  they be left to individual components in a system?

61
5th-Blocking access to the layer below

• Every protection mechanism defines a security
  perimeter (boundary).
• The parts of the system that can disable the
  mechanism lie within the perimeter, the parts of
  the system that can malfunction without
  compromising the mechanism lie outside.
• Attackers try to bypass protection mechanisms
  corollary to the second design decision
• How do you stop an attacker from getting access
  to a layer below your protection mechanism?

62
The Layer Below - Examples

• Recovery tools like Norton Utilities recover data
  by reading memory directly and then restoring the
  file structure. Such a tool can be used to
  circumvent logical access control as it does not
  care for the logical memory structure
• Unix treats I/O devices and physical memory
  devices like files. If access permissions are
  defined badly, e.g. if read access is given to a
  disk containing read protected files, then an
  attacker can read the disk contents and
  reconstruct the files.

63
More examples

• Object reuse in a single processor system, when
  a new process becomes active, it gets access to
  memory positions used by the previous process.
  You have to avoid storage residues, i.e. data
  left behind in the memory area allocated to the
  new process.
• Backup whoever has access to a backup tape has
  access to all the data on it. Logical access
  control is of no help and backup tapes have to be
  locked away safely to protect the data.
• Core dumps same story again

64
Summary

• Security terminology is ambiguous with many
  overloaded terms
• Distributed systems security builds on
  communications security and on computer security
• In computer security, two major challenges are
  the design of access control systems that fit the
  requirements of the Internet and the design of
  secure software
• In security, understanding the problem is more
  difficult than finding the solution