Security Attribute Evaluation Method: A Cost Benefit Analysis

1
Security Attribute Evaluation Method A Cost
Benefit Analysis

• Shawn A. Butler
• Computer Science Department
• Carnegie Mellon University
• 27 November 2001

2
Hey Boss, we need more security. I think we
should get the new Acme 2000 Hacker Abolisher
We always seem to need more security! Dont we
have enough?
3
Trust me, we will be more secure!
What are my alternatives?
What is it going to cost?
What is the added value?
4
Alternatives?
Value?
S
5
Problem

• Security managers lack structured cost-benefit
  methods to evaluate and compare alternative
  security solutions.

6
Security Architecture Development Process
Develop Security Architecture
Security Architecture
7
The Multi Attribute Risk Assessment

• Determine threats and outcomes
• Assess outcome attribute values
• Assess weights
• Compute threat indices
• Sensitivity Analysis

8
Determine Threats and Outcomes

• Threats
• Scanning
• Procedural Violation
• Browsing
• Distributed Denial of Service
• Password Nabbing
• Personal Abuse
• Signal Interception
• 29 Threats

• Outcome Attributes
• Lost Productivity
• Lost Revenue
• Regulatory Penalties
• Reputation
• Lives Lost
• Lawsuits

9
Scanning in More Detail
Outcomes Attacks Lost Producti-vity (hrs) Lost Revenue () Regulatory Penalties (scale 0-6) Reputation (scale 0-6)
Scanning 10,220/yr Low .3 0 0 1
Scanning 10,220/yr Expected .5 2 0 1
Scanning 10,220/yr High 1 1,000 0 4
.01 plow ? (?jattributesWj ? Vj(xj low))
.07 pexpected ? (?jattributesWj ? Vj(xj
expected))
.00 phigh ? (?jattributesWj ? Vj(xj high))
10,220 ? (.01 .07 .00) ? 886.57
10
Risk Assessment Results
Threat Frequency Low Expected High Total
Scanning 10,220 .0084 .0750 .0034 886.57
Procedural Violation 4380 .0000 .0773 .0065 367.03
Browsing 2920 .0000 .0742 .0035 226.71
Dist Denial of Service 156 .0085 .1530 .0060 26.12
Password Nabbing 365 .0001 .0008 .0009 .62
Personal Abuse 110 .0000 .0003 .0009 .13
TOTAL 1,507.18
11
Risks as a Percentage of Threat Index Total
12
But what about the numbers?
13
Sensitivity Analysis is Key!!

• How sensitive are the answers to estimation
  errors?
• Does it matter if the estimates are not accurate?
• How accurate do they have to be before the
  decision changes?
• When is it important to gather additional
  information?

14
Security Attribute Evaluation Method (SAEM)

• Evaluation Method
• Assess security technology benefits
• Evaluate security technology benefits
• Analyze Costs
• Assess coverage
• Sensitivity Analysis

Prioritized Risks
15
Assess Security Technology Benefits
Scanning 50 75 66 66 33 33 50
Procedural Violation 50 40 25
Browsing 30
Dist Denial of Service 75
Password Nabbing 50
Personal Abuse 40
16
Prioritized Technologies
Technology ?Value Threat Index Overall Rank
PKI/Cert .24 28
Auditing 241 11
Auth Policy Server 161 15
Host-IDS 589 2
Net-IDS 293 10
Smart Cards 103 16
One Time Psswrd 340 7
Single Sign-on 0 35
17
Analyze Costs
589
? Host IDS
? Net IDS
? Auditing
Threat Index ?
? Auth Policy Server
? Smart Cards
? Single Sign-on
? PKI Cert
0
20,000
0
Purchase Cost
18
Assess Coverage
19
Host Intrusion Detection Coverage
20
Auditing Coverage
21
Preliminary Results

• Risk Assessment threat indices reflect security
  managers concerns
• based on interviews and feedback
• Security managers are able to estimate technology
  benefits
• based on experience, organizational skill levels,
  and threat expectations
• Sensitivity Analysis is key to method
• based on uncertainty of assumptions