Risk assessment

1
Risk assessment

• Tor Stålhane
• NTNU / IDI

2
What is risk - 1

• Risks are characterized by three factors
• They are concerned with events that may or may
  not happen in the future.
• The events are identifiable but their effect and
  probability are uncertain.
• The outcome of the events can be influenced by
  our actions

3
What is risk - 2

• A risk is something that can be a problem in the
  future. It is defined by two parameters
• The probability - p. What is the probability that
  the risk will become a problem?
• The consequences - C. What will happen if the
  risk becomes a problem?
• The risk R is defined as R Cp

4
How large is the risk - 1

• In order to find the size of a risk, we need
  values for p and C.
• In some cases we can estimate these values from
  historical data but in most cases we will have to
  use expert opinions or other subjective data
  sources.
• It is not always possible or meaningful to
  assign a numerical value to a consequence, e.g.
  loss of lives.

5
How large is the risk - 2

• Even though assessment is a subjective activity
  it is not about throwing out any number that you
  like.
• To be useful, an assessment must be
• Based on relevant experience.
• Anchored in real world data, e.g. How bad can it
  get?
• The result of a documented and agreed-upon
  process. Having a process makes it possible to
  later improve the process based on experiences.

6
Assessing risk

• The quality of an assessment increases when the
  background info gets more specific.
• Dont ask What is the consequence of X? or
  What is the probability of Y?
• It is better to ask What is the consequence of
  X in scenario S? or What is the probability of
  Y in scenario S?

7
Assessment and scenarios - 1

• If the probability of scenario Si is p(Si), and
  pi and Ci are the probability and consequence of
  an accident in scenario Si, we have that

• The method is critically dependent on the
• Quality of the scenario descriptions
• Independence of the scenarios

8
Assessment and scenarios - 2

• We can improve our assessments even more if we do
  not ask for consequences in general but for
  consequences for one particular asset. Thus, in
  scenario i we have consequence Cj,i for asset j.

9
Assessing C and p

• We can assess consequences and probabilities in
  several ways
• Textual categories e.g. High, Medium, Low.
• Numerical categories e.g. values from 1 to 10.
• Value intervals.
• Statistical distributions.

10
Textual categories 1

• When using categories, it is important to give a
  short description as to what each category
  implies. E.g. it is not enough to say High
  consequences. We must relate it to something
  already known, e.g.
• Project size
• Company turn-over
• Company profit

11
Textual categories 2

• Two simple examples
• Consequences we will use the category High if
  the consequence will gravely endanger the
  profitability of the project.
• Probability we will use the category Low if
  the event can occur but only in extreme cases.

12
The CORAS consequence table
Consequence values Consequence values Consequence values Consequence values Consequence values Consequence values
Category Insignificant Minor Moderate Major Catastrophic
Measured related to income 0.0 0.1 0.1 1.0 1 5 5 10 10 100
Measured loss due to impact on business No impact on business. Minor delays Lost profits Reduce the resources of one or more departments Loss of a couple of customers Close down departments or business sectors Out of business
13
The CORAS frequency table
Frequency values Frequency values Frequency values Frequency values Frequency values Frequency values
Category Rare Unlikely Possible Likely Almost certain
Number of Unwanted incidents per Year 1/100 1/100 1/50 1/50 - 1 1 - 12 gt 12
Number of Unwanted incidents per Demand 1/1000 (1/500) 1/50 (1/25) 1/1
Interpretation of number of demands Unwanted incident never Occurs Each thousand time the system is used Each five times the system is used Each tenth time the system is used Every second time the system is used
14
Consequence and probability - 1
Consequence Consequence Consequence
Probability H M L
H H H M
M H M L
L M L L
15
Consequence and probability - 2

• The multiplication table is used to rank risks.
  It can not tell us how large they are.
• We should only use resources on risk that are
  above a certain, predefined level.

16
Numerical categories -1

• We can use numbers instead of names. This does
  not make the assessment more precise but will
  free us from the need to define a multiplication
  table in order to identify risks.
• In principle we can use any numbers. The best
  solution is, however, to just assign number to
  the three aforementioned categories

17
Numerical categories 2

• The following values are often used in practice,
  both for consequences, benefits and
  probabilities
• 10 high
• 4 medium
• 1 low
• Thus, a medium consequence and a low probability
  will give a risk of 41 4.

18
Numerical categories 3
Consequence Consequence Consequence
Probability H / 10 M / 3 L / 1
H / 10 H / 100 H / 30 M / 10
M / 3 H / 30 M / 9 L / 3
L / 1 M / 10 L / 3 L / 1
19
Value intervals

• If we have more info available we can give better
  estimates. Even though we cannot give exact
  values, we can give our assessments as intervals.
  
• An interval has a start and an end value
  denoted a and b. We denote the interval I as I
  a, b
• In our case, the width of the interval is a
  measure of our uncertainty.

20
Simple interval arithmetic

• As long as all interval limits are positive, we
  can write
• I I1 I2, I a1a2, b1b2
• I I1 I2, I a1 a2, b1 b2
• I I1 - I2, I a1 - a2, b1 - b2
• I I1 / I2, I a1b2, b1/a2
• If we use intervals for consequence (C) and
  probability (p) we get
• R C1p1,C2p2

21
Statistical distributions - 1

• We can use statistical distribution for C and p.
  In this case, the distributions are used to show
  our uncertainty.
• Practical solutions could be
• Beta distribution for p
• Gamma distribution for C

22
Statistical distributions - 2

• Based on the distributions of p and C, we can
  compute the distribution of the risk in three
  ways
• Mellin transforms
• Monte Carlo simulation
• Approximation methods
• We will only look at the third alternative.

23
Statistical distributions - 3

• The following approximation holds

24
Risk approximation

• Using the expressions from the previous slide we
  get the following approximations

It is now straight forward to find the expected
value and variance for R
25
Simple risk assessment

• In order to a simple risk assessment we need to
  identify
• Dangerous events
• Each events
• consequence C
• probability p
• Possible barriers changes or controls
• Person responsible for each risk - Resp.

26
Simple risk table
Event C p R Barriers Resp



27
Events

• We start by identifying dangerous events. The
  simple way to do this is to use brainstorming
  just sit down and envisage your worst nightmares
  related to the activities under consideration.
• Be realistic only consider things that you
  believe can happen.

28
Barriers

• Barriers can be realized through
• Prevention we change the system so that the
  event cannot occur.
• Mitigation we can
• change the system in order to reduce the events
  probability or consequences.
• define activities that will reduce the problems
  if the event occurs.

29
(No Transcript)
30
Benefits

• It is important to bear in mind that
• We usually expect to gain something through
  change new products, new ways to work etc.
• Risks stem from changes.
• Reducing risk is a cost factor
• We need to look at the total picture.

31
The total picture - 1

• The total picture of the situation shows the
  risks and the benefits that stem from a planned
  change.
• This is not a mechanism that can be used to
  identify the best solution.
• It is, however, an important input when we want
  to make a decision.

32
The total picture - 2

• The total picture shows risks and benefits. Risk
  can be shown in two ways
• Unmitigated risks
• Mitigated risks include the effect of risk
  reduction activities, e.g. barriers. This can be
  done by
• Modifying the risk assessment
• Indicate how the risk will move in the diagram

33
Consequences and benefits
B H Reduced number of MMI-related defects
B M
B L
p L M H
C L Extra work needed for MMI-specification
C M
C H
34
Unmitigated risks
B H Reduced number of MMI-related defects
B M
B L
p L M H
C L Extra work needed for MMI-specification
C M
C H Large disagreements between designers and MMI experts Partnership does not work
35
The mitigation effect
B H Reduced number of MMI-related defects
B M
B L
p L M H
C L Extra work needed for MMI-specification
C M
C H Large disagreements between designers and MMI experts Partnership does not work
36
Including benefits
B H Reduced number of MMI-related defects Better MMI for existing products Better MMI requirements will reduce imp. costs
B M
B L
p L M H
C L Extra work needed for MMI-specification
C M
C H Large disagreements between designers and MMI experts Partnership does not work
37
C and p as intervals - 1
38
C and p as intervals - 2
39
The tyranny of either or

• All too often we are confronted by the statement
  that we can get only get X if we are willing to
  suffer Y.
• This is the wrong attitude. The right attitude is
  that we will
• Do what is needed to get X
• Perform activities that will remove or reduce the
  bad effects of Y.

40
Leverage

• Leverage is a prioritizing mechanism
• Leverage (Benefit Cost) / Cost
• Leverage will prioritize activities with
• Large net benefits
• Small costs

41
Extended risk table -1

• We can use cause consequence chains or event
  trees for a risk to identify the best place to
  insert a barrier.
• For each barrier, we need to assess
• Cost - the cost of implementing it. We will use
  the scale H 10, M 3 and L 1.
• E how effective is the barrier? We will use the
  scale h 1.0, m 0.5 and l 0.2

42
Extended risk table - 2
Event C p R Barrier Cost E L Resp.



43
Barrier leverage

• Leverage (CpE Cost) / Cost
• The leverage will prioritize barriers which
• Have low costs Cost is small
• Have high efficiency E is large
• Attack important risks Cp is high

44
Barrier example
Event Cons. p R Mitigation E Cost L Resp
Partnership does not work business conflicts 10 3 30 Do a thorough research on selected partners business goals 0.5 10 0.5 John
Customers do not prioritize project participation 10 3 30 State the conditions and consequences of customer participation in the contract 1.0 3 9.0 Pete
45
Some comments on barriers

• It is important to remember that
• Each risk will usually need a different barrier
  a barrier that works against one risk can be
  valueless against another risk.
• It is important to consider the three main
  barrier strategies
• Prevent the risk from becoming a problem
• Control the problem to avoid the consequences
• Reduce the consequences

46
ALARP and GALE

• There are two competing principles in the
  assessment of risk
• ALARP As Low As Reasonably Possible- We have
  done all that is reasonable to prevent problems
  and dangers.
• GALE Globally At Least Equivalent. E.g.
  introducing a new process will not increase the
  risks compared to what it is today.

47
ALARP

• ALARP requires that we analyze each risk
  separately and then implement mitigation
  activities.
• A reasonable goal is to reduce each risk until
  the extra mitigation costs exceed the value of
  the risk reduction achieved.
• All that we have seen up till now fits into an
  ALARP policy .

48
GALE

• GALE requires us to look at the total risk of a
  change. In this way we can start by attacking the
  cheapest risk or the risk with the largest
  leverage.
• The problem with the GALE principle is that we
  need to perform arithmetic on risks. E.g. we need
  to decide how many medium risks we need before we
  have a large risk

49
ALARP vs. GALE

• The one important thing with using the GALE
  principle is that it forces us to ask What is
  the current risk level?
• All too often we act as it the current way of
  doing things is risk free and all risk stems from
  changes.
• This stance is enforced by the human tendency to
  underestimate the risk of status quo.

50
Using GALE

• Important points
• GALE is a method for risk analysis. Benefits must
  be included elsewhere
• We need to look at both our current risk and the
  risk resulting from the proposed changes.
• Always perform a sensitivity analyses.

51
Risk status quo vs. change

• In many cases, maybe even in most of them, we do
  risk assessment because we want to compare two or
  more alternatives, e.g.
• Status quo no changes
• One or more changes - improvements

52
Event identification

• All significant dangerous events must have been
  identified.
• There must be a minimal overlap between the
  dangerous events .
• There must be a maximum of commonality between
  the dangerous events considered for the status
  quo and for the system after the proposed changes

53
The three event sets

• The previous rules split the dangerous events
  into three sets dangerous events that
• Apply both to the status quo and to the new
  system.
• Are unique to the status quo
• Are unique to the new system

54
GALE and risk assessment - 1

• GALE uses the following parameters for risk
  assessment
• FE the event frequency
• PE the probability that the event will lead to
  an accident
• S the severity score of an event

55
GALE and risk assessment - 2

• We can compute individual and accumulated risk
  indices
• IE FE PE S
• IGR log Sumi(10I)
• IE is the risk index for a hazardous event
• IGR is the global risk index

56
The GALE scoring scheme

• The scoring scheme of GALE
• Focuses on deviations from current average. This
  is reasonable, given that it is mainly concerned
  with comparing status quo to a new situation.
• Must be tailored to each situation. The next
  slide shows an example from road safety. We need
  a scheme adapted to SPI.

57
Road safety - frequency score for event
Frequency classification Occurrences / year on M42 ATM section Occurrences / year on M42 ATM section FE
Very frequent 10000 Hourly 6
Frequent 1000 A few times a day 5
Probable 100 Every few days 4
Occasional 10 Monthly 3
Remote 1 Annually 2
Improbable 0.1 Every 10 years 1
Incredible 0.01 Every 100 years 0
58
Frequency score for event
Frequency classification Occurrences per project Occurrences per project FE
Very frequent 200 Every project 6
Frequent 100 Every few projects 5
Probable 40 Every 10th project 4
Occasional 10 Every 100th project 3
Remote 1 A few times in the companys lifetime 2
Improbable 0.2 One or two times during the companys lifetime 1
Incredible 0.01 Once in the companys lifetime 0
59
Probability score for event
Classification Interpretation PE
Probable It is probable that this event, if it occurs, will cause a problem 3
Occasional The event, if it occurs, will occasionally cause a problem 2
Remote There is a remote chance that this event, if it occurs, will cause a problem 1
Improbable It is improbable that this event, if it occurs, will cause a problem 0
60
Severity score for event
Severity classification Interpretation S
Severe The portion of occurring problems that have serious consequences is much larger than average 2
Average The portion of occurring problems that have serious consequences is similar to our average 1
Minor The portion of occurring problems that have serious consequences is much lower than average 0
61
Sensitivity analysis

• The global risk index is made of many indices.
  Each index will have a certain degree of
  uncertainty connected to it.
• Usually, a few indices will have a large
  influence on the result while the rest will have
  but little influence.
• Paretos rule applies - we need to identify the
  few important indices.

62
Important things to remember - 1

• The most important things to remember
• Risk assessment is by its nature subjective.
• Use group techniques and include all stakeholders
• Use simple techniques so that you do not exclude
  one or more stakeholders
• Anchor it in experience and available data will,
  however, improve the quality
• Subjective values like High must be anchored in
  each companys reality. One companys High may
  be another companys Low.

63
Important things to remember - 2

• Include the effect of choosing status quo in all
  risk analyses.
• Always include opportunities
• Consider the three barrier categories
  prevention, handling and reduction
• Rank risks and opportunities according to their
  leverage
• The results from a risk assessment is just one of
  several inputs to a decision