Risk Assessment, Risk Modelling and Decision Analysis

1
Risk Assessment, Risk Modelling and Decision
Analysis

• Canadian Water Network Retreat
• Victoria, B.C. June 25th 2008
• Aamir Fazil

2
Risk

• Risk is a function of both the probability and
  impact of an event
• Dependent on what and who is assessing the risk

3
Outline

• Risk Assessment and Risk Modelling
• What Risk Modelling contributes to the decision
  making process
• Multi-Criteria Decision Analysis
• Incorporation of other factor into the decision
  making process

4
Why Do Risk Assessment

• Estimate the risk
• Regulation
• Acceptability

• Gain an understanding of the system
• Appropriate mitigation
• Research direction

5
Understanding the system through risk modelling
System
System Modelling
Adapted from Law Kelton
6
Quantitative Risk Modeling

• Quantitative Risk Modeling assists in decision
  support by
• Focussing Attention
• Risk reducing areas
• Research areas
• Formulating Strategy
• Exploring Strategy

7
FARM
C. jejuni in chicken risk model
Flock Prevalence
Contamination Level
PROCESSING
Scalding
Defeathering
Evisceration
CONSUMPTION
Amount Consumed
Dose-Response
Washing
Chilling
STORAGE
RISK
HOME
8
Decision Support (1) Focusing Attention
Example C. jejuni in chicken Sensitivity Analysis
9
Decision Support (1) Focusing Attention
Salmonella 135,000 (60,000 - 230,000)
Campylobacter 400,000 (228,000 - 615,000)
10
Decision Support (1) Focusing Attention

• Cost per case
• lost days of work physician visits
  hospitalization death and chronic sequelae
• Salmonella 3650 / Case
• Campylobacter 2400 / Case

11
Decision Support (1) Focusing Attention
Current annual cost associated with infection
Salmonella Mean 430M
Campylobacter Mean 820M
12
Decision Support (1) Focusing Attention
Cost savings associated with pathogen reduction
Salmonella 18M - 170M Cost Savings
Campylobacter 82M - 409M Cost Savings
13
Decision Support

• In addition to identifying where to focus
  attention
• Facilitates the formulation of management
  strategies to determine best alternatives
• Tool to explore estimated effect of candidate
  strategies

14
Decision Support (2) Formulating Strategy
15
Decision Support (3) Exploring Strategy

• What impact does disinfectant use in processing
  have on pathogen risk
• Scenario 1 Use of chlorine in washer
• Scenario 2 Use of Acidified Sodium Chlorite
  spray
• Scenario 3 Use of TSP (chlorine alternative
  spray)
• Scenario 4 Use of chlorine in the chill tank
• Scenario 5 Combination of 1 and 4
• Scenario 6 Combination of 2 and 4
• Scenario 7 Combination of 3 and 4.

16
Risk Reduction Estimates
1) Chlorine in wash 10 2) ASC spray 70 3)
TSP spray 55 4) Chlorine in chill tank 69
5) Combination 1 and 4 75 6) Combination 1 and
4 95 7) Combination 1 and 4 91
17
Risk-Benefit

• Previous slide sets the stage for risk benefit
  comparison
• Quantifies benefits of pathogen reduction as a
  result of disinfectants
• Comparing benefits from pathogen reduction
  against risks from disinfectant by-products still
  a challenge
• Outcomes are different
• GI infections, sequellae, etc. vs. longer terms
  outcomes, cancers, etc.
• Need to use a common metric (e.g DALYs)

18
Multi-Criteria Decision Analysis

• Risks and Benefits are only part of the decision
  making process
• Typically many other factors also play a role
• Perceptions / societal impact / practicality etc
• How can we incorporate other factors ?
• Multi-Criteria Decision Analysis has application

19
Decision Making Problems

• All decision problems are actually multi-criteria
  problems
• If only one alternative, then no real choice
• If only one criteria, then no decision problem
• Select best in that criteria
• When there are only a few alternatives, criteria,
  and stakeholders problem can be solved in an
  ad-hoc manner
• Quickly becomes difficult to do this

20
Intervention selection

• The Decision Problem
• Several interventions available to reduce risk
  from a pathogen
• We want to select the best intervention
• All interventions have varying degrees of
  strengths and weaknesses in various criteria
• Best is a compromise based upon performance in
  various criteria

21
Defining the Criteria

• In the intervention selection problem criteria
  are
• Weight of evidence
• Effectiveness
• Cost
• Practicality
• Could include other criteria depending upon the
  decision
• Perception
• Societal Benefit

22
Criteria to Evaluate Interventions
MCDA Promethee Analysis
Weight of Evidence
Systematic / Critical Review
Effectiveness
Risk Modelling
Selection of Optimum Intervention
Cost
Expert Opinion
Practicality
23
Performance Matrix
Weight of Evidence
Effectiveness
24
Ranking
Intervention D gt Intervention C gt Intervention B
gt Intervention E and A

• Many algorithms available to conduct the analysis
• Utility based approaches, pairwise comparisons,
  outranking
• Biggest benefit is the possibility to explore the
  decision transparently (e.g. sensitivity
  analysis, stakeholder scenarios)

25
Sensitivity Analysis Impact of Effectiveness
(POI)
26
Sensitivity Analysis Impact of Effectiveness
(POI)
27
Sensitivity Analysis Impact of Effectiveness
(POI)
28
Sensitivity Analysis Impact of Effectiveness
(POI)
29
Sensitivity Analysis Impact of Effectiveness
(POI)
30
Sensitivity Analysis Impact of Effectiveness
(POI)
31
Sensitivity Analysis Impact of Effectiveness
(POI)
32
Scenario AnalysisDifferent Stakeholders

• Government
• Greater emphasis on effectiveness, less on cost
  and practicality

33
Scenario AnalysisDifferent Stakeholders

• Industry
• Greater emphasis on cost, practicality and effect
  at point of application, less on others

34
Scenario AnalysisDifferent Stakeholders
Int. C
Int. C
Int. D
Int. D
Int. A
Int. B
Int. B
Int. E
Int. A
Int. E
35
Summary Conclusion

• Focussing on more than two or three criteria is
  difficult without support
• Identifies conflicts and helps in the
  understanding of conflicts and tradeoffs
• A transparent decision tool
• The integration of stakeholders is supported
• Negotiations among stakeholders can be
  facilitated.
• Subjectivity is made explicit

36
Conclusions

• Risk Assessment / Modelling
• Contributes to the understanding of the system
• Helps identify critical factors that most
  significantly influence risk
• Risk mitigation / control implications
• Research direction implications
• Potential discrimination between information that
  is unknown and relatively unimportant vs. unknown
  and important.
• Provides guidance on expected impact of risk
  mitigation strategies

37
Thank you for your attention