Title: Use of Microbial Risk Assessment in Decision-Making
1Use of Microbial Risk Assessment in
Decision-Making
Slide show on www.risk-modelling.com/firstmicrob
ial.htm
- David Vose Consultancy
- 24400 Les Lèches
- Dordogne
- France
- www.risk-modelling.com
- Email
- David Vose's secretary David Vose
2Introduction
- Applying CODEX guidelines in reality
- Difficulties
- Other ways of thinking
- Experience with microbial modelling
- Some survey results
- The Dutch experience
- Some US experience
- Modelling challenges
- Comparison of some complete models
- Reviewing a model in context
3Codex Alimentarius CommissionFAO/WHO (1995)
- Microbial risk assessment is a scientifically-base
d process consisting of four steps - Hazard Identification The identification of known
or potential health affects associated with a
particular agent - Exposure Assessment The qualitative and/or
quantitative evaluation of the degree of intake
likely to occur - Hazard Characterization The quantitative and/or
qualitative evaluation of the nature of the
adverse effects associated with biological,
chemical and physical agents that may be present
in food For biological agents a dose-response
assessment should be performed if the data is
available - Risk Characterization Integration of Hazard
Identification, Hazard Characterization and
Exposure Assessment into an estimation of the
adverse effects likely to occur in a given
population, including attendant uncertainties.
4OIE experience
- OIE produced guidelines for animal import risk
assessments (for the management of disease
spread) - Now in its second edition
- Guidelines were offered as a way to help member
(including developing) countries understand how
to perform a r.a. - First Ed. guidelines were used too literally,
both by analysts and lawyers, and found to be
often impractical or irrelevant to the risk
question - Lesson keep guidelines non-specific, encourage
understanding rather than prescribing a formulaic
approach - Popular interpretation of CODEX guidelines suffer
similarly
5(1) Risk analysis uses observations about what
we know to make predictions about what we dont
know. Risk analysis is a fundamentally
science-based process that strives to reflect the
realities of Nature in order to provide useful
information for decisions about managing risks.
Risk analysis seeks to inform, not to dictate,
the complex and difficult choices among possible
measures to mitigate risks...
Society for Risk Analysis Principles for Risk
Analysis
6(2) Risk analysis seeks to integrate knowledge
about the fundamental physical, biological,
social, cultural, and economic processes that
determine human, environmental, and technological
responses to a diverse set of circumstances.
Because decisions about risks are usually needed
when knowledge is incomplete, risk analysts rely
on informed judgment and on models reflecting
plausible interpretations of the realities of
Nature. We do this with a commitment to assess
and disclose the basis of our judgments and the
uncertainties in our knowledge.
Society for Risk Analysis Principles for Risk
Analysis
7Current modelling
- Microbial QRA is a developing science
- Were making a lot of progress, but it is still
in infancy - Mostly producing farm-to-fork
- Models the whole system but very poorly
- Not designed to model any decision question well
- Often relies on poor data, surrogates, and
guesses - Almost never is a decision question posed
beforehand - Assessors have probably over-sold QRAs
usefulness - Managers have expected too much
8F2F Achilles Heels
- Very little data available, system being modelled
is hugely complex! - Uncertainty, variability, inter-individual
variablility - Take too long to complete, too easy to make
mistakes - F2F considers only pathogen on the food source
- E.g. not E.coli produced during life of animal,
appearing in water, vegetables, farmers exposure - Predictive microbiology still unreliable
- Broth data doesnt translate well to food
(usually overestimate, but some data Tamplin,
USDA shows lag period can be shorter, e.g.
E.coli in ground beef, Listeria in processed
hams) - Models often not based on physical/biological
ideas, so we dont learn - Attenuation may not be death, and ignores
reactivation of bacteria - D-R models inadequate
- Dont describe variability observed
- P(illdose, infected) P(illinfected)?
- Feeding trial data dont match epi data can
hugely underestimate the risk - Little cost-benefit analysis effort made
- Including actions affecting several risk issues
- Requires enormous resources impractical for
many countries
9Dutch observations on past QRAHavelaar, Jansen
(2002)
- The lessons learnt from risk analysis
experiences - Risk management has not always been an integral
part of risk analysis so far - Risk managers should be trained to understand
risk assessment, and risk assessors should be
trained to explain their work - Available data are often of limited use for risk
assessment and communication of data needs
between risk assessors, food scientists and risk
managers is a critical issue - The risk manager questions usually require rapid
results, whereas (farm-to-fork) risk assessment
projects require several years to complete.
Solving this conflict requires open
communication - Uncertainty is often large.
10Our surveyInternet based, voluntary
participation, 39 valid responses
11(No Transcript)
12(No Transcript)
13(No Transcript)
14Completion times of some farm-to-fork QRAs
Final report
Final report
Draft report
Being revised
Draft report
Final report
15Salmonella dose-responseEpi and feeding trial
comparison
Review by Amir Fazil in FAO/WHO (2001) D-R
mathematical models review by Haas (2002)
16USDA-FSIS-FDA Salmonella Enteritidis
- Although the goal was to make the model
comprehensive, it has some important limitations.
It is a static model and does not incorporate
possible changes in SE over time as either host,
environment or agent factor change. For many
variables, data were limited or nonexistent. Some
obvious sources of contamination, such as food
handlers, restaurant environment, or other
possible sites of contamination on or in the egg
(such as the yolk), were not included. And, as
complex as the model is, it still represents a
simplistic view of the entire farm-to-table
continuum. Finally, the model does not yet
separate our uncertainty from the inherent
variability of the system. Much more work is
needed to address this, and all other,
limitations.
17USDA-FSIS-FDA Salmonella Enteritidis
- Original model impetus was to evaluate effect of
refrigeration temp from laying to retail on food
safety - Empirically must have little affect since it only
deals with a few days in the life of an egg - No cost-benefit attached
- Now being redone to focus on level of performance
required for shell, and liquid egg pasteurisation - i.e. much more decision focused
18FDA Listeria risk assessment
- No specific decision questions attached
- Attempted to look at relative importance of a
large list of Listeria-carrying foods - Given the data available, perhaps the only method
possible to estimate which food types contribute
the greatest risk - So a good QRA application
19Remedies focusing on decisions
- Consider what is known about the risk problem,
and data available immediately or within
acceptable time frame - Use epi data as much as possible
- Collect more epi data (e.g. Japan, Denmark)
- Consider what analysis could be done with this
knowledge - i.e. a risk-based reasoned argument for
evaluating particular actions - Estimate the possible magnitude of benefit for a
risk action - Note that it may not be possible to evaluate all
actions - Perform a cost-benefit analysis on these actions
- Perform a Value of Information analysis
- Determines whether it is worth collecting more
data before making a decision - Consider strategy to validate whether predicted
improvement occurs - Train data producers to supply maximally useful
data - E.g. microbiologists taken more than one cfu from
a plate - More inter-agency unity
- E.g. Farm (APHIS) Þ Slaughter (FSIS) Þ Retail
(FDA)
20Make it as simple as possible example
- Risk Human illness from SE in eggs
- A shell-egg selection system proposed that will
reduce by 30 the number of contaminated eggs
going to market - Currently, 30,000 people a year suffer from SE
from eggs - What will be the reduction in cases if the new
system is implemented?
Reduction in cases 3030,000 9,000
people/year
No need for models of D-R, bacterial growth,
handling, etc. Vulnerability to assumptions
smaller than from using F2F model
21An example of the way forwardHavelaar, Jansen
2002
- Campylobacter Risk Management and Assessment
- Dutch proposal
- The main objectives of the project are to advise
on the effectiveness and efficiency of measures
aimed at reducing campylobacteriosis in the Dutch
population. The two key questions are - What are the most important routes
(quantifiable?)? - Which (sets of) measures can be taken to reduce
the exposure to Campylobacter, what is their
expected efficiency and societal support?
22The way forward cont.
- The target of the assessment is not limited to
estimating the possible reduction in disease
incidence but to evaluate both costs and benefits
of possible interventions and to access their
acceptance by stakeholders. - Interventions with low social support will
require more effort to uphold, which increases
their costs and reduces their efficacy.
23Danish Vet Service Salmonella QRAA Bayesian
Approach to Quantify the Contribution of
Animal-food Sources to Human Salmonellosis -
Hald, Vose, Koupeev (2002)
Estimated number of cases of human salmonellosis
in Denmark in 1999 according to source Model
ranks food sources by risk. Easily updateable
with each years data. Bayesian update improves
estimate and checks validity of assumptions.
24Fluoroquinolone-resistant Campylobacter risk
assessment
Model Contaminated carcasses after slaughter
plant probability affected people
25Broiler
house
Transport
Slaughterhouse
model
Slaughter house
Hanging
Example of Farm-to-Fork model Campylobacter
in poultry Draft report 2001 Institute of Food
Safety and Toxicology Division of Microbiological
Safety Danish Veterinay and Food Administration
Scalding
Defeathering
Evisceration
Washing
Chilling
Export
Chicken parts
Whole
chickens
Chilled
Frozen
Further
Import
Processing
Retail
Catering
Cross
contamination
Consumer
model
Heat
treatment
Consumer
Cross
contamination
Behaves the same way as CVM model if prevalence
is reduced
Heat
treatment
Dose response
Risk estimation
26AHI model Fluoroquinolone resistant Campylobacter
in poultry AHI / Cox 2000 A dynamic simulation
model (i.e. follow the path of a random
chicken) Used same data as CVM where
available. Strong potential because it reduces
model complexity (at the expense of simulation
time)and easy to follow. Difficulty is
availability of data to make model parameters
meaningful. Most model parameters in current
version represent nothing physical (factors),
so dont enlighten us as to what actions to
take. D-R and consumer handling difficulties
remain.
27Reviewing a risk assessment
- Risk assessment should be decision focused
- It is not appropriate to review a risk assessment
independently from the question(s) the assessment
is addressing - Eg because a point is moot if the decision is
insensitive to the argument - It uses science but is not itself scientific
research - So we have to go with the best weve got
28Finally - risk assessors should gain hands-on
experience to ensure their models reflect the
real world
29References
- Haas, C.N., (2002), Conditional Dose-Response
Relationships for Micro-organisms Development
and Applications. Risk Analysis 22 (3) 455-464. - Havelaar, H. and J. Jansen, (2002), Practical
Experience in the Netherlands with quantitative
microbiological risk assessment and its use in
food safety policy. Draft paper, RIVM, Bilthoven,
The Netherlands. - Hope, B.K., et al. , (2002), An overview of the
Salmonella Enteritidis Risk Assessment for Shell
Eggs and Egg Products. Risk Analysis 22 (3)
455-464. - Joint FAO/WHO Expert Consultation on the
Application of Risk Analysis to Food Standards
Issues (Joint FAO/WHO, 1995). - Joint FAO/WHO Expert Consultation on Risk
Assessment of Microbiological Hazards in Foods
Risk characterization of Salmonella spp. in eggs
and broiler chickens and Listeria monocytogenes
in ready-to-eat foods. (2001), FAO headquarters,
Rome. - Teunis, P.F.M. and A.H.Havelaar, (2001), The
Beta-Poisson Dose-Response Model Is Not a
Single-Hit Model. Risk Analysis 20 (4) 513-520.