FSISs Needs for Peer Review of Risk Assessments

1
FSISs Needs for Peer Review of Risk Assessments

• Carol Maczka, Ph.D.
• Senior Scientist for Risk Assessment
• FSIS/USDA
• September 30, 2003
• Presented at
• JIFSAN/SRA/RAC Symposium
• On Peer Review of Risk Assessment

2

• FSIS Peer Review Process to date
• Lessons Learned
• Needs and Challenges

3
Peer Review Mechanisms

• Salmonella Enteriditis Risk Assessment in Shell
  Eggs and Egg Products (1995)
• Inter- and Intra-Agency review by specialists
• Independent review by select subject area
  specialists
• Publications in peer review literature
• E. coli 0157H7 Risk Assessment in Ground Beef
• Inter- and Intra-Agency review by specialists
• NACMCF review
• Independent review by select subject area
  specialists
• Public comment
• External peer review by National Academies of
  Science

4
Peer Review Mechanisms (Continued)

• Risk of E. coli 0157H7 in Tenderized versus
  Non-Tenderized Steaks/Roasts
• Inter-Agency review
• NACMCF review
• Harvard BSE Risk Assessment
• Commission risk assessment (Independence)
• Inter- and Intra-Agency review
• External peer review using independent contractor
• Listeria Risk Assessment - Product Contamination
  from Food Contact Surfaces
• Inter- and Intra-Agency review
• ORACBA review
• Public comment
• External peer review using independent contractor
• ORACBA/OMB review

5
Peer Review Mechanisms (Continued)

• Other Risk Assessments (Currently in Progress)
• Perfringens Risk Assessment in RTE Meat and
  Poultry Products
• Risk Assessment for Salmonella Enteriditis in
  Shell Eggs and Salmonella spp in Egg Products
• Salmonella Risk Assessment in Raw Beef and
  Poultry Products
• Salmonella, E. coli 0157H7 Risk Assessment in
  RTE Meat and Poultry Products
• External Peer Review using Independent Contractor

6
FSIS Peer Review Process to Date
Independent External Peer Review
Intra-Agency Review ORACBA
Inter-Agency Review
RA
ORACBA
R
R
R
Public Comment
R
OMB
R Revised in response to comments
7
Elements of a Good Peer Review Process
Lessons Learned

• Independent External Peer Review especially for
  major regulations or influential information
• Access to broad range of scientific expertise
  (modelers, subject area experts)
• Timely and cost-effective review
• Appropriate staging of peer review and public
  comment to avoid unnecessary revisions
• Reviewers are given a focused charge and are
  invited to provide general comments.
• At a minimum, reviewers should be asked to
  comment on Overarching logical structure of the
  model Validity and appropriateness of the data
  used, reasonableness of assumptions made, models
  mathematics and equations, whether risks have
  been appropriately characterized key sources of
  variability and uncertainty identified critical
  assumptions data gaps. Computerized models
  should be audited.
• Reviewers are instructed to avoid policy
  determinations

8
Elements of a Good Peer Review Process
Lessons Learned (Continued)

• Reviewers are provided with relevant background
  information on potential sources of controversy
• Reviewers are held accountable
• Review is balanced
• Reviewers do not have a conflict of interest
• Procedures for documenting response to comments
  comment/response document
• Outside firm (entity) supervises the review.
  Comments are provided to Agency blinded
• SOPs for conducting peer reviews

9
Needs and Challenges

• Large pool of external peer reviewers (expert
  registry)
• Requirements for Peer Reviewers
• Scientific and Technical Expertise/Experience
• Multiple areas of expertise (e.g., modelers,
  epidemiologists, microbiologists)
• No real or perceived conflict of interests
• No advocated positions (balance)
• Have not conducted multiple peer reviews for the
  Agency in recent years
• How can an expert registry be created ?
• How and When are the pool of potential candidates
  screened for bias/conflict of interests

10
Needs and Challenges (Continued)

• Appropriate Mechanisms for Peer Review
• Contract with Independent Body
• National Academies of Science
• Outside Firm
• NACMCF
• Risk Assessment Consortium
• Consortium of Universities ?
• Pro Bono
• Others ?
• Need to ensure Timely/Cost Effective/Independent
  Peer Review Mechanism
• Standard Operating Procedures for Conducting Peer
  Review
• Need to ensure consistency with other agencies
  conducting significant regulatory review

11
Summary of Major Needs

• Pool of External Reviewers from which to select
• Mechanisms for accessing peer reviewers
• Establish SOPs for external peer review