Developing EPAs Peer Review Program

1
Developing EPAs Peer Review Program

• Joint JIFSAN/SRA/RAC Symposium
• Dorothy E. Patton, Ph.D., J.D.
• September 30, 2003

2
Overview Parallel Positions

• EPA in 1992
• External recommendations from SAB, GAO
• Mixed Policies and Practices
• All Federal Agencies in 2003
• External Recommendations from OMB
• Assumption Mixed Policies and Practices

3
Agenda

• Brief History Pre-1990 Policy and Practice
• 1992 Science Advisory Board Report
• Implementation Themes and Tasks
• Re-education
• Regularity
• Rigor
• Recommendations
• Addendum

4
EPAs Peer Review History

• Peer Review Practices 1970-1992
• Statutory Mandates
• Academic Model
• Evolution and expansion
• Initiatives for change 1992-2000
• 1992 Science Advisory Board Report
• 1993 Reilly Administration Policy
• GAO Report
• 1994 Browner Administration Policy

5
1992 Starting Point . . .

• Require credible, independent peer review of
  all scientific and technical efforts of ORD and
  the program and regional offices (including model
  development and use, data collection and
  evaluation, monitoring plans, research, technical
  studies, scoping studies, and assessments.)
• EPA Science Advisory Board

6
. . . Message

• General Concerns
• Lack of Review
• Non-conforming reviews
• Specific Concerns
• Credible and independent reviews
• All scientific and technical efforts
• ORD, program offices, regional offices

7
Implementation Program

• First Generation learning and testing
• Generic template
• Office-specific approaches
• Models and examples
• Second Generation Peer Review Handbook
• Multi-office teams
• First Edition initial statement of principles,
  methods
• Second Edition reflects Agency experience

8
Implementation Tasks

• Rigor as to criteria and standards
• Regularity as to internal process
• Re-education to assure agency-wide
  implementation

9
Rigor Criteria Standards

• Readiness of Product for Peer Review
• Consultation vs. peer review
• Charge to Peer Reviewers
• Specific issues and general invitation
• Form of Peer Review
• Matching form and product
• Identification of Peer Reviewers
• Measures of Success

10
Regularity Process Decisions

• Coverage
• products requiring peer review
• Offices conducting peer review
• Form
• Letter reviews
• Panel reviews
• Reviewers
• Peer Reviewer independence
• Peer Reviewer expertise

11
Regularity Resources

• Threshold Considerations
• Time allocation
• Budget allocation
• Staff allocation
• Staffing Considerations
• Value of a Champion
• Top down and bottom up
• Investment in training
• Investment in models

12
Re-education

• Goals
• Scientific and regulatory credibility
• Enhanced agency product
• Important Distinctions
• Peer review as intermediate vs. end product
• Scientific reliability vs. policy preferences
• Inquiry vs. endorsement

13
Recommendations TangibleConsiderations

• Early attention to resource issues
• Consensus on criteria and standards
• Focus on Using peer review comments
• Draw on EPA Handbook, as appropriate
• Offer models of sound peer review
• Establish reasonable implementation timelines

14
Recommendations Intangible Considerations

• Identify a champion
• Educate and engage managers
• Tailor to program-specific factors
• Staff participation and team building
• Recognize Change-the-culture issues
• Aim for scientific quality

15
Summary Three Rs

• Rigor See the Handbook
• Regularity See the Handbook
• Re-education iterative combination of
  top-down and bottom-up approaches

16
Additional Information

• EPAs Peer Review Handbook
• www.epa.gov
• EPA 100-B-00-001, December 2000
• Neutral Science Panels (Federal Judicial Center,
  2001)
• NAS blue book

17
Addendum Post-EPA notes

• Risk Assessment as a multi-disciplinary technical
  analysis
• Reliance on many different studies
• Weight-of-evidence conclusions
• Policy choices
• Regulatory Decision
• Risk assessment or other technical analysis
• Non-technical considerations

18
Addendum (cont.)

• OMB guidance
• scientific and technical Products
• scientifically rigorous review
• the science that underlies federal regulation
• study
• Studies that have already been subjected to
  adequate independent review
• regulatory information . . . any scientific or
  technical study . . . relevant to regulatory
  policy

19
Addendum (cont.)

• Economics and Social Sciences
• Submissions from industry, public interest
  groups, other agencies, academics