Introduction to ORD

1
Introduction to ORD

• Kevin Y. Teichman, PhD
• Director, Office of Science Policy
• Office of Research and Development
• November 17, 2005

2
Office of Research and Development

• 1950 employees
• 600 million budget
• 70 million extramural research grant program
• 13 lab or research facilities across the U.S.
• Credible, relevant and timely research results
  and technical support that inform EPA policy
  decisions

3

• Office of Research and Development
  
• Mission
• Advance scientific knowledge to solve the
  environmental problems the Agency faces
• Perform human health and ecological effects
  research that provides scientific discoveries
  responsive to the environmental questions the
  Agency must address
• Support EPA Program Offices, Regions, and other
  governmental and non-governmental organizations
  through scientific and technical advice and
  assistance so that their operations benefit from
  the most up-to-date science
• Provide scientific leadership in identifying,
  studying, and resolving critical environmental
  health and ecological effects issues and in
  shaping the environmental health and ecological
  effects research agenda

NHEERL Organizational Strategy 2000-2005
4
Support for EPAs Mission
EPA Mission Protect human health and
safeguard the natural environment air,
water, land upon which life depends
REGIONAL OFFICES Primary Interface with States
PROGRAM OFFICES (Air, Water, Waste,
Pesticides/Toxics)
Policies, Regulations
Congressional deadlines
National Decisions
Implementation
OFFICE OF RESEARCH AND DEVELOPMENT
Scientific Foundation
5
Office of Research and Development
Immediate Office of the Assistant
Administrator George Gray, Assistant
Administrator Bill Farland, Acting Deputy
Assistant Administrator for Science Lek Kadeli,
Deputy Assistant Administrator for
Management Michael Brown, Associate Assistant
Administrator
Office of the Science Advisor Bill Farland,
Acting
National Program Directors Air
Dan Costa Drinking Water Gregory Sayles,
Acting Water Quality Chuck Noss Pesticides
and Toxics Elaine Francis Land Randy
Wenstel Human Health Hugh Tilson, Acting
Ecosystem Protection Kevin Summers, Acting
Global Change/Mercury Joel Scheraga
Office of Science Policy Kevin Teichman
Office of Resources Management and
Administration Jack Puzak
National Center for Computational Toxicology Robe
rt Kavlock
National Risk Management Research
Laboratory Sally Gutierrez
National Center for Environmental Research Gary
Foley
National Homeland Security Research Center Andy
Avel, Acting
National Center for Environmental Assessment Pete
r Preuss
National Health and Environmental
Effects Research Laboratory Hal Zenick, Acting
National Exposure Research Laboratory Larry
Reiter
6
How ORD Evolves its Research Program
ORD Executive Council Corporate Decisions on
What We Do . . . and . . .
How We Do It
Decision Inputs Programs and Regions (RCTs) EPA
Strategic Plan Administrations
priorities Congressional mandates BOSC
Reviews SAB, NAS, other external
advice Stakeholders NPDs, SC, MC, EC
Evaluation Program and Regional Office
Feedback BOSC Program Evaluations NAS, NAPA,
and other advisory bodies PART Reviews
Planning the Program
Implementing the Program
L/C Directors Responsible for Developing ORDs
Research Products
NPDs Decide What Research Area-Specific Work We
Do and When We Do It
L/C Directors Decide How ORD Produces its
Research Products
NPDs Responsible for Communicating Products to
Clients
7
Implementing an ORD Evaluation Framework
Clients
Research Program
Environmental Outcomes and Results
Research Activities
Research Outputs
Short-Term Outcomes
Intermediate Outcomes
Long-Term Outcomes
Resources
Policy Development
Strategic Goals
Strategic Objectives

Independent Expert Evaluation
Focus Outcome-oriented progress RD investment
criteria Evidence MYPs synthesis products
performance data client feedback Attribution
Sphere of influence that includes ORD, EPA
clients partners
8
Purpose of the Multi-Year Plans

• Planning tool to address Agencys high priority
  science questions
• Provides information to assist and support
  resource decisions
• Demonstrates how program contributes to Agency
  strategic goals
• Assists in determining accountability of
  performance
• Provides information to be used in Program
  Assessment Rating Tool Review by OMB
• Communicates research inside and outside ORD

9
Multi-Year Plan Elements

• Long-Term Goals (LTG)
• Identify time-frame to deliver work
• Determine ORD role and role of others

• Annual Performance Goals (APG)
• Identify sequence to provide results
• Integrate research from all sources

• Annual Performance Measures (APM)
• Determine who will accomplish work (in-house lab
  or center or STAR research)
• Ensure work can be done with available resources

10
Multi-Year Research Plans

• Ecological Research
• Economics Decision Sciences
• Human Health
• Pollution Prevention

• Air Toxics
• Land Preservation and Restoration
• Drinking Water
• Endocrine Disruptors
• Global Change
• Mercury
• Particulate Matter
• Safe Pesticides/Safe Products
• Water Quality

11
From Planning to Program Evaluation
Planning
Performing Research
Communicating Research Results
Evaluating Performance
12
BOSC Program Evaluations

• Help answer the questions
• Are we doing the right science?
• Are we doing the science right?
• Provide guidance for evolving the research
  program
• Provide evidence for OMB evaluations using the
  Performance Assessment Rating Tool

13
Program Assessment Rating Tool(PART)

• Evaluates program effectiveness in four areas
  Purpose/Design, Strategic Planning, Program
  Management, and Program Results
• Programs receive numerical score and rating
  (Effective, Moderately Effective, Adequate,
  Results Not Demonstrated, Ineffective)
• Results based on annual and long-term performance
  goals with emphasis on outcomes
• External program evaluations are addressed in
  both the Strategic Planning and Results sections

14
RD Criteria

• RD specific questions in PART reflect OMB/OSTP
  Investment Criteria for RD Quality, Relevance,
  Performance
• RD investments must be clearly planned to be
  relevant to national priorities, agency missions,
  and customer needs
• Programs must maximize the quality of the
  research they invest in
• RD programs must demonstrate performance by
  setting annual and long-term goals and
  demonstrating progress toward outcomes

15
RD Criteria

• Relevance
• The purpose of the research program is clear
• The program responds to a specific existing
  environmental problem relevant to EPAs mission,
  national priorities, and primary clients
• The program demonstrates an outcome-oriented
  design
• The programs benefits (e.g, contribution to
  outcomes) are unique or extend beyond similar
  government or private sector contributions
  program coordination is effective in minimizing
  or avoiding duplication
• There are a small number of performance goals
  focused on scientific progress to answer key
  questions (or reduce uncertainty) linked to the
  programs outcomes

16
RD Criteria

• Quality
• Merit-based procedures are used to ensure the
  programs scientific quality and leadership. The
  program compares favorably to similar programs
  (e.g., in other agencies).
• When the program allocates funds extramurally
  (e.g., through assistance mechanisms) it ensures
  merit-based competition, relevance to the
  programs objectives, and independent review by
  subject matter experts.
• When the program allocates funds
  non-competitively (e.g., to federal laboratories)
  appropriate merit-based procedures are used.
• The program may conduct benchmarking of
  scientific leadership and other factors as one
  means of assessing program quality.

17
RD Criteria

• Performance
• The program identifies relevant inputs (e.g,
  stakeholder guidance, human capital, research
  infrastructure) to ensure that implementation
  actually results in the intended research
  activities outputs
• The program demonstrates the ability to produce
  identifiable results
• Conceptual frameworks (risk paradigm, logic
  model) link research questions, performance
  goals, clients, and outcomes with a specific
  environmental problem
• Performance goals serve to answer key research
  questions and track how the program will improve
  scientific understanding and its application.
• The program periodically assesses research
  progress and priorities as new scientific
  knowledge is developed
• The program demonstrates that it meets
  performance goals
• The program obtains client feedback and
  demonstrates that progress is being made to
  achieve outcomes.

18
Summary

• ORD seeks input from many sources to enhance and
  evolve its research program.
• BOSC program evaluations are one of the most
  important inputs.
• Your review of the Land Preservation and
  Restoration Program will be of great value, and
  is much appreciated.