Evaluating the EHR Portfolio

1
Evaluating the EHR Portfolio

• Judith A. Ramaley
• Assistant Director
• Education and Human Resources

2
Current Forms of Review

• Committees of Visitors (focus on a single
  program)
• GPRA Reports (focus primarily on individual
  awards i.e., nuggets)
• Third Party Evaluation (usually a single program)
• None of these approaches encompasses a portfolio
  model

3
Concept of a Portfolio

• The securities held by an investor
• A flexible case for carrying loose papers,
  pictures or pamphlets
• Source Websters Ninth New Collegiate Dictionary

4
A Policy Framework

• Defining the K-16 STEM Program Portfolio and its
  Relevance to Meeting National Needs
• Addresses areas of critical national need
• Furthers the NSF mission
• Well designed
• Promotes collaboration between education and
  research communities
• Demonstrates accountability produces research
  findings that translate theory into practice
• Fosters implementation of successful models in
  new or expanded settings
• Increases participation of underrepresented
  groups in STEM education and careers

5
Application of the Policy Framework

• Portfolio Review
• Review of Newly Proposed Programs
• Program Accountability and Revitalizing the
  Portfolio

6
The NSF Mission

• The Mission of the NSF is
• To promote the progress of science to advance
  the national health, prosperity, and welfareand
  to initiate and support basic scientific research
  and programs to strengthen scientific research
  potential and science education programs at all
  levels (Source National Science Foundation Act
  of 1950)
• Through discovery, learning and innovation that
  will ensure that the nation has the people, ideas
  and tools necessary for a healthy science and
  engineering enterprise

7
The NSF Mission

• The Mission of the NSF is
• Achieved through the core NSF-wide strategies of
  development of intellectual capital, integration
  of research and education, and the promotion of
  partnerships
• In order to (a) make it possible for the U.S. to
  uphold a position of world leadership in all
  aspects of science and engineering (b) promote
  the discovery, integration, dissemination, and
  employment of new knowledge in service to
  society and (c) achieve excellence in U.S.
  science, technology, engineering and mathematics
  education. (Source Core Strategies to Realize
  the Outcome Goals, NSF GPRA Strategic Plan FY2001
  2006.)

8
The EHR Portfolio

• EHRs mission is to achieve excellence in U.S.
• science, technology, engineering and mathe-
• matics (STEM) education at all levels and in all
• settings (both formal and informal) in order to
• support the development of a diverse and well-
• prepared workforce of scientists, technicians,
• engineers, mathematicians and educators and
• a well-informed citizenry that have access to the
• ideas and tools of science and engineering.
• The purpose of these activities is to enhance the
• quality of life of all citizens and the health,
• prosperity, welfare and security of the nation.

9
Goals of the EHR Portfolio

1. Prepare the next generation of STEM professionals
   and attract more Americans to STEM careers.
2. Increase the technological and scientific
   literacy of all Americans so that they can
   exercise responsible citizenship in an
   increasingly technological society and acquire
   knowledge of science, mathematics and technology
   that is appropriate to the development of
   workforce skills and life-long career
   opportunities.

10
Goals of the EHR Portfolio

1. Broaden participation (diversity) and achievement
   in STEM.
2. Attend to critical workforce needs requiring
   significant math and science skills and
   knowledge, both by attracting new people to these
   STEM careers and by support for the development
   and retooling of the current STEM workforce.

11
Capacity-Building Strategies

1. Identify effective ways to prepare and support
   teachers and faculty who can inspire and
   challenge students in the STEM disciplines and to
   provide them with effective materials and
   strategies to promote and assess learning.
2. Invest in research in the science of learning,
   facilitating the translation of research into
   practice, and create supportive learning
   environments and STEM pathways by developing
   models of reform/systemic change at both
   institutional and multi-institutional levels
   through networking, partnerships, alliances and
   collaborations.

12
Capacity-Building Strategies

1. Ensure that the STEM community is broadly
   representative of the nations individuals,
   geographic regions, types of institutions and
   STEM disciplines.
2. Identify effective ways (formal and informal) to
   address the STEM knowledge requirements of adults
   so that they can be productive members of the
   workforce and informed and active citizens.

13
Elements of the EHR Portfolio

• Supporting individuals to enhance the
  capabilities of individuals preparing for STEM
  careers (including K-12 teachers) and people
  currently in the workforce, and to attract a
  diverse group of people to become scientists,
  engineers, technologists, mathematicians and
  educators
• Developing ideas, tools and communication
  strategies that will enhance the quality of STEM
  education inform all citizens about science,
  technology and mathematics and enhance public
  understanding of current research and its
  implications
• Utilize a solid body of evidence created by an
  analysis of the existing research and examination
  of practice to guide further development of the
  NSF portfolio as well as to support the diffusion
  and implementation of strategies that promote
  educational excellence.

14
Elements of the EHR Portfolio

• Supporting the development of links among
  networks of researchers and practitioners that
  help create the capacity to introduce and sustain
  change in order to enhance the quality of STEM
  education and broaden participation in STEM
  careers and
• Developing institutional environments and
  enhancing system level interactions and
  partnerships that support change and the
  introduction of strategies that will enhance the
  preparation and development of the STEM workforce
  creating the infrastructure to support
  collaboration across disciplines and across
  institutions that promote excellence in STEM
  education and facilitate the application of
  innovation to economic development.

15
Evaluating the EHR Portfolio

• Adopt a thematic approach
• Utilize an external panel of researchers and
  educators
• Conduct two reviews a year, each requiring
  approximately 6 months
• Select topics based on changes in NSF priorities,
  emerging opportunities, strong public interest,
  and budget availability
• First topic will be mathematics education (K-12
  and transition to higher education)
• Second topic will be use of technology in
  education
• Additional topics to be developed in consultation
  with the EHR Advisory Committee

16
Purpose of the Review

• Assess the condition of the field and its most
  significant challenges and opportunities.
• Identify the domains where our strategic
  investments can have the most impact utilizing
  the Road to Excellence Policy Framework as a
  guide.
• Assess the quality and impact of the current
  portfolio of awards.
• Identify areas of programming that deserve
  greater attention as well as opportunities for
  revision or elimination of program emphases.

17
Design of the Thematic Review

• First Workshop
• Identify domains where NSFs strategic
  investments can have the greatest impact.
• Articulate an analytic framework for organizing
  the portfolio that will permit an assessment of
  program alignment identification of unmet needs
  options for revisions of existing programs and
  areas of possible unnecessary duplication.

18
Design of the Thematic Review

• Second Workshop
• Review the portfolio and assess its quality and
  how well it addresses critical issues in the
  field.
• Identify areas for future emphasis and
  revision/elimination.
• Recommend approaches to evaluating the portfolio
  in the future.

19
Portfolio Review Letter Report

• Evaluation of the portfolio
• Assessment of the alignment of the portfolio with
  needs of the field
• Recommendations for future action by NSF