The NIH review process

1
The NIH review process
Kathryn Lee, RN, PhD April 3, 2009 MDP
2
(No Transcript)
3
Grant Mechanisms

• R series (research projects)
• R01 -individual research
• R03- small, short-term
• R21- exploratory/developmental
• R15 certain institutes with little NIH funding

4
1946
One Review Platform for 62 years
The First NIH Study Section
5
The Letter of Intent (LOI)

• Used by federal agencies/foundations
• Filter applications to their interest area
• Appoint appropriate reviewers
• Specific to the agency, typically ask for
• Abstract
• NIH Biosketch
• Nomination letter (some, but not all)

6
Writing an LOI- Start Early

• Limited to 1-2 pages
• Title of proposal
• Background of applicant (or Biosketch)
• Objectives
• Design and Methods
• Statistical analysis plan

7
LOI Can really help your process

• finalize key aims/questions
• get prepared to submit a well designed
  application in short time frame
• Start a relationship with your future program
  officer

8
Letters of Intent

• Theyre Not Easy !
• If I had more time, I would have
• written you a shorter letter.
• Mark Twain

9
National Institutes of Health

• NIH Mission
• Promote biomedical and behavioral research to
  help improve the health of all Americans
• Carried out through 27 Institutes and Centers

10
FY 2007 NIH Budget is 28.6 Billion
Spending at NIH 4.5 B
Spending Outside NIH 24.1 B
11
Goal of Peer Review

• Independent, fair, thorough, and competent review
  of each application
• Identify and rank appropriately those
  applications that show the greatest promise of
  advancing biomedical science and/or improving
  disease prevention, diagnosis and treatment

12
Role of Reviewer

• Provide judgment of the scientific merit of each
  application NOT TO DETERMINE FUNDING
• Criteria for selecting reviewers
• Record of scientific excellence
• Able to see big picture
• Fair and balanced
• Willing to follow guidelines and stay w/in
  required time frame
• Articulate opinions in a clear, concise manner
• Open minded to the views of other reviewers

13
Conflict of Interest

• Worked with key personnel in past 3 years or
  currently
• Financial gain
• Close relative of key personnel
• PI is from your institution
• Recognized scientific disagreement
• PI was your student/major dissertation advisor
• You are applying for job at PIs institution
• PI is applying for job at your institution

14
Review Details

• Each application 3-4 assigned reviewers
• Primary , Secondary full written critique
• Discussant(s)- summary paragraph

15
Tips for Reviewers

• Focus on science, not grantsmanship
• Keep in mind big picture
• Distinguish between major problems and minor
  concerns or differences in approach
• lack of detail what is missing and why it
  matters
• Trust me proposal

16
Written Critiques

• Address each of 5 review criteria
• Address human subjects/inclusion
• Overall evaluation/summary paragraph
• Be specific, constructive, and concise
• Not a time for mentoring
• Do not identify yourself

17
Review Criteria

• Significance 1 exceptional 9 poor
• Approach (1 - 9)
• Innovation (1 - 9)
• Investigators (1 - 9)
• Environment (1 - 9)
• Overall IMPACT

18
New Scoring

Guidance on weighing strengths and weaknesses
Overall Impact Score
Exceptional
1
High Impact
Strengths
Outstanding
2
Excellent
3
Very Good
4
Moderate Impact
Good
5
Satisfactory
6
Fair
7
Low Impact
Marginal
8
Weaknesses
Poor
9
19
Chairs Role

• Ensure that all appropriate viewpoints are
  expressed
• Ensure that discussion is fair, balanced, and
  appropriate
• Promote consistent scoring
• Summarize panels views
• Time management

20
Percentiles

• Scores are translated into percentiles which are
  used to make funding decisions
• Currently most institutes funding at 8-15th
  percentile
• Score clustering results makes it easier to fund
  out of order

21
Dual Review System for Applications
1st Level of Review Scientific Review Group
(SRG)

• 2nd Level of Review
• NIH Institute/Center Council

22
Streamlining

• Purpose make more efficient use of time at
  meetings
• Goal streamline 50-60, so proposals that are
  not competitive are not discussed
• Streamlining does NOT equal BAD

23
CSR Peer Review 2008

• 77,000 applications received
• 56,000 applications reviewed
• 16,000 reviewers
• 240 Scientific Review Officers
• 1,600 review meetings

24
Major Complaints About NIH Peer Review

• Process is too slow
• Not enough senior/experienced reviewers
• Process favors predictable research instead of
  significant, innovative, or transformative
  research
• Time and effort required to write and review are
  a heavy burden on applicants and reviewers

25
Assign Applications Accurately Efficiently

• Retooled for electronic submission
• Applications are now submitted electronically
• Assign applications using text fingerprinting,
  and text mining programs
• Full Implementation by early 2009

26
Fund best research earlier reduce burden on
applicants, reviewers, NIH

• More flexible deadlines
• Abolish A2 applications

27
Improve Quality Transparency of Peer Review

• May-July meetings 2009
• Shorten summary statements, follow template for
  each criteria
• Change the rating system
• Use 1-9 integers
• Score each criteria
• Provide score for all applications (even those
  not discussed)
• Spring 2010
• Shorten applications, aligning with review
  criteria
• Impact, investigator, innovation (if applicable),
  research strategy, facilities

28
Number of Applications Submitted
Historical Growth
29
Reviewers Load
Applications Per Reviewer
October Council Rounds
30
RO1 Resubmission Within 4 Months of Original
Application
31
Confidentiality

• What happens in study section stays in study
  section
• Materials are proprietary
• Dont discuss outside of the meeting
• Dont show application to anyone else
• Avoid web sites associated with grant

32
NIH resources

• Proposal writing guides
• Avoiding common mistakes in an application
• SON website link to NIH website at 
• http//nurseweb.ucsf.edu/www/ix-rs.shtml.