Responding to Review Critiques and Revising Grant Applications

1
Responding to Review Critiques and Revising Grant
Applications

• Sheri Schully, Ph.D.
• Epidemiology and Genetics Research Program
• Pebbles Fagan, Ph.D., M.P.H.
• Behavioral Research Program
• National Cancer Institute
• New Investigator Meeting
• May 28, 2009

2
Role of Program Director

• The PD will discuss general comments, not
  specific comments of each reviewer, and provide
  context.
• The PD will answer questions that you may have
  about the reviewer statements.
• The PD will also point out major weaknesses and
  strengths based on the consistency of responses
  among reviewers.

3
Role of the Principal Investigator

• Retrieve and review the summary statement after
  it has been released in NIH Commons.
• Develop a list of questions for the PD.
• Make an appointment with the PD to discuss the
  summary statement.
• Discuss summary statement with PD.
• Develop a plan for addressing the comments as
  well as anticipate any new comments that may
  evolve
• Revise and resubmit the grant if warranted.

4
Critical Elements of the Summary Statement

• Program Contact
• Impact/Priority Score
• Percentile (if applicable)
• Resume and Summary
• Reviewer critiques and individual scores for 5
  areas
• Council, Meeting, and Start Dates
• Human Subject and Vertebrae Concerns
• Administrative Budget Notes (not part of scoring)
• Official Meeting Roster

5
Critical Elements of the Reviewer Critiques and
Individual Scores

• Overall Impact
• (scores range from 10-90)
• Each of the five areas will
• receive a score from 1-9
• Significance
• Approach
• Innovation
• Investigators
• Environment
• Overall Evaluation

• Human Subject Protection
• Inclusion of Women,
• Minorities, Children
• Vertebrate Animals
• Resubmission, Renewal,
• Revise
• Biohazards
• Data Sharing
• Budget (not part of scoring)

6
Reviewing, Interpreting, and Responding to
Summary Statements
7
Where to Begin

• You will have the grants priority score long
  before you get the SS try to keep an open mind
  about the written critique.
• If you suspect that funding is unlikely, read the
  SS once, then set it aside for a few days.
• Strive to read everything that is written, not
  just what you want to see.
• Look for common elements in the comments of the
  different reviewers .

8
Where to Begin

• Ask 2 or 3 experienced grant writers and
  reviewers to read your SS after you have read it
  once, before you dwell on it.
• After early emotions have subsided and you and
  your colleagues have calmly discussed the review,
  call your Program Director to gain deeper
  insight.

9
Summary Statement Resume and Summary of
Discussion

• Represents the overall consensus, provides the
  reviewers level of enthusiasm, and highlights
  the major strengths and weaknesses of the
  application.
• Description tells you if you have fatal flaws
  that the reviewers believe 1) cannot be overcome,
  Or 2) can be addressed in a revision.

10
Summary Statement Human Subjects

• Review notes to determine if there are any
  concerns since these factors could impact your
  score.
• Be prepared to address these concerns in your
  revised grant or if grant is awarded, be prepared
  to respond to your program director.

11
Summary Statement Budget

• Not part of scoring, but you may have to revise
  your
• budget based on reviewer comments.
• Ex. You have proposed a study for 5 years when
  the reviewers feel that you can do it in 4 years.
• Ex. You have too much support (level of effort)
  for senior researchers on your grant, which
  increases the cost for your grant.
• Ex. If modular budget, reviewers may adding or
  subtracting a module

12
Summary Statement Investigator

• Reviewers may feel that you are not ready or
  have not assembled the appropriate team. Based
  on their comments, you may have to consider the
  following solutions
• Publish more work in your area of expertise
  before resubmission.
• Collect more data to demonstrate your expertise.
• Revamp your team to include a stronger
  investigators with the appropriate expertise. May
  need to expand your networks outside your
  department or institution.
• Reorganize your team.
• Better describe how you will integrate expertise
  on your team.
• Revise the study so that it fits your area of
  expertise.

13
Summary Statement Significance

• Your study may have a significant impact, but you
  have to convince the reviewers.
• Your job is to first evaluate whether it truly is
  solving an important problem that will change
  scientific knowledge, technical capability, or
  practice in the field.

14
Summary Statement Significance

• If your reviewers feel that it is not
  significant, then its important to repackage your
  product and re-tell your story.
• This requires vision on your part and addition
  discussions with your team members.
• Ex. What is so important about developing digital
  mammography screening?
• Ex. What will the genetic associations tell us
  about the disease?
• Ex. Why is it important to understand how
  cigarette excise taxes impact quitting behaviors
  among smokers?

15
Summary Statement Innovation

• Prior to submission you should ask what is new
  and innovative about the study?
• Does it use novel theoretical approaches,
  methods, instruments, or interventions? If so,
  did you market that innovation as such?
• The reviewers may feel that it is significant and
  will contribute to the field, but not innovative.
  
• Like any business person, your job is to sell
  innovation .

16
Responding to and Dealing with Common Criticisms
on Significance and Innovation

• Solution
• Make sure colleagues agree the problem is
  important explain its importance in the Intro to
  the revision, also in the Background section.
• Point out specifically what is innovative
• Review NIH priorities before
• resubmitting revise accordingly

• Problem
• Problem not important
• Lacks innovation
• Does not fit NIH priorities

17
Summary Statement Approach

• Common problems
• Measurements, study eligibility, analysis plan,
  careful consideration of all factors impacting a
  behavior or condition, not considering
  alternative strategies, feasibility, appropriate
  study arms, recruitment and retention, adequate
  power to detect differences, contamination,
  fidelity, human subject issues.
• If you have to make a change to one section and
  it effects another, be sure to make those changes.

18
Responding to and Dealing with Common Criticisms
to the Approach

• Problem
• Too ambitious
• Approach or methods not appropriate
• Lacking Methods details
• Unfocused

• Solution
• Limit the number and scope of Specific Aims
  match them with your budget, focus your writing
• Ask experts in your field to help you revise
  Research Design and Methods. Work closely with
  your team members to redress the sections that
  he/she has written.
• Provide more detail get expert advice from
  reviewers of your drafts
• Revise the proposal (2-3x) with input from
  experienced grant writers and reviewers

19
Summary Statement Environment

• You should evaluate support systems in your
  environment prior to taking the job.
• If your institution does not have the appropriate
  support, then you may consider adding a
  collaborator at another institution that does.
• Ex. You want to collect genetic samples among
  adolescents but, your institution does not have a
  history of doing so or analyzing these samples.
• Ex. You want to collaborate with a CBO, but you
  have not written in the appropriate equipment for
  the CBO to perform its part of the study.

20
Summary Statement Overall Evaluation

• Review each evaluation since additional concerns
  may be raised even though concerns may not be
  embedded in the summary critique.
• Since you may not always know the weight of the
  concern, it is important to address concerns even
  if they are not in the summary critique.
• This is a great place to gauge the individual
  reviewers level of enthusiasm for your study.

21
Final Pointers Regarding Revisions
22
Keep a Constructive Attitude Towards Reviewers

• If the reviewer is wrong, most likely its
  because there is a problem with the application
• Lack of clarity?
• Unfocused writing?
• Essential details missing, esp. in Methods
  section?
• Let reviewers help you use their feedback to
  improve the application.
• Dont let your own defenses keep you from
  reapplying.
• Remember you are a scientist, which means your
  work is subject to and sometimes strengthened
  through critiques.

23
Evaluate Your Study Section

• Consider changing Study Sections if
• The first one clearly lacked appropriate
  expertise ask for a change in a covering letter
  after reviewing SS charters. However, you should
  review the roster prior to submission (available
  on the CSR website)
• Your grant was assigned to a second best SS
  because a co-investigator serves on the preferred
  group. Consider removing that person from the
  grant. (Add him/her back after the project is
  funded.) Prior to submission however, you can
  ask for the study section in your cover letter.
• CSR does not always grant requests to change
  study sections. It is up to the SRO of that SS to
  switch your application

24
Take Care in Rewriting Your Grant

• Call your PD again if you have questions about
  your summary statement.
• Your PD often has an idea of study section
  atmosphere and to which study sections particular
  grants are reviewed.
• Use the 3 page Introduction to a revised proposal
  to respond thoroughly to the critique.
• The Introduction should make sense by itself.
• Distinguish unchanged from revised text in the
  proposal (italics, vertical line in the margin)

25
Take Care in Rewriting Your Grant

• Respond to each criticism
• Agree and make changes, OR
• Disagree diplomatically, on scientific grounds,
  explain why
• Address each concern directly, point by point,
  whether you agree or disagree. Reviewers will
  recognize when you have not addressed a concern,
  brushed them off, or ignored them.
• Review each critique and look for consistencies
  among reviewers.
• Re-critique your own study
• Have several others review your grant prior to
  resubmission.
• If you are called to serve on study section,
  accept. It will give you an inside perspective on
  the review process.