How Not to Write an Unsuccessful Grant Proposal

1
How Not to Write an Unsuccessful Grant Proposal

• Daniel Clauw, M.D.
• Assistant Dean of Clinical and Translational
  Research
• Director, Center for the Advancement of Clinical
  Research
• The University of Michigan

2
What is the end result you are aiming for?Good
science by a good investigator.
- Science
25 of applications Good idea, but Investigator
is unprepared to do the work.
50 of applications Bad idea, and poorly
prepared investigator.
15 of applications Great idea, proposed by the
perfect person to test it.
- Investigator
10 of applications Bad idea, but capable
Investigator.
Table developed by James Ferrara, M.D.
3
How to Get a Grant

• Design a sound study
• Write a good grant

4
Early Steps in Clinical Research

• The idea
• The hypothesis
• The design
• The study subjects
• Can I do it?

5
The idea

• Most clinicians can come up with good ideas about
  clinical research based on gaps in existing
  knowledge
• Has question been asked before?
• If not, why?
• If so, how does my question or design add to the
  existing literature?

6
At the Beginning . . . Think of the End

• At the inception of a clinical research project,
  ask a series of questions about the end product
• Who is the audience?
• Where would the article be published if it is
  successful?
• Is a negative study interesting or publishable?
• How would the results impact clinical care
• i.e. Who would care?

7
The First in a Series of Compromises
8
Potential audience

• Academics
• Peer review process
• Practicing clinicians
• Regulatory agencies (e.g. FDA)
• Industry (marketing bias)
• Public

9
Early Steps in Clinical Research

• The idea
• The hypothesis
• The design
• The study subjects
• Can I do it?

10
Developing the Study Hypothesis / Goal(s)

• A surprising number of even established clinical
  researchers begin designing the study before they
  define the hypothesis
• Keep It Simple (KISS)
• Unless you are in the position to do a
  hypothesis-generating study, have a single
  primary hypothesis and no more than 2 3
  secondary objectives

11
Early Steps in Clinical Research

• The idea
• The hypothesis
• The design
• The study subjects
• Can I do it?

12
Compromises in Study Design
Efficacy
Effectiveness
Long trial
Short trial
Single site
Multicenter
Randomized, Blinded
Not
Active comparator Placebo Open

13
Early Steps in Clinical Research

• The idea
• The hypothesis
• The design
• The study subjects
• Can I do it?

14
Inclusion and Exclusion Criteria

• Might be most important aspect of study design
  with respect to
• Ability to recruit subjects
• For clinical trials, this will largely determine
• Efficacy of treatment
• Placebo response rate
• For mechanistic studies, this needs to be
  considered for both patient (clearly define) and
  control groups

15
Selecting Participants
Heterogeneous Homogeneous
Extrapolatability of results Smaller n, more
responsive
Tertiary Care
Population Easy to find / unresponsive
High placebo response

16
Selecting Controls

• Should ideally have all of the characteristics of
  the patient, except the condition being studied
• Not always possible
• Controlling for confounds
• Sampling
• Statistics
• What is a healthy normal control?

17
Recruitment

• Advertising
• Apples vs. oranges
• gt Screen failures
• Higher placebo and drop-out rates

• Existing patients
• Easy to find
• Pre-screened
• Stay in study longer

18
Early Steps in Clinical Research

• The idea
• The hypothesis
• The design
• The study subjects
• Can I do it?

19
ResourcesDo I have enough?

• Mentorship
• Individual
• Institutional (K-30, K-12)
• Patients / subjects
• Do I have enough?
• Do I have the right ones?
• Staff
• Money
• Federal grants
• Institutional resources (e.g. GCRC)
• Foundations
• Industry

20
How to Get a Grant

• Design a sound study
• Write a good grant

21
Get FREE Money from the Government!!!
OK, so maybe you dont need this guys book. But
you need some of his skills . . . . .
22
Besides good research, you need

• Communication Skills
• Marketing Skills
• Management Skills
• Flexibility and ingenuity (who can you sell your
  research to?)
• Ability to follow directions, even the apparently
  meaningless ones
• Ability to plan ahead and commit significant time

Adapted from Christine Black, UM DRDA
23
When? Earlier than you think

• Internal vs. external deadlines
• Should get science of grant done in plenty of
  time to have others read final grant, and the
  best people to read grant have the least time to
  do so
• Administrative shell of grant typically needs to
  be turned in 10 days before grant is due
• Study design issues, and especially sample size
  calculations, are necessary for administrative
  shell
• Need for preliminary data

24
Do your homework

• What has the agency funded recently? (CRISP,
  Community of Science, www.guidestar.org for
  foundations).
• Read recent successful applications by
  colleagues.
• The program officers are your resource, contact
  them early and often.
• Colleagues are essential for collaboration and
  consultation senior colleagues may know who will
  likely review your grant.
• Consult a biostatistician early in the process.

25
Do your homework, part 2

• Check study section rosters!
• Search the literature to determine roster
  members expertise.
• If there are content area experts on the study
  section, does your literature review cite their
  (appropriate) publications?

26
Do your homework, part 3

• Are their aspects of your project or methodology
  that may need extra justification because they
  will be unfamiliar to this roster of people?
• For foundation grants, trustees or boards may
  make funding decisions, so be sure that key
  sections like specific aims are in lay language.
• P.S. Do not attempt to contact reviewers!

Inouye, S.K., Fiellin, D.A. An Evidence-Based
Guide to Writing Grant Proposals for Clinical
Research, Ann Intern Med. 2005142274-282
27
Relationships are key

• Step 1 Get the right team together. Find people
  with expertise in different, specific areas
  working with this specific patient population,
  using this specific methodology or technique,
  statistically analyzing this type of data, etc.
• Step 2 Buy your grants administrator a nice
  gift. You will be depending on this person!

28
Why you should be nice to your grants
administrator
This is the easy part. The challenge will be
figuring out the indirect cost rate. Chronicle
of Higher Education, Carol Cable
29
Care and feeding of the reviewer

• They are
• Overworked (they will at most spend a few hours
  reviewing your grant)
• Mindful of the need to further their own careers
• Not interested in doing outside homework in a
  new topic area just so they can understand your
  proposal
• Mature
• Translation they dont want to strain their eyes
  deciphering figures and captions that youve
  shrunk down to 8-point font!

Adapted from Christine Black, UM DRDA
30
Care and feeding of the reviewer, part 2

• Assume reviewers are intelligent and savvy about
  research
• ..but perhaps have little in-depth experience in
  your area of interest.
• Avoid jargon and topic-specific abbreviations.
• Many reviewers will read only the abstract and/or
  specific aims.

Inouye, S.K., Fiellin, D.A. An Evidence-Based
Guide to Writing Grant Proposals for Clinical
Research, Ann Intern Med. 2005142274-282
31
Care and feeding of the reviewer, part 3

• Make reviewer like you and your study in first
  two pages so that you start with a 1.0
• Teach them something new
• Convince them that you are the best person in the
  world to do this terrific study
• Science Fiction Novel approach the world will
  be a better place after this study is done
• Disneyland approach make them smile while you
  are sucking money out of their wallets
• If you can start with a 1.0 and eliminate
  targets youll be fine

32
Targets

• Controversial aspects of a grant
• Its OK to do it, but dont write it
• Omissions
• Failure to follow rules

33
Common Problems in Applications (from NIH and
NIAID)

• Lack of knowledge of published relevant work
• Lack of experience in the proposed methodology or
  techniques
• Uncertainty concerning future directions
• Unrealistically large amount of work
• Failure to discuss potential obstacles or
  alternative approaches

Adapted from Christine Black, UM DRDA
34
Major issues identified in NIH grant proposals

• Results from a review of pink sheets from 66
  R01 applications the authors categorized major
  problems found in unfunded grants.
• Findings presented for each major grant section.
• Inouye, S.K., Fiellin, D.A. An Evidence-Based
  Guide to Writing Grant Proposals for Clinical
  Research, Ann Intern Med. 2005142274-282

35
Abstract

• The broad, long-term objective is . . . .
• The Hypothesis is . . . .
• The Specific Aims are . . . .
• The Research Design is . . . .
• The health-related relevance of this project is .
  . . .
• Avoid
• First person
• Excessive summary of past accomplishments
• Amounts of money
• Going outside the box

36
Specific Aims/Hypotheses

• This (and abstract) are most important parts of a
  grant.
• Write this early in the process, send it around,
  and keep re-writing to make it clearer, stronger,
  and more concise.
• This should be a two page executive summary of
  the following 23 pages of the grant.
• Writing tips
• ½ to one page of setting the stage
• Extremely clear, (one sentence if possible)
  hypothesis
• Specific aims (between 2 and 4) stated as action
  items
• To determine, evaluate, confirm, show . . . .
• Common critiques
• Poorly Focused
• Too Ambitious

37
Comments from NIH Reviewer

• In addition to proposing a research design that
  is a fishing expedition, the applicant also
  proposes to use every type of bait and piece of
  tackle known to mankind.

Adapted from Christine Black, UM DRDA
38
Special advice for Training (e.g. K-) awards

• Safe study is more important than
  ground-breaking science.
• Should be a study that leads to other studies, no
  matter what the results.
• Serves as a vehicle for your training.
• Should ideally encompass all of the elements of a
  research project, especially those that you have
  inadequate experience with.
• Environment
• Increasing preference for mentoring teams.
• Letters are extremely important, and read for any
  nuances.
• If your mentor cannot take the time to write a
  strong gt 2 page letter, they may not have the
  time or commitment to mentor you.

39
Background and Significance

• Begin by describing current state of science.
• Identify knowledge gaps.
• Justify the need for this specific study.
• Move science forward, not just laterally.
• Writing tips
• Dont provide too much extraneous background
  information.
• Dont overstate the significance of your study.
• For each area covered in this section, explicitly
  state the relationship to your proposed project.

40
Comments from an NIH Reviewer

• This application is characterized by ideas that
  are both original and scientifically important.

• Unfortunately the ideas that are scientifically
  important are not original and the ideas that are
  original are not scientifically important.

Adapted from Christine Black, UM DRDA
41
Preliminary Studies

• Should demonstrate
• Feasibility
• You can recruit the required study participants.
• The team has successfully worked together.
• Promising data
• Youve partly done what you are asking for money
  to do.
• If there are novel methodologies that are being
  used, that these have been fully developed
• Writing tips
• Abstract of each study is good start.
• Show how each study links to the proposed work.

42
Methods

• At least 50 of the page allowance should go to
  methods.
• Underdeveloped or vague methods are the most
  common reviewer complaint.
• Describe the study design in detail how will
  you randomize, blind, select controls, etc.
• Inclusion/exclusion criteria justify your
  criteria and address any biases they may cause.

43
Methods, part 2

• Availability of participants
• esp. with regards to women, minorities, and
  children.
• Data collection and procedures
• Discuss each instrument you propose to use
• Describe quality assurance or staff training
• Consider a table of variables

Variable Instrument Characteristics (when available) Reference
Level of Function SF-36 Functional Sub-score Sensitivity Specificity Reliability Ref.
Auto-antibody level SuchandSuch Assay Sensitivity Specificity Ref.
44
Interventions and Outcomes

• Ensure adequate randomization and blinding. What
  will happen if the blind is broken?
• Ensure standardization of your intervention.
  Will you provide training?
• Ensure your intervention is meaningful.
• Ensure that adherence is monitored.
• Clearly define all outcome measures.

45
Statistical Analysis

• Work with your biostatistician often and early.
• Address what you will do about missing data.
• Include realistic attrition rates in your power
  calculations. Address what you will do with data
  from subjects who drop out.
• Present the plan clearly, referencing each
  specific aim when appropriate.

46
Writing style

• Business not creative
• Active voice
• will or expect to not would like to or
  may
• Anticipate questions
• Target analogy

47
Final Touches

• Realistically assess weaknesses or potential
  roadblocks and explain how you would overcome
  them. Dont blow off this section its your
  opportunity to think strategically.
• Provide a timetable or chart summarizing study
  activities.
• Proofread. A lot. Then give it to others.
• Follow all the directions with regards to fonts,
  margins, page limits, deadlines, etc.
• Finish with enough time for administrative
  approval, signatures, etc. check with your
  grants administrator. This process may take 10
  days or more!

48
How to deal with rejection(Youre playing
baseball, not shooting free throws)

• Write scathing letter of rebuttal.
• Throw it away.
• Start with a paragraph of gratitude for
  reviewers insight.
• List reviewers criticisms.
• Do not try to convince reviewers that they are
  idiots
• Address each criticism objectively in the
  introduction to your resubmission.

Adapted from Christine Black, UM DRDA