Trends and Updates: Research on Research Integrity

1
Trends and Updates Research on Research
Integrity

• RRI

• Nicholas H. Steneck, PhD
• University of Michigan
• 2009 ORI RRI Conference
• THE CONFERENCE CENTER NIAGARA FALLS, NY
• May 15-17, 2009

2
Origins of RRI at ORI

• 1999
• Chris Pascal / Larry Rhoades request 1M for RRI
• HHS (NIH source of funds) approves
• Mary Scheetz, ORI program officer
• now Cynthia Ricard ltCynthia.Ricard_at_hhs.govgt
• Contract to study of RRI literature
• Contacted researchers for advice
• July 1999, contact N. Steneck regarding four
  projects
• November advisory committee meeting
• National RRI Conference
• RRI program
• RRI Literature review

3
ORI justification (Pascal 2000)

• Since its establishment in 1992, the Office of
  Research Integrity (ORI) has conducted several
  studies on research misconduct and research
  integrity in an attempt to develop a knowledge
  base on important issues, such as the impact of
  misconduct allegations on exonerated scientists,
  the experience of whistleblowers in the aftermath
  of making allegations, the research guidelines
  adopted by medical schools, and the causes of
  research misconduct.
• Over time, it became apparent to ORI that a more
  comprehensive, coordinated effort in
  collaboration with extramural research scholars
  was needed to develop the science base on
  research integrity issues. This recognition led
  to development of this "Research Conference on
  Research Integrity" and the related "Research on
  Research Integrity" program announcement jointly
  issued by ORI and the National Institute of
  Neurological Disorders and Stroke (RFA
  NS-01-008).

4
Planning Committee

• Ruth Fischbach, Ph.D., M.P.E., National
  Institutes of Health
• Mark Frankel, Ph.D., American Association for the
  Advancement of Science
• Paul Friedman, M.D., UC, San Diego
• Edward Hackett, Ph.D., Arizona State University
• Stanley Korenman, University of California, San
  Diego
• Francis Macrina, Ph.D., Virginia Commonwealth
  University
• Barry Markovsky, Ph.D. University of Iowa
• Kathleen Montgomery, Ph.D., UC, Riverside
• John Perhonis, Ph.D., National Science Foundation
• Drummond Rennie, M.D., F.R.C.P., F.A.C.P., UC,
  San Francisco
• Allan Shipp, M.H.A., Association of American
  Medical Colleges
• Peter Yeager, Ph.D., Boston University

5
Major events

• RRI ( RCR) Conferences
• November 2000, Bethesda MD (Marriott)
• November 2002, Potomac MD (Bolger Conference
  Center)
• November 2004, San Diego CA (Paradise Point
  Resort)
• UC San Diego School of Medicine
• December 2006, Tampa FL (Safety Harbor Resort)
• University of South Florida College of Medicine
• April 2007, St. Louis MO (Washington University)
• Washington University in St. Louis School of
  Medicine
• May 2009, Niagara Falls NY (Conference Center)
• Roswell Park Cancer Institute
• RRI Program, December 15, 2000, deadline, 1st
  round
• ORI does not have granting authority
• Early 2000 explored administration options

6
May NIH presentation, IC Directors
7
Closing slides

• NINDS agreed to be 1st co-sponsor

8
RRI Program History

• 2001, RO1, two-year, 100,000/year, direct cost
• 2004, three-year, 250,000/year direct cost
• Most proposals received
• Lowest percentage of proposal funded
• 2005, two-year, 175,00

• 2008, R22

9
Funding rate average 18
31
28
16
32
19
10
12
13
10
Source of funds / totals

• NIH provides over 50

11
ORI / Other funding (60 / 40)
12
Primary focus, all proposals
13
Areas Proposed / Funded
14
RRI apart from ORI

• NIH funds training and research ethics, not RRI
• Ethics bioethics, not research ethics/integrity
• RCR funded through training grants
• RRI can be considered but few applications
• NSF broad array of programs
• STS gt EVS, HPS, SSS, and SPS
• Research ethics training (REU, IGERT, etc.)
• Peer Review Congresses
• Research on peer review and biomedical
  publication
• Peer generated and supported no funding

15
NSF vs. NIH, RCR education

• ORI/NIH RCR Enhancement Initiative (2002)
• Budget
• National planning and coordination effort
  (1.1m),
• Institutional Support Program (11.5m), and
• Instructional Enhancement Program (6.1m).
• Approved by NIH Director in 2004, never
  implemented
• National Science Foundation, research ethics
• 2001-2009, 328 research ethics awards
• 270,616,429.00 total funding
• Most to support instructional development
• No special attention to or coordination of RRI

• ORI has provided some RCR support

16
Global interest in RRI?

• Some interest in Europe
• Early pioneer, Croatia, now joined by other
  countries
• European Science Foundation has RRI committee
• Just getting organized
• UK RIO has considered, but not yet undertaken
• No support from European Commission
• China is supporting some RRI
• Developing an instrument to assess integrity
• Has investigated plagiarism
• Elsewhere, support for RRI is scattered

17
Observations Professional development

• RRI has become a recognized field of research
• Established researchers anxious to pursue
• Peers have reviewed and endorsed proposals
• Funding agencies provide some support
• Literature and professional community has emerged
• Influences on professional development of RRI
• FUNDING interest grows/declines with support
• Funding mechanism RO1 vs. R22
• External events
• Human subjects concerns
• Conflict of interest

18
Observations Tensions

• Researcher-funder conflict
• Researchers interested in broad (basic) studies
• Funders interested in IC-specific (applied)
  topics
• Research-policymaker conflict
• Researchers interested in broader (basic) studies
• Policy makers want studies that are relevant to
  policy making
• Professional apathy / hostility
• RRI is important but low priority
• RRI is conspiracy to undermine science

19
RRI track record (ORI other)

• Misconduct worldview 1970s
• Serious misconduct is rare
• Self-regulation keeps in check
• Misconduct is difficult to detect
• Misconduct cannot be prevented
• Apart from misconduct, standards are high
• RRI has cast doubt on all five assumptions
• FFP 0.1gt1.0 QRP 5 gt 50
• Self-regulation has serious shortcomings
• Better self-regulation would prevent FFP / QRP
• Research integrity can be improved (considerably)

20
Challenge demonstrating relevance

• 2006 Study (NEJM)
• Early detection study
• CT scan vs. X-ray
• With CT scan, 80 of cases could be
  cured
• March 28, 2008 NY Times
• Did not report study partly funded by tobacco
  company
• Vector Group (Liggett) gtgt Foundation for
  Lung Cancer
• Foundation listed in NEJM article, along with 31
  additional funding sources
• Did not report held patents on CT-scan-related
  technology
• Impact/profit 48M former smokers / 40M current
  smokers (US)
• Others involved as members of Foundation for
  Early Detection
• Dean Vice-Chair Board of Overseers, Weill
  Cornell Medical College
• Controversial cases get attention

see also Paul Goldberg, Cancer Letter Inc. (2008)
21
Comparison to careful RRI study

• Gorman, Evaluation Program Planning (2007)
• study of National Registry of Effective
  Promising Programs for drug and alcohol use
  prevention
• 78 of published evaluations had developer as an
  author
• Developers had a significant financial interest
  in outcomes
• Evaluation in one program had serious
  methodological flaws
• Texas spent 35M on one questioned program in
  2007
• Why is this story not national headline news?
• COI gt wasted funds and probably loss of life
• Conclusion future of RRI depends on relevance
• Need more studies of impacts (economic / human)
• RRI community must become proactive

22

• Thanks
• on with the conference

• nsteneck_at_umich.edu