Promoting the RCR Video Case Studies: Resources and Evaluation

1
Promoting the RCRVideo Case Studies Resources
and Evaluation

• Paul Braunschweiger, Ph.D.
• Professor, Radiation Oncology
• CITI Co-Founder

Council of Graduate Schools Graduate Education
in a Global Context , 12-3-08
2
Disclaimer and Disclosures

• Dan Vasgird Ph. D. UN-L.
• CITI Developer Group
• RCR working group.
• 2006 CGS Grant to develop video vignettes
  (trigger cases) for RCR education.
• 2007 Cases added to CITI RCR Program.
• Disclaimer and Disclosure.

3
Talking Points

• RCR education paradigms.
• University of Nebraska RCR videos
• Application of RCR videos in the CITI Program.
• Distribution of the RCR videos.
• CITI Program.
• Global Context.
• Evaluation of the RCR video resources.
• Learner Feedback
• Resources

4
Integrity in The Research Enterprise

• Starts with Institutional Commitment
• Promoted and nurtured by the IRB, CoIC, IBC,
  IACUC, Sponsored Programs, Research
  Administrators.
• Embraced by the investigators, staff and students
  as just
• The right thing to do

"Integrity" by Canneto, Columbus, OH
5
Why Does Integrity and Professionalism Matter?

• Risks to subjects, Risks to society.
• Federal inquiry / intervention/ audit /
  suspensions.
• Wasted resources
• Institutional embarrassment
• Personal embarrassment
• Loss of funding, Loss of livelihood
• Legal actions, , Jail
• Undermines the Public Trust
• Research becomes politicized.
• CoI and Pharmaceutical Industry
• Justification and / or vindication for groups
  with anti-research agenda. (PETA, ALF)

6
The Foundations for Trust are Based in the
Documentation of Ethical Behavior and Education.
7
Mandated Ethics Training

• Human Subjects Protections.
• Basic for investigators, staff and students.
• GCP
• IRB Professional
• Lab Animal Welfare
• Basics for Investigators, staff and students
• IACUC Member
• Model / species specific / procedure specific
• HIPPA (Privacy and Security)
• Investigators
• Administrators
• Biological Safety and Security
• Basics for Investigators, staff and students
• The Responsible Conduct of Research
• Basics for investigators, staff and students
• Administrators.
• Grants and Contracts, Sponsored Programs
• Grants management
• Financial responsibility

8
Institutional Approaches Research Ethics Education

• Bottom up Approach
• Research Ethics is integrated in the academic and
  professional develop of students and fellows by
  the research faculty.
• Top down Federal /institutional mandate.
• Animal Welfare
• Human Subjects Protection
• HIPAA

9
Delivering the Message

• 11 training
• Effective, labor and time intensive, costly
• Small group training
• Effective, labor intensive, less time intensive,
  costly,
• Subcontracted
• Large group training,
• less time intensive, Less effective, very costly,
  may not be convenient.
• External consultants
• Can be effective, very costly.
• Web based training.
• Convenient, effective, inexpensive, timely,
  assessment.
• Provides knowledge base for advanced training.
• Lacks direct interaction with instructor, most
  programs are very textual, problems with
  presentation, formatting, ISPs and the internet.
  15 would prefer something else.
• Combined programmatic approach

10
Web based training

• Assuming a similar time commitment, would you
  prefer another vehicle for this mandated
  instruction?
• HSR Lab Animal Welfare
  RCR

CITI Learner Satisfaction Survey results 1-1-08
11
Video Presentation Paradigms

• Virtual Video Libraries
• Reference Resource
• UNL, SU, ORI, AAAS etc.
• Optional exercises in a web based tutorial.
• NIU
• Query-Video Presentation-Query (QVQ)
• UAB
• 2-3 minute video clips of exemplars providing
  expert opinions on various aspects of the RCR.
• Columbia University
• Cases and videos embedded in the RCR foundation
  text.
• CITI Program

12
University of Nebraska RCRvideo vignettes

• CGS Funded Project
• Grad. Faculty and Students
• 3 topics of greatest interest.
• Student actors
• Primary target at UN-L about 100 grad
  students/year.
• Live and online evaluations
• http//www.unl.edu/gradstudies/facstaff/rcr/

13
http//www.unl.edu/gradstudies/facstaff/rcr/
14
Integration of the UN-L RCR Vids into the CITI
RCR Courses.
15
CITI Program RCR Courses
Biomedical Sciences
Social Behavioral Sciences
Physical Sciences
Arts Humanities
Administrators
Supported by a resource development contract from
DHHS / ORI
16
The Responsible Conduct of Research

• Research Misconduct
• Historical perspectives
• Whistle Blowing
• Data Acquisition and Management
• Responsible Authorship
• Responsible Peer Review
• Mentoring
• Conflicts of Interest
• Responsible Collaborative Research
• Human Subjects Protection
• Lab Animal Welfare

17
RCR Program Paradigm

• Public Access, subscriber access
• Customized courses.
• Text, cases, videos and quizzes
• Helpdesk (305 243-7970) for students and trainers
• Completion Certificates
• Institution specific training materials easily
  incorporated to the curriculum.
• Training data storage and retrieval system
• Voluntary Learner Satisfaction Survey

18

• Research Misconduct
• Discipline Specific Title
• Links
• Introductory Video Scenarios
• Exemplar video presentations
• Imbedded Case Studies
• Quiz

19
Who should take advantage of RCR education?I
believe that it is very important that the entire
research team and not just the PI, should have a
clear understanding of the Responsible Conduct of
Research / human subjects protections.
RCR Learner Satisfaction Survey
HSR Learner Satisfaction Survey
11 78
7 84
CITI Learner Satisfaction Survey results 1-1-08
20
What is The CITI Program www.citiprogram.org

• 8 web based courses
• Human subjects Protection
• Basic courses Biomedical
• Basic course SBR
• Refresher Courses - Biomedical and SBR
• Good Clinical Practice
• Lab Animal Welfare
• Health Information Privacy and Security
• The RCR
• 5 discipline specific course.
• RCR for engineers.
• RCR for the humanities.

21
CITI Support Staff
22
CITI Program Milestones

• 10-2000
• 10 Participating Institutions.
• 2006
• New SBR Course added.
• 2008
• 950 Participating Institutions
• 22,000 Learners per month. 40-60 students
• gt337,000 Learners since 1-1-2008.
• gt960,000 Learners have completed a course since
  9-1-2000.
• Global Context
• International Initiatives
• Multi-language Course Site. 11-2007
• Spanish
• Portuguese
• Chinese, Japanese, Russian, Mongolian, French,
  Thai

23
CITI Worldwide Footprint
World wide n 950 International n 47sites /
35countries
24
Pan American Bioethics InitiativeCITI and U of M
Ethics Programs

• Fogarty International / NIH Funded R25 grant
• P. Braunschweiger, K. Goodman and S. Litewka.
• Research Ethics Education and Training Program
  development in Latin America, Brazil, and the
  Caribbean.
• Decision markers, policy architects in
  government, academia and industry.
• Charge to the Fellows
• Develop robust research ethics education program
  for their organization, implement, promote.
• PABI Workshops, ethics committee capacity
  building, CITI Platform.
• Fellows recruit and mentor peers.

25
RCR Program Evaluation
Aristotle, this web based ethics education stuff
is really cool, but, what are your measurables
and deliverables.
From The School of Athens, Raphaelo. 1509
26
User Satisfaction SurveyProgram Evaluation

• Voluntary, Anonymous. IRB Approved.
• Topics addressed
• Demographic Information.
• Role, age, gender, country, language
• Usability
• Registration, navigation, quizzes, Internet,
  Presentation, Time
• Knowledge base.
• Confidence in the new knowledge base
• Willingness to apply the new knowledge
• Intent to promote the RCR
• Overall rating
• Other

27
Evaluation

• RCR Course Usage
• Case study usage
• Video vignette usage
• Learner feedback

28
RCR Course UsageCertificates Awarded
Year Year
2008 2007 Total
Public Access 2,290 2,600 4,890
Participating Institutions 14,726 3,669 18,395
Total 17,016 6,269 23,285
December 1, 2008
29
How Many Introductory Videos Did You Review
30
The 3 minute video case studies were
31
The Introductory video case studies were helpful
to frame the content before I started reading.
Responders, n818
32
After completing the CITI RCR Course, I have a
better understanding of the ethical issues
central to the research enterprise so that I can
conduct my work with the highest level of
integrity.
Strongly disagree
Strongly
agree
33
Now that I have completed the CITI RCR Course, I
am more confident in my ability to advise a
student or a colleague on an issue involving the
Responsible Conduct of Research.
12
66
Strongly disagree

Strongly agree
34
Having completed the CITI Course in the
Responsible Conduct of Research and gained a
better understanding of how damaging research
misconduct can be to students, to the
institutions and to other investigators, I am now
more likely to report a clear act of research
misconduct to an appropriate person at my
organization.
Strongly disagree

Strongly agree
35
New CITI RCR Videos

• Peer Reviewhttp//medweb02.med.miami.edu/CITIvide
  o/2peer.wmv
• Mentoringhttp//medweb02.med.miami.edu/CITIvideo/
  3mentor.wmv
• Data Managementhttp//medweb02.med.miami.edu/CITI
  video/4data.wmv
• Collaborative Researchhttp//medweb02.med.miami.e
  du/CITIvideo/1collab.wmv

36
Summary

• The foundation of research integrity is research
  ethics education.
• Early, often, multiple formats
• Video
• Evaluate your education programs. Are they
  working? How can you improve them?
• Listen to the feedback from your learners.

New Integrity by Artibella Avanti
37
Summary

• Web based educational approaches to ethics
  education
• Can increase the ethics knowledge base of a wide
  audience.
• Can prepare the learner for advanced instruction
  and debate.
• Can provide the opportunity to customize the
  message to multiple audiences.
• Should be a tool in a larger institutional RCR
  Program.
• Web based education is not for everyone.
  Alternatives.
• The Responsible Conduct of Research is beyond
  simply being compliant with Federal regulations
  and Institutional Policies
• It is just the right thing to do.

38
RCR Video Resources
39
RCR Video ResourcesORI / DHHS www.ori/dhhs.gov

• General
• 17 Video Vignettes on Research Ethics and
  Academic Integrity. Syracuse University
  http//gradschpdprograms.syr.edu/resources/videos.
  php
• Data acquisition and management
• NIU, http//www.niu.edu/rcrportal/datamanagement/d
  mmain.html
• CITI, www.citiprogram.org
• AAAS http//www.aaas.org/spp/video/thevideos.htm
• Conflicts of Interest
• Cleveland State University (flash)
• UN-L http//www.unl.edu/gradstudies/facstaff/rcr/
  
• AAAS http//www.aaas.org/spp/video/thevideos.htm
• Research Misconduct, Whistle Blowing
• UN-L http//www.unl.edu/gradstudies/facstaff/rcr/
  
• UAB, http//www.uab.edu/researchintegrityandimages
  /default.html
• CITI www.citiprogram.org
• AAAS http//www.aaas.org/spp/video/thevideos.htm

40
RCR Video ResourcesORI / DHHS www.ori/dhhs.gov

• Mentoring
• UAB, http//www.uab.edu/researchintegrityandimages
  /default.html
• CITI, www.citiprogram.org
• NIU Flash simulation http//www.niu.edu/rcrportal/
  mentoring/overview/overview.html
• Collaborative Research
• CITI www.citiprogram.org
• NIU (Flash) http//www.niu.edu/rcrportal/cfront.ht
  ml
• Authorship
• UAB, http//www.uab.edu/researchintegrityandimages
  /default.html
• UNL http//www.unl.edu/gradstudies/facstaff/rcr/
  
• CITI www.citiprogram.org
• AAAS http//www.aaas.org/spp/video/thevideos.htm
• Peer Review
• CITI www.citiprogram.org

41
RCR Video ResourcesORI / DHHS www.ori/dhhs.gov

• Lab Animal welfare
• AALAS, Wake Forest University, IACUC Inspection
  Virtual Walkthrough
• OLAW. http//grants.nih.gov/grants/olaw/TrainingVi
  deos.htm
• Nonhuman Primates, Laboratory Dog, Survival
  Rodent Surgery, Basic Biomethodology for
  Laboratory Mice.
• Human Subjects
• HSRA - Protecting Human Subjects Training.
• Childrens Hospital Boston, video
• Case studies http//ori.hhs.gov/education/products
  /ucla/default.htm
• Other
• Journal editors tool http//www.uab.edu/research
  integrityandimages/default.html
• The role of the RIO , Michigan State University
  http//ori.hhs.gov/rio/riovideo/
• RCREC http//rcrec.org/

42
RCR Web Resources

• Tutorials and courses

43
Human subjects research tutorials

• PRIMR - IRB 101/ 201 CD
• Dunn and Chadwick Book
• NIH/NCI new version
• CITI Program
• Biomedical
• Social and Behavioral

44
Good Clinical Practice

• ACRP
• SOCRA
• GeneEd / learner
• Infonetica GCP http//www.gcptraining.org.uk/
  /learner
• Multiple languages
• Kaplan /learner
• ClinfoSource https//www.clinfosource.com/index.as
  p / learner
• CITI /institution or organization

45
Lab Animal Welfare

• AALAS, LAWTE
• Basic courses investigators, staff, students
• IACUC members
• Species / model / procedure specific modules
• OLAW , Webinars
• IACUC Members
• IACUC Coordinators
• IO
• CITI
• Basic courses investigators, staff, students
• IACUC members
• Species / model / procedure specific modules

46
Health Insurance Portability and Accountability
Act (HIPAA)

• Investigator , Administrator
• http//myhipaatraining.com/
• www.HRClassRoom.com/HIPAA
• http//www.training-hipaa.net/
• Health Information Privacy and Security (HIPS)
• CITI Program, www.citiprogram.org

47
The Responsible Conduct of Research

• Biological Safety
• Basics for Investigators, staff and students
• UN Medical Center http//www.unmc.edu/dept/ibc/ind
  ex.cfm?L1_ID7CONREF11
• The Responsible Conduct of Research
• Basics for investigators, staff and students.
  (ORI funded)
• UC Davis http//ethics.ucsd.edu/courses/integrity/
  index.html
• Columbia University http//ccnmtl.columbia.edu/pro
  jects/rcr/
• NIU http//www.niu.edu/rcrportal/
• University of Montana http//www.umt.edu/research/
  ethics/research_ethics.html
• UNH http//www.unh.edu/rcr/index.html
• Boston College http//www.bc.edu/research/rcradmin
  /
• ORI website http//ori.hhs.gov/
• CITI RCR Program www.citiprogram.org

48
Research AdministrationProfessional Societies

• NCURA
• Sponsored Programs
• Grants management
• Financial Administration
• Contracts Management
• Clinical trials management

• SRAI
• Management Operations
• Research Integrity
• Financial Management
• Research Law
• Sponsored Prog. Administration
• Sponsors Agencies

Boston College / ORI Administrators and the RCR
www.CITIProgram.org
49
(No Transcript)
50
CITI Program Resources.

• Responsible Conduct of Research.
• Public Access at www.citiprogram.org

51
I plan to take a more active role in promoting
ethical research practices by
I intend to do little more than I am currently doing. 33
Attending more ethics related seminars 32
Becoming a mentor to a student or young faculty member 27
Encouraging my institution to adopt a formal RCR education requirement for students and faculty 16
Taking an advanced research ethics course 13
Adding a research ethics module to a course I teach 11
n3896
52
RCR for Engineers

• Coming Winter 2009

53
Adapted for CITI from materials developed by
Steve Erickson Ph.D. and Karen M.T. Muskavitch,
Ph.D Administrators and the Responsible Conduct
of Research funded by ORI/DHHS
http//www.bc.edu/research/rcradmin/default.html
54
CITI RCR Course Design

• Publication and Authorship
• Discipline Specific Title and hyperlinks
• Introductory Video Scenarios
• Imbedded Case Studies
• Quiz

55
CITI Program 11-2008
Participating Institutions and Organizations
(950)
CITI Developer Group (60)
Founders
RCR
CITI Executive Advisory Committee
L.A.W.
CITI Editorial Board (15)
Intl.
HSRP
GCP
HIPS
www.citiprogram.org
CITI Administration Office of Research
Education, University of Miami