Medical Education Research 101

1
Medical Education Research 101

• Darshana Shah PhDAssistant Dean Professional
  DevelopmentMarshall University Joan C. Edwards
  School of Medicine

2
Great Teachers Educational Research

• Ilene Harris, PhD University of Illinois-Chicago
• Larry D. Gruppen, PhD University of Michigan
  Medical School
• Carol Hodgson, PhD University of Colorado
• Sheila Chauvin, PhD LSU

3
Medical Education Research!!!!

• Why?

4
Medical Education Research

• Needs assessments
• Program development
• Curriculum evaluation
• Performance evaluation
• The act of extending or transforming knowledge
• Scholarly Activity

5
Glassick Criteria Teaching
0
5
6
Medical Education Research

• Qualitative Data Collection/ Analysis
• Quantitative Data Collection and Analysis
• Mixed Methods approaches

7
Qualitative Research Data

• What is qualitative data?
• Data in the form of words, rather than numbers,
  based on
• Asking open-ended questions in
• Interviews
• Groups
• Surveys
• Examination of documents
• Observation of situations and actions, recorded
  in field notes

8
Common Features of AnalyticMethods

• Coding Develop/affix CODES
• Reflection Note reflections in margins
• Pattern Identification Sort/sift to identify
  themes patterns
• Representation Display in tables, figures,
  narrative

9
Designing the Study

• Asking the question
• Collecting data
• Measuring the outcomes
• Presenting the results

10
Development ProcessFormulating Research Question

Find questions that you typically ask about
teaching Learning
Create list of resources matched to the questions
Create relevant links to literature
Connect questions to Literature Topics
Survey educational research literature
Pub MED ERIC (Educational information
Clearing house) Psych Literature
11
Collecting Data Types of Data

• Performance Assessments
• Examinations (in house, shelf exams, USMLE),
  OSCEs, SPs, chart reviewed
• Ratings
• Faculty/student course evaluations, evaluations,
  program evaluations
• Surveys
• student write-ups

12
What is a Survey?

• Information is collected from a group of people
  (e.g. students, faculty members)
• To determine some aspects or characteriristics of
  the population of which that group is part
• demographic characteristics,
• abilities
• opinions
• attitudes
• beliefs
• knowledge

13
Survey Issues

• Sampling Form and method of administration
• Direct
• Telephone
• Mail
• interview

14
Guidelines for Structured Questions

• Keep the question short
• Use common language
• Avoid use of terms that might bias response

15
Guidelines for Structured Questions

• Poor
• The new PBL curriculum is strongly supported by
  the Associate dean for curriculum. Rate the
  effectiveness of the curriculum for your learning
  of the basic science
• Better
• Rate the effectiveness of the new problem-based
  learning curriculum for your learning of the
  basic sciences

16
Guidelines for Structured Questions

• Question should be unambiguous
• N/A Poor Fair Good V. Good Excellent
• 1 2 3 4 5
• Poor
• Rate your interaction with the attending faculty
• N/A 1 2 3 4 5
• Better
• Rate the quality of feedback from attending
  physicians at the end of this clinical rotation
• N/A 1 2 3 4 5

17
Guidelines for Structured Questions

• Keep focus a simple as possible
• N/A Poor Fair Good V. Good Excellent
• 1 2 3 4 5
• Poor
• Rate quality of supervision by attending and
  residents, as well as tutor
• NA 1 2 3 4 5
• Better
• Rate quality of supervision by residents
• NA 1 2 3 4 5

18
Example

• Compare this ambulatory care rotation to other
  clinical rotations you have taken. What SPECIAL
  CONTRIBUTIONS, if any, did in make in preparing
  you to become a physician, e.g., knowledge,
  clinical competence, career development?

Ilene Harris 2004
19
Answers

• Comment 1 The independence allowed me to gain
  more confidence in my clinical skills. The
  hurried pace helped me to become more efficient.
• Comment 2 Outpatient management of disease is
  not really addressed in any other rotation.
  Also, the sheer number of patients seen really
  allows you to hone your H and P skills and gain
  confidence in patient interaction. This is also
  the only rotation where you see the
  administrative side of medicine.
• Comment 3 Caused me to like Family Medicine
  even better. Dramatically increased my
  confidence. Dispelled fears that ambulatory
  medicine might be boring. Gave me a good start
  at learning to prioritize in a clinic visit of a
  finite time period.

20
Qualitative Data Analysis
Data reduction Coding

• Data Display Table, figure, narrative

• Drawing conclusions

21
Coding Inductive From Data Set

• Comment 1 The independence allowed me to gain
  more confidence in my clinical skills. The
  hurried pace helped me to become more efficient.
  
• Comment 2 Outpatient management of disease is
  not really addressed in any other rotation.
  Also, the sheer number of patients seen really
  allows you to hone your H and P skills and gain
  confidence in patient interaction. This is also
  the only rotation where you see the
  administrative side of medicine.
• Comment 3 Caused me to like Family Medicine
  even better. Dramatically increased my
  confidence. Dispelled fears that ambulatory
  medicine might be boring. Gave me a good start
  at learning to prioritize in a clinic visit of a
  finite time period.

22
Example of coding - I

• CURRICULUM THEMES
• EDUCATION EXPERIENCE
• OUTCOMES

23
Comment 2 Outpatient management of disease is
not really addressed in any other rotation.
Also, the sheer number of patients seen really
allows you to hone your H and P skills and gain
confidence in patient interaction. This is also
the only rotation where you see the
administrative side of medicine.

• CURRICULUM THEMES
• CUM Outpatient management of disease not
  addressed in any other rotation
• EDUCATION EXPERIENCE
• EI Independence helps increase confidence in
  clinical skills
• EE Quick pace helps to increase efficiency
• EV High volume of patients contributes to
  learning
• EA Saw administrative side of medicine

24
Example of Coding II OUTCOMES

• Gained confidence in clinical skills
• Gained confidence in patient interaction
• Learned about administrative side of medicine
• Learned to become more efficient
• Learned to prioritize in a clinic visit
• Improved physical examination skills
• Improved history taking skills
• Found ambulatory medicine to be interesting
• Increased interest in family medicine

25
Example of Coding II OUTCOMES

• OCC Gained confidence in clinical skills
• OCPI Gained confidence in patient interaction
• OLA Learned about administrative side of
  medicine
• OE Learned to become more efficient
• OPr Learned to prioritize in a clinic visit
• OSPx Improved physical examination skills
• OSH Improved history taking skills
• OIAM Found ambulatory medicine to be interesting
  
• OIF Increased interest in family medicine

Skill, Confidence, career choice
26
Example of Coding - II

• ROTATION ATTRIBUTES CONTRIBUTING
• To Student Learning
• AP Quick pace
• AV High volume of patients
• AI Independent work
• Student Knowledge
• KA Clinic administration
• KO Outpatient management
• KN Nature of ambulatory medicine

27
Example of Coding - II

• STUDENT SKILLS
• SH Hone skills in H and P
• SP Learning to prioritize issues in clinic visit
  
• Student Attitude
• AC Increased confidence in patient interaction

28
Results display

• Bar chart
• Narrative

29
Trustworthiness of Qualitative Data Analysis
Reliability, Validity

• Peer Review (Audit Trail)
• Review of themes by peer(s) not involved in
  research
• Member Checking
• Sharing interpretations with individuals who were
  data sources
• Triangulation Comparing with data from other
  sources,
• Comparing focus group analysis with results from
  quantitative surveys
• Negative Case Analysis and Saturation
• Refining coding scheme until it encompasses all
  data points

30
Research in pathology education

• Integrating Basic Science and Clinical Medicine
  using a Heath Fair Format for Case-Based
  Teaching.

31
Qualitative Analysis used in Pathology

• Attitude and Views of Second Year Medical
  Students Towards Pathology Following a Pathology
  Laboratory Rotation

32
Attitude and Views of Second Year Medical
Students TowardsPathology Following a Pathology
Laboratory Rotation

• I learned that pathology as a career is not for
  me, but the surgical pathology experience has
  made me appreciate the work that the pathologists
  do.
• My perceptions of the pathologists role were
  all based on hearsay and television shows prior
  to starting medical school and the pathology
  course in particular. So before starting my
  surgical pathology rotation I was unsure of how a
  pathologist fit into the management of a
  patients health care.
• I thought that it was really interesting to go
  and see how pathologists are such an intricate
  part of the surgical process. It was a great
  learning experience. Now I understand how
  surgeons and pathologists are together able to
  try and isolate a cancer during surgery

33
Coding Theme

• Knowledge
• Appreciative of the discipline
• Career choice

34
Attitude and Views of Second Year Medical
Students Towards Pathology Following a Pathology
Laboratory Rotation
35
Any Money for educational research?

• Seed / innovation grants from your school or
  university
• Grants - community resources
• Non profit organization C-Change Cancer
  competency curriculum
• Large grants
• NBME Stemmler Medical Education Research Fund
• NSF
• FIPSE
• NIH K series for faculty development

36
Womens Health Fair Clinical case Breast Cancer
37
Survey Results
38
IRB Approval Is Required for Educational Research

• http//www.marshall.edu/research/ori/hrpp.asp

39
Present your work!!!!!

• Group on Educational Affairs (GEA)
• All AAMC member schools are affiliated with a GEA
  region. Regions host annual meetings, online
  forums and resources to support medical
  educators.
• Southern
• Central
• Northeast
• Western
• GRIPE
• IAMSE

40
Publish your work!!!!!

• Medical Education-related Journals
  (websites)http//www.academicmedicine.org/http/
  /www.iamse.org/http//meducational.com/journal

41
Some more tips before you get started!!!

• Consult with a colleague
• Describe your issue, question, hypothesis and why
  you think is important.
• Ask your colleague to challenge you to defend
  your choice!
• Does his/her feedback suggest new questions or
  new ways of conceptualizing your issue?

42
Literature Resource

• Medical Education Journals
• Academic Medicine
• Teaching and Learning in Medicine
• Advances in Health Sciences Education,
• Theory and Practice
• Medical Education
• Medical Teacher

43
Strategies

• Collaborate with colleagues
• Be creative
• Think of an outlet/audience for every type of
  data who else would be interested?
• Play on strengths Barter/share
• Think retrospective to prospective
• Find free help

44
References

• Research design Qualitative, Quantitative and
  Mixed Methods Approaches John W. Creswell
• Qualitative Methods Denzin, Norman and Lincoln,
  Yvonna. Handbook of Qualitative Research.
  Thousand Oakes Sage, 1994.
• Miles, Matthew and Huberman, A. Michael.
  Qualitative Data Analysis An Extended
  Sourcebook (2nd edition). Thousand Oakes Sage,
  1994.
• Strauss, Anselm and Corbin, Juliet. Basics of
  Qualitative Research Grounded Theory Procedures
  and Techniques. Newbury Park
• Sage, 1999 Sages Qualitative Research Methods
  series