Using Studentcollected Encounter Data for Course Evaluation and Student Education

1
Using Student-collected Encounter Data for Course
Evaluation and Student Education

• Curtis Stine, MD
• Nancy Clark, MEd
• David Steele, PhD
• Florida State University College of Medicine

2
Outline

• The Clinical Data Collection System
• Nancy Clark
• Using the Data for Program Evaluation and
  Planning
• Dave Steele
• Educational Uses of Data
• Curt Stine

3
Handouts

• Problems and Procedures List
• Student Data
• PowerPoint and handouts will be posted at FMDRL
  at the end of this conference
• http//www.med.fsu.edu/informatics/workshops.asp

4
The FSU COM CDCS Patient Encounter System

• Nancy Clark, M.Ed.
• Director, Medical Informatics Education

5
Arcstream/Avantgo/SQL System

• RFP in Spring 2002
• Implementation CDCS Fall 2002
• Revised continuously since
• Added Evaluation Module Summer 2003
• Support
• 1 FTE programmer in IT
• 2 FTE coordinators in OME

6
Clinical Data Collection System

• Data Entry via PDA or Web
• Enter patients on Axim (PDA) most efficient
• Use CDCS website to
• Edit patients
• Add notes
• Look at cumulative reports
• Evaluate faculty, clerkship, etc.

7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
Demographics
12
Problem
13
Level of Care
Minimal 10 minutes or less contact (review xray,
look at lesion, etc.)
Moderate Everything else
Full focused H P, participate in disposition
(scrub on patient, discuss treatment plan, write
orders, and discuss with attending)
14
Procedures
15
PDA Version
16
(No Transcript)
17
68
?
Hispanic
18
(No Transcript)
19
Sync PDA

• Copies all new encounters over web
• Into SQL Server
• Adds new encounters to list on PDA and web

20
Student Reporting Tool
21
Faculty Reporting Tool
22
Using the Clinical Data Collection System for
Program Evaluation and Planning

• David J. Steele, PhD
• Assistant Dean for Curriculum and Evaluation
  Director, Office of Medical Education

23
Major Uses Now

• Documenting attainment of clinical experience
  objectives (LCME ED-2)
• Assuring the comparability of educational
  experiences (LCME ED-8)
• Documenting and describing clinical experience
• Quality improvement (a component of all)

24
Establishing quantified criteria for types of
patients and level of student responsibility
(LCME ED-2)

• Benchmark Top 23 conditions / diagnoses seen by
  Family Physicians (JFP 1998 pp 377-389)
• Comparison of benchmark conditions with 1 year
  of experience as measured by CDCS
• Minimum number of encounters required to
  demonstrate familiarity with presentation, hx and
  PE, evidenced-based testing, diagnosis, and
  treatment (aka educated guess!) (Range 1 4
  encounters)

25
Comparability Monitoring

• There must be comparable educational
  experiencesacross all alternate instructional
  sites within a given discipline (LCME ED-8)
• Level of participation
• Clinical experience (problems, diagnoses,
  clinical activities)
• Demographics (age, gender, ethnicity)

26
Example Comparability of Educational Experience
(Participation) Family Medicine Clerkship
27
Example Comparability of Clinical Experience by
Category (Organ System or Clinical Activity)
28
Comparing Student Experiences Across Clerkship
Disciplines (Top 10)
29
Comparing Experiences Across Clerkships

• Shared problems
• Hypertension
• DMII
• Hyperlipidemia
• Back pain
• Chest pain
• Depression
• Chest pain
• Problems unique to FM
• Routine PE
• Cold, URI
• Sinusitis
• Problems unique to IM
• Pneumonia
• COPD
• CHF
• CAD
• Health Maintenance

30
Monitoring Student Experience Procedures--FM
31
Top 10 Procedures IM Clerkship
32
Major Uses Future

• Curriculum planning and program development
• Opportunities for integrated learning?
• Value added curriculum development?
• Educational research
• What is the relationship between student clinical
  experience and performance?
• How does experience influence career choice?
• How does clinical experience influence attitudes
  and values?

33
Educational Uses of Student-Entered Encounter Data

• Curtis Stine, MD
• The Florida State University COM
• STFM Annual Meeting, 2005

34
Methodology

• Students record patient-encounter data during the
  first 5 weeks of a 6-week Family Medicine
  Clerkship.
• Data entry is monitored and written feedback
  provided to the students at the end of the second
  and fourth weeks.
• Final reports for each student are printed out
  during week 6, and reviewed by Clerkship
  Director.
• These final reports are distributed and discussed
  with students at the Clerkship debriefing on the
  final day of the Clerkship.

35
Report Review Gender

• Gender distribution of patients seen?
• Common gender distribution 60/40
• If variance, consider explanatory variables
• Faculty gender?
• Other providers in practice?
• OB services provided?
• Student gender?
• Other variables?
• Potential impact of gender on health problems
  seen in the practice?

36
Report Review Ethnicity

• Ethnic distribution of patients seen?
• Ethnic distribution highly dependent on
  explanatory variables
• Faculty ethnicity?
• Other providers in practice?
• Location of practice? Urban, suburban, rural? If
  urban, what neighborhood?
• Other variables to explain ethnic distribution?
• Potential impact of ethnicity on health problems
  seen in the practice?

37
Report Review Age of patients

• Age distribution of patients seen?
• Age distribution dependent on explanatory
  variables
• OB services by faculty?
• Age of supervising faculty?
• Location of practice? Urban, suburban, rural?
• Other variables to explain age distribution?
• Potential impact of age on health problems seen
  in the practice?

38
Report Review Setting of Care

• Clerkship objectives/competencies focus on
  ambulatory care.
• Potential impact of setting of care on health
  problems seen in the practice?

39
Report Review Diseases and Conditions

• Disease/Condition categories discussed
• Health Maintenance
• Most common acute diseases/conditions
• Most common chronic diseases/conditions
• Most common undifferentiated symptom
• Most common emotional problems/mental health
  conditions and
• Miscellaneous other common diseases/conditions.

40
Report Review Health Maintenance

• Multiple coding possibilities
• Physical Exam, Routine
• Health Maintenance
• Well Child Care
• Question Other than a visit scheduled
  especially for the purpose of health maintenance,
  did you see the family physician deliver health
  maintenance services by any other mechanism?

41
Report Review Most Common Acute
Diseases/Conditions

• Acute infections
• Respiratory URI, Sinusitis, Pharyngitis
  (Influenza).
• Gastrointestinal (Gastroenteritis)
• Urinary UTI
• Skin Impetigo, Cellulitis, Abscesses
• Injuries/trauma
• Musculoskeletal

42
Report Review Most Common Chronic
Diseases/Conditions

• Hypertension
• Diabetes Mellitus, Type II
• Hyperlipidemia
• Depression
• Low Back Pain
• Other diseases of aging, especially, end-organ
  disease from high BP, BG or lipids
• What Chronic Diseases/Conditions often dont
  appear on the students list? Why not?

43
Report Review Most Common Undifferentiated
Conditions

• Abdominal Pain
• Chest Pain
• Headache
• Dizziness
• Fatigue
• Dealing with undifferentiated conditions
• Recognizing diseases/conditions that acutely
  threaten organ system or even, life
• Dealing with uncertainty
• Use of time as a diagnostic tool and,
• Use of empiric therapy as diagnostic tool

44
Report Review Most Common Emotional
Problems/Mental Health Conditions

• Big 3 in Family Medicine
• Mood disorders
• Anxiety disorders and,
• Substance Use disorders
• Management approaches you witnessed?

45
Report Review Miscellaneous common
diseases/conditions.

• Number of patients with musculoskeletal
  diseases/conditions.
• Number of patients with dermatologic
  diseases/conditions.
• Old diseases/conditions arent coded as often
  today?
• New diseases/conditions that werent commonly
  recognized/coded yesterday?

46
Report Review Miscellaneous common
diseases/conditions.

• Question Up to now, weve been concentrating on
  the common diseases/conditions you saw. What was
  the most unusual patient problem you saw?
• One of the challenges of being a family physician
  is to be able to recognize the zebra in the
  herd of horses.

47
Summary

• Common uses of student-entered encounter data
• course evaluation,
• site evaluation, and
• confirming the comparability of student
  educational experiences.
• Educational uses of data a reflective
  educational activity for students.
• review, summarize, clarify and reinforce
  clerkship objectives,
• assess clerkship competencies, and
• validate student perceptions about family
  medicine.