Faculty Development Issues in Evaluation

1
Faculty Development Issues in Evaluation

• Definition
• Types of Programs
• Types of Evaluation Done
• Evidence of Success
• Believed Utility
• Reason for Continuation of programs
• Need / Evidence of Success
• Risks to current evaluation methods
• Impetus for future work

2
Faculty Development Definition

• activities designed to improve an individuals
  knowledge and skills in areas considered
  essential to the performance of a faculty member

McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC,
1997
3
Types of Faculty Development

• Instructional development
• Professional development
• Leadership development
• Organizational development

Irby, 1995, from Berquist and Phillips
4
Faculty Development Schemes for Program
Evaluation

• The Faculty
• Teacher Attitudes/Knowledge/Behaviors
• Pedagogy
• Pedagogical Content Knowledge
• Content Areas, e.g., End of Life, Geriatrics,
  Organizational Leadership, Professional Skills
• Their Students
• The System

5
Faculty Development

• Schemes for Program Evaluation
• Reaction (customer satisfaction)
• Faculty Learning (change in Knowledge, Skills,
  Attitudes)
• Behavior Change (result of program)
• Results (change in the system)

Blumberg, Deveau, Medical Teacher, 1995
6
Faculty Development

• Schemes for Program Evaluation
• Academic dissemination (presentations/publications
  )
• Product development (materials/workshops)
• Implementation ( s trained, maintained and
  disseminated)
• Institutionalization

Blumberg, Deveau, Medical Teacher, 1995
7
Faculty Development

• Schemes for Program Evaluation
• Process Evaluation (participant ratings)
• Content Evaluation (Change in knowledge, skills,
  attitudes)
• Outcome Evaluation (long term change)
• Impact Evaluation (Impact on the system)

Holly, 1992
8
Faculty Development Schemes for Program
Evaluation

• The Faculty
• Their Students
• The System

9
Faculty Development in CanadaEvaluation methods

• 3 outcomes
• performance
• cognitive learning
• self-assessment re teaching skills
• Outcome measures positive direction, but
  without statistical significance, but small N.

Nasmith, Steinert, Saroyan, Daile, Franco, TchLng
Med, 1997
10
Lessons from Faculty Development in Clinical
Teaching

• Research Evidence for Benefits
• Teachers and learners
• POSITIVE RATINGS OF EXPERIENCE
• IMPROVED EDUCATIONAL OUTCOMES FOR FACULTY -
  knowledge, skills, and attitudes
• SELF-RATINGS
• RATINGS OF BEHAVIORS ON VIDEOTAPES
• KNOWLEDGE ACQUISITION BY TEACHERS
• IMPROVED STUDENT RATINGS OF TEACHERS
• IMPROVED OUTCOMES FOR LEARNERS -- MOSTLY IN
  HIGHER ED LITERATURE
• Institutional / Societal Benefits ?

Stanford Faculty Development Program
Murray in Perry and Smart, 1997 Effective
Teaching in Higher Education
11
Faculty Development Outcomes

• Reviewed different types of programs in family
  medicine
• Fellowships
• Workshops
• Results
• Changes measured, but no significant performance
  changes
• Few publications
• Different needs for different faculty, tenure vs
  non-tenure

Reid, Stritter, Arndt, Fam Med, 1997
12
Faculty Development Schemes for Program
Evaluation

• The Faculty
• Their Students
• The System

13
AAMC - 2000

• Faculty Development /Faculty Affairs Offices 76
  of 125 institutions -gt 93 faculty affairs and 21
  development offices

Moahan, et al, Acad Med, May 2002
14
Faculty Development - Evaluation

• Faculty development offices focused on use of
  services and mentoring

Moahan, et al, Acad Med, May 2002
15
AAMC - 2000

• Faculty Development /Faculty Affairs Offices 76
  of 125 institutions -gt 93 faculty affairs and 21
  development offices

Moahan, et al, Acad Med, May 2002
16
Faculty Development in Canada

• 1986 positive commitment poor faculty
  participation
• Recommendations
• establish committees
• fund activities
• Recognize participation in promotion/tenure
• 1986-96 Major changes
• Health care financing
• Downsizing
• Inpatient -gt outpatient
• Multi-institutional and multidisiplinary care
• Curriculum reform
• Less stress on information acquisition
• Problem solving / interpersonal skills/attitudes

McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC,
1997
17
Faculty Development in Canada

• All 16 schools responded
• 16 with Faculty Development Program vs 12 in 1986
• Sabbatical programs
• Grants to improve instruction/ courses, with
  major emphasis on information technology/electroni
  c media
• Awards for teaching excellence
• Increasing feedback on teaching from faculty or
  administrators (13/16 schools)
• Systematic review of teaching by administrator- 6
  schools
• Senior faculty work with junior faculty 2
  schools
• Workshops on management / research / scholarship
• Fac Devel committees in 9 schools
• ½ of schools indicate inadequate budget
• Outcomes reported at 4 schools

Stanford Faculty Development Program
McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC,
1997
18
Faculty Development in Canada

• Most effective practices
• Teaching instruction
• Use of computers/information mngt
• Sabbatical leaves
• Teaching Awards
• Inexpensive faculty development
• Local faculty development stimulated by
  clinicians with M Ed Degrees

Stanford Faculty Development Program
McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC,
1997
19
Faculty Development in Canada

• Stimuli for change
• Curricular change, needing more tutors
• Curricular change Uncertainty in faculty
• Changes in health care examination of career
  and personal goals

McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC,
1997
20
Faculty Development in Canada

• Prognosis - - Good Collegiality and
  Institutional Support
• Problem lack of evaluation on impact
• Need to identify outcomes and methods

Stanford Faculty Development Program
McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC,
1997
21
Lessons from Faculty Development in Clinical
Teaching

• Research Commentary
• Hard to do controlled experiments
• Few measures of pre-post change
• Few measures of long-term change
• Mostly participant ratings
• Few measures of actual behavioral change in
  faculty
• Difficult for measures of changes in student
  learning
• Little research on institutional change

Stanford Faculty Development Center
22
Faculty Development Outcomes

• Are the data hidden in the process we are
  observing?
• Curricular Change?
• Emphasis on Teaching?
• New Teaching Methods?
• New Observations of Faculty Needs?
• Requirements for Teaching Improvement Methods by
  Accrediting Agencies?
• Areas for Further Work
• Further Enhancement of Pedagogy Improve the
  best
• Pedagogical Content Knowledge (Shulman)
• Needs of the Profession

23
Types of Faculty DevelopmentAreas for Further
Work

• Instructional development
• Professional development
• Leadership development
• Organizational development

Irby, 1995, from Berquist and Phillips
24
Types of Faculty DevelopmentYour Beliefs

• Instructional development
• Should be improved?
• Can be improved?
• Has been improved?
• Professional development
• Should be improved?
• Can be improved?
• Has been improved?
• Leadership development
• Should be improved?
• Can be improved?
• Has been improved?
• Organizational development
• Should be improved?
• Can be improved?
• Has been improved?

25
Lessons from Faculty Development in Clinical
Teaching

• Current Challenges/Opportunities
• INSTITUTIONAL
• Other Alligators
• Attitude - good enough mentality?
• TIME / MONEY/ ADMINISTRATIVE SUPPORT?
• EVIDENCE FOR NEED Present, ? compelling
• WHO IS GOING TO PAY?
• TEACHERS THEMSELVES
• Philosophical/practical/self-assessment/
• METHODS Applicabililty to current environment

Stanford Faculty Development Center
26
Areas for Increased Emphasis in Faculty
Development

• INSTITUTIONAL BENEFITS
• IDENTIFICATION OF NEW AREAS FOR IMPROVEMENT -
  Consolidation seminar - -gtgt
• STRUCTURAL RECOMMENDATIONS
• APPLICATION OF KNOWLEDGE RECOMMENDATIONS FOR
  INSTITUTIONS, E.G., MDM POLICIES, PATIENT CARE
• EDUCATIONAL RECOMMENDATIONS, E.G. TEACHING
  RESOURCES, ADMINISTRATIVE RESOURCES
• SOCIETAL BENEFITS
• IMPROVED OUTCOME FOR LEARNERS --gtgt FUTURE
  PHYSICIANS/SCIENTISTS

Stanford Faculty Development Center
27
Lessons from Faculty Development in Clinical
Teaching Data vs Belief

• Importance of Faculty Development
• Needs of Educators -
  Academic / Community
• Topics of interest and importance (Content and
  process)
• Benefits of Development Methods
• Current Challenges and Opportunities

Stanford Faculty Development Program