Teaching Effectiveness

1
Teaching Effectiveness
Past, Present, Future
2
Tracy Chapman, M.Ed.Executive Director for
eLearning and TechnologySchool of Pharmacy and
Health Professionstchapman_at_creighton.edu
Ken Keefner, R.Ph., PhDAssociate ProfessorVice
ChairDepartment of Pharmacy SciencesSchool of
Pharmacy and Health Professionskenkeefner_at_creight
on.edu
3
Todays Session
Overview of the challenges we faced and prompts
to make changes Process employed to affect
change Current status and future plans Our
context
4

• I never teach my pupils I only attempt to
  provide the conditions in which they can learn.
• Albert Einstein

5
Challenges and Prompts

• Course evaluations as sole data source for
  dossiers to document teaching
• Faculty consistently received the poorest ratings
  (PT)
• No validity/reliability tied to tools used for
  course evaluations
• No consistency in tools/questions used
• Looking for a new tool for course evaluations
• Very little formative assessment
• Death by survey

6
Existing Structure
OLAT
7
Transforming the Culture
Change in Institutional Culture
Course Evaluation Teaching Effectiveness
Assessing Teaching Effectiveness
Build Shared Guide-lines
Build Shared Trust
Build Shared Motivation
Build Shared Language
Adapted from Angelo 1999
8
Process

• Stakeholder involvement
• External expertise
• Collaboration Communication
• Tenacity
• Someone leading the charge
• Worked at multiple levels

9
Process
School of Medicine
Office for Faculty Development Assessment
InternalConsultants
ExternalConsultants
IDEA
SGID Facilitators
Peer Coaches
10

• A change is something you do, and a fad is
  something people talk about.
• Peter DruckerManaging in the Next Society

11
What did we end up with?

• Process/Plan for using multiple data points
• Multiple Tools for data collection
• Agreement on data ownership
• Beginning to think about student learning in a
  more robust way
• Cross disciplinary examination/discussion on
  teaching effectiveness
• Decreased death by survey

12
Tools Developed/Adapted
Teaching Effectiveness Plan Teaching
Effectiveness Table what to use when and for
what purposes Annual Course Report form Small
Group Instructional Diagnosis (SGID) Peer
Mentoring Policies/Procedures Individual
Development Educational Assessment (IDEA)
13

• New tool opportunity for change
• Appropriate use of student ratings student
  comments
• Shared language

14
Shared Language

• Course evaluations ? Student ratings of
  instruction
• Teaching evaluation ? Teaching effectiveness
• Faculty focus ? Student (learning) focused

15
New Tool An Opportunity for Change

• Student Learning Focus
• Diagnostic Component
• Scores Adjusted for Extraneous Influences
• What was instructors influence on learning?
• Documented Validity and Reliability
• National Comparative Data
• Group Summary Reports
• Program Review
• Accreditation

16
Appropriate Use of Student Ratings Student
Comments
Student Ratings can have a positive impact
if... The instrument Is learning focused and
provides a diagnostic The emphasis for
summative faculty evaluation is
appropriate 30-50 of the overall evaluation of
teaching Use additional sources of
evidence Results are not over-interpreted 3-5
performance categories Appropriate use of
student comments Results are representative of
instruction 6 to 8 classes representing all or
nearly all classes taught More if classes are
small
17
Students Are Not Qualified to Assess

• Faculty expertise
• Appropriateness of goals, content, and
  organization of course
• Materials used in delivery
• How student work is evaluated, including grading
  practices

18
Transforming the Culture
Change in Institutional Culture
Course Evaluation Teaching Effectiveness
Assessing Teaching Effectiveness
Build Shared Guide-lines
Build Shared Trust
Build Shared Motivation
Build Shared Language
Adapted from Angelo 1999
19
References
Selected References Arreola R. Developing a
Comprehensive Faculty Evaluation System. Bolton,
MA Anker Pub, 2000. Hutchings P. Ethics of
Inquiry Issues in the Scholarship of Teaching
and Learning. Menlo Park, CA The Carnegie
Foundation for the Advancement of Teaching,
2002. McKeachie W. Teaching Tips. 11th Edition,
Boston, MA Houghton Mifflin Co, 2002. Penny A,
Coe R. Effectiveness of consultation on student
ratings feedback a meta-analysis. Review of
Educational Research, 74(2) 215-253,
2004. Seldin P. Changing Practices in
Evaluating Teaching, Bolton, MA Anker Pub,
1999. Shulman L. Teaching as Community Property
Essays on Higher Education, San Francisco, CA,
2004. Wergin J. Departments that Work Building
and Sustaining Cultures of Excellence in Academic
Programs. Bolton, MA Anker Pub, 2003.