Getting Ahead of the Course Evaluation Curve

1
Getting Ahead of the Course Evaluation
Curve Using student course evaluations to learn
about courses, curriculum, and students
with CoursEval3
Brian R. Hopewell Director of Business
Development bhopewell_at_academicmanagement.com Acad
emic Management Systems www.academicmanagement.com
CALI, Las Vegas, 18 June 2007
2
Course Evaluations Context is Key

• Everybodys doing it.
• Summative (rank, salary, tenure)
• Formative (teaching improvement)
• And writing about it.
• Three thousand articles counting.
• Three million Google citations (Seldin, 2006)
• And arguing about it

3
Controversy

• What do students know about teaching and
  learning?
• How (and when, and by whom) will the information
  be used?
• What else is important to consider when
  evaluating teaching?

4
Some Agreement on Fundamentals

• Students do know and are able to report on
  certain elements of teaching with validity and
  reliability.
• Evaluations do not depend unduly on teacher
  popularity
• Evaluations are stable across time and ratings of
  same teachers by other students.
• Evaluations are not dependent on the grades
  students get or expect to get.
• Evaluations are not dependent on extraneous
  variables (time of day, gender, race, teacher
  status, etc.)

5
How (and when, and by whom) will the evaluations
be used?

• Best practice is to keep summative and formative
  evaluations separate.
• Strict attention to evaluation privacy and
  anonymity is crucial.
• Student evaluations are only one piece of the
  puzzle. Serious danger of making too much of too
  little.

6
On-line v. Not on-line?

• Precipitating factors
• New Dean/Provost on a mission
• Accreditation self-study or imminent team visit
• Mechanical, financial, personnel problems with
  paper-based system
• Student request
• The Big Question
• What will this do to our response rate?

7
Response Rate

• Moving On-line works best with a well-prepared
  roll-out strategy
• Development of a suitable instrument
• Faculty buy-in
• Active administrative support
• Personnel and resources
• Sticks and carrots
• Captive audiences do not guarantee either quality
  or quantity responses.

8
CoursEval3

• Academic Management Systems started at University
  of Buffalo
• CE3 designed with X values in mind
• Security and anonymity of data
• Flexibility of use for students, faculty admin.
• Scalable
• Multiple survey formats formative, summative,
  peer and self-assessments non-course surveys
• Timely results, understandable reports

9
MyCoursEval Remote Portal Plug-InBlackboard,
WebCT, Moodle, Sakai and other campus portal
systems
10
Peter Gold, College of Arts and Sciences 1-12-07
11
Email reminders
12
(No Transcript)
13
Peter Gold, College of Arts and Sciences 1-12-07
14
Report Customization
15
Response report
16
Participant report
17
Export results
18
If interested in exploring

• Online webinar with admin, faculty, IT.
• Prospective client site
• No-cost Pilot in fall or spring
• Price quote based on FTE.

19
References Resouces

• Cohen, P.,1980, Student ratings of instruction
  and student achievement a metaanalysis of
  multisection validity studies, Review of
  Education Research, v 51, pp. 281-309
• Seldin, P., ed., 2006 Evaluating Faculty
  Performance, Anker,
• Theall, M. and Franklin J. (eds.) 1990, Student
  Ratings of Instruction Issues for Improving
  Practice, Jossey-Bass,
• Brigham Young University Trav Johnson
  http//onset.byu.edu/onset/index.php?titleMain_Pa
  ge