Student Diaries

1
'So how did I do? Using the right Evaluation
Tools to listen to our students Comparing four
alternative evaluation tools with the standard
end-of-module questionnaire by Mark Huxham,
Phyllis Laybourn, Morag Gray, Norrie Brown
(HLSS), Sandra Cairncross (FECCI), Judy Goldfinch
(NUBS), Shirley Earl (EdDev) For more detail, see
full article in Assessment Evaluation in HE
2008 vol 1 pp1-12

• Research says .
• both students staff are cynical about the
  effectiveness of end-of-module feedback (Penny
  2003)
• students often complete questionnaires
  superficially and give different responses if
  given greater opportunity to reflect (Robertson
  2004)

• H-forms
• Completed by groups of 4 to 5
• Individuals score module and agree comments
• Scores averaged
• Individual suggestions added and voted for
• Facilitator unknown to students

• What did we do?
• Evaluated four different ways of collecting
  student feedback
• For 12 modules over 7 schools involving 495
  students, part of the class completed the usual
  closed question end-of-module questionnaire and
  part completed one of
• Rapid feedback sheets
• H-forms
• Focus groups
• Student diaries

• Rapid feedback sheets
• Completed by individuals
• Anonymous
• Took 5 minutes of student time and under 1 hour
  of staff time

How did the Information from the alternatives
compare with the questionnaire results for the
same module?

• Focus Groups
• Completed by groups of 5 volunteers with an
  interviewer and recorded
• Interviewer unknown to students

• Questionnaires
• Standardised
• Very general
• No indication of importance of issues

• Other Methods
• Detailed comments
• Very specific issues mentioned
• Wider range of issues including the social
  climate, the timing and length of classes
• Indicated importance of issues to students
• Only about 1/3 of the issues raised corresponded
  with questionnaire questions and the ranking of
  these was usually different

• Student Diaries
• Completed by individual volunteers
• Reflection after each class,
• E-mailed to confidential researcher every 4 weeks
• Prompted if late
• Too time-consuming for students and only 3 out of
  18 completed

The more positive the scores on the questionnaire
for a module, the more positive the H-form
scores and the more positive the general
impression at the interviews.
What did students think of the other methods?

• Majority preferred the alternative methods
• Felt they allowed greater time and thought to go
  into the feedback
• Felt greater ownership
• Sceptical about reliability of the questionnaire
• These ideas are how we feel answering the
  questionnaire is very impersonal
• I dont put any thought into the questionnaire

June 2008