Student assessment: lightening the load while increasing the learning

1
Student assessment lightening the load while
increasing the learning

• Dr Chris Rust
• Head, Oxford Centre for Staff and Learning
  Development
• Deputy Director, Assessment Standards Knowledge
  Exchange (ASKe)
• Centre for Excellence in Teaching and Learning
• Oxford Brookes University, UK

2
3 possible ways of reducing staff assessment load

• Involve the students
• Mechanise
• Strategic programme decisions

3
Self-assessment

• Strengths of this piece of work
• Weaknesses in this piece of work
• How this work could be improved
• The grade it deserves is..
• What I would like your comments on

4
Marking exercise

• Immediate results participants av. mk non
  participants av. mk.
• Cohort 1 (99/00) 59.78 54.12
• Cohort 2 (00/01) 59.86 52.86
• Cohort 3 (01/02 55.7 49.7
• Results 1 year later
• Cohort 1 57.91 51.3
• Cohort 2 56.4 51.7
• Rust, C., Price, M ODonovan, B.(2003)
  "Improving students learning by
• developing their understanding of assessment
  criteria and processes
• Assessment and Evaluation in Higher Education,
  Vol. 28, No. 2

5
Peer marking using model answers (Forbes
Spence, 1991)

• Scenario
• Engineering students had weekly maths problem
  sheets marked and problem classes
• Increased student numbers meant marking
  impossible and problem classes big enough to hide
  in
• Students stopped doing problems
• Exam marks declined (Average 55gt45)
• Solution
• Course requirement to complete 50 problem sheets
• Peer assessed at six lecture sessions but marks
  do not count
• Exams and teaching unchanged
• Outcome Exam marks increased (Av. 45gt80)

6
Peer feedback (Rust, 2001)

• Scenario
• Geography students did two essays but no apparent
  improvement from one to the other despite lots of
  tutor time writing feedback
• Increased student numbers made tutor workload
  impossible
• Solution
• Only one essay but first draft required part way
  through course
• Students read and give each other feedback on
  their draft essays
• Students rewrite the essay in the light of the
  feedback
• In addition to the final draft, students also
  submit a summary of how the 2nd draft has been
  altered from the1st in the light of the feedback
• Outcome Much better essays

7
Mechanise assessment

• Statement banks
• Assignment attachment sheets
• Optical mark reader
• Computer aided-assessment

8
Mechanise assessment - 1 Statement Banks

• Write out frequently used feedback comments,
• for example
• I like this sentence/section because it is clear
  and concise
• I found this paragraph/section/essay well
  organised and easy to follow
• I am afraid I am lost. This paragraph/section is
  unclear and leaves me confused as to what you
  mean
• I would understand and be more convinced if you
  gave an example/quote/statistic to support this
• It would really help if you presented this data
  in a table
• This is an important point and you make it well
• etc.

9
Weekly CAA testing - case study data
(Brown, Rust Gibbs,1994)
10
CAA quizzes (Catley, 2005)

• Scenario
• First term, first year compulsory law module
• A new subject for most (75) students
• High failure rate (25), poor general results
  (28 3rd class, 7 Ist)
• Solution
• Weekly optional WebCT quizzes (50 take-up)
• Outcome
• Quiz takers 4 fail, 14 3rd class, 24 Ist
• Non-quiz takers same pattern as before
• Overall 14 fail (approx half previous figure)
• 21 3rd class
• 14 1st (double previous figure)

11
Strategic reduction 1

• Need audits
• - Of assessment methods
• - Of learning outcomes
• Need Departmental learning, teaching
• and assessment strategies

12
Strategic reduction 2

• Reduce time spent on assessment
• reduce repetition (eg lab reports)
• more variety (eg less essays)
• identify different course goals
• ? pass/fail (at least in 1st year)
• just grades, no marks
• assess selectively

13
Assessing a selection (Rust, 2001)

• Scenario
• Weekly lab reports submitted for marking
• Increased student numbers meant heavy staff
  workload and increasingly lengthy gap before
  returned so feedback of limited/no use
• Solution
• Weekly lab reports still submitted
• Sample number looked at, and generic feedback
  e-mailed to all students within 48 hours
• At end of semester, only three weeks lab reports
  selected for summative marking
• Outcome
• Better lab reports and significantly less marking

14
Strategic reduction 3

• Reduce time spent on feedback
• give general feedback / whole class
• use feedback forms
• use students
• use statement banks
• use computer-based tests
• give focused feedback (one criterion)
• give audio-recorded feedback