Teacher Assessment versus TestsExams

1
Teacher Assessment versus Tests/Exams

• Peter Tymms
• CEM, Durham University

2
Overview

• The Issue
• Fairness and bias
• Teacher assessment
• Conclusions

3
The Issue

• Teacher assessment is unfair because it is
  unreliable and biased.
• Test are simply snapshots and are
  unrepresentative of the work that has really be
  done

4
What is bias

• Bias appears in a test when part of an assessment
  is harder for a particular group.
• Or when an assessor systematically downgrades a
  group or an individual for construct irrelevant
  reasons

5
Example of item bias
Pigeon Turtle
6
Examples of teacher bias

• Annecdote
• By Sex (eg baseline page 17 Harlen)
• By ability judgement anchored by experience
• By Ethnicity assault experiments
• By social class
• By behaviour (origin of ability testing. Binet)
• By Age (EPICure study)
• By incident eg spilling a glass of water.
• The halo (or horns) effect (EYFS Profile P
  scales)

7
Trusting teachers judgement Harlen 2005

• The findings of the review by no means constitute
  a ringing endorsement of teachers assessment
  there was evidence of low reliability and bias in
  teachers judgements

8
Is it OK for teachers to assess their own pupils
for High Stakes exams?

• How does the power to grade affect relationships?
• Would you give McEnroy a B?

9
Exam/test reliability

• Typically around 0.9 but
• Distinguish the assessment of
• Convergent questions
• Divergent questions

10
Are Exams inappropriate snapshots?

• Issue 1 Questions must be representative samples
  of the course under exam conditions.
• Issue 2 Constraint on the nature of the
  assessment
• Issue 3 Impact of stress on performance
• Positive Negative (links to introversion)

11
Introvert and Extrovert
Effort
Stimulus
12
NB We need to match format to content

• Some things must be assessed by judgement
• Social interactions
• Behaviour
• Quality of research
• Poetry
• Art
• Some things are best assessed left to tests
• Mental arithmetic
• Spelling
• Phonological awareness
• Diagnostic assessments (e.g. INCAS)

13
Conclusion Judgements or tests?

• Should we do both? (Profiles)
• But, how do we ensure that judgements and tests
  are independent?
• How can judgements be kept free from bias?
• Virtually impossible in high stakes tests
• Essential for formative work

14
No easy solutions
Thank you
15
References

• Cooper, B. (1998). Using Bernstein and Bourdieu
  to understand children's difficulties with
  "realistic" mathematics testing an exploratory
  study. Qualitative Studies in Education, II(4),
  511-532.
• Eysenck, H. J. (2006) The Biollogical Basis of
  Personaility.Transaction publishers
• Harlen, W. (2005). Trusting teachers' judgement
  research evidence of reliability and validity of
  teachers' assessment used for summative purposes.
  Research Papers in Education, 20(3), 245-270.
• Johnson, S., Hennessy, E., Smith, R., Trikic, R.,
  Wolke, D., Marlow, N. (2009). The EPICure
  Study Academic attainment and special
  educational needs in extremely preterm children
  at 11 years. London Nottingham/London/Warwick.
• Koretz, D., Stecher, B. M., Klein, S. P.
  McCaffrey, D. (1994) The Vermont Portfolio
  Assessment
• Program findings and implications, Educational
  Measurement Issues Practice, 13, 516.
• Tymms, P. (1997). Value-added Key Stage 1 to Key
  Stage 2. London School Curriculum and Assessment
  Authority.
• Tymms, P., Jones, P., Albone, S., Henderson, B.
  (2009). The first seven years at school.
  Educational Assessment and Evaluation
  Accountability, 21, 67-80.
• Tymms, P., Merrell, C., Heron, T., Jones, P.,
  Albone, S., Henderson, B. (2008). The
  importance of districts. School Effectiveness and
  School Improvement, 19(3), 261-274.
• Tymms, P., Merrell, C., Jones, P. (2004). Using
  baseline assessment data to make international
  comparisons. British Educational Research
  Journal, 30(5), 673-689.
• Teacher on GCSE paper