Formative assessment: the research evidence

1
Formative assessmentthe research evidence

• Dylan Wiliam

2
Classroom assessment

• Five key strategies
• Questioning
• Feedback
• Sharing learning expectations
• Self-assessment
• Peer-assessment
• One big idea
• Using evidence about learning to adjust teaching

3
Kinds of questions Israel
Which fraction is the smallest?
Success rate 88
Which fraction is the largest?
Success rate 46 39 chose (b)
Vinner, PME conference, Lahti, Finland, 1997
4
(No Transcript)
5
Misconceptions
3a 24 a b 16
6
Molecular structure of water?
7
Perfect teaching?
2.3 x 10 2.30
8
Kinds of feedback Israel

• 264 low and high ability grade 6 students in 12
  classes in 4 schools analysis of 132 students at
  top and bottom of each class
• Same teaching, same aims, same teachers, same
  classwork
• Three kinds of feedback scores, comments,
  scorescomments

Feedback Gain Attitude scores none top ve
bottom -ve
comments 30 all ve
Butler(1988) Br. J. Educ. Psychol., 58 1-14
9
Responses
Feedback Gain Attitude scores none top ve
bottom -ve
comments 30 all ve
What do you think happened for the students given
both scores and comments A Gain 30 Attitude
all ve B Gain 30 Attitude top ve, bottom
-ve C Gain 0 Attitude all ve D Gain 0
Attitude top ve, bottom -ve E Something else
Butler(1988) Br. J. Educ. Psychol., 58 1-14
10
Kinds of feedback Israel (2)

• 200 grade 5 and 6 Israeli students
• Divergent thinking tasks
• 4 matched groups
• experimental group 1 (EG1) comments
• experimental group 2 (EG2) grades
• experimental group 3 (EG3) praise
• control group (CG) no feedback
• Achievement
• EG1gt(EG2EG3CG)
• Ego-involvement
• (EG2EG3)gt(EG1CG)

Butler (1987) J. Educ. Psychol. 79 474-482
11
Kinds of feedback Canada

• 80 Grade 8 Canadian students learning to write
  major scales in Music
• Experimental group 1 (EG1) given
• written praise
• list of weaknesses
• workplan
• Experimental group 2 (EG2) given
• oral feedback
• nature of errors
• chance to correct errors
• Control group (CG1) given
• no feedback
• Achievement EG2gt(EG1CG)

Boulet et al. (1990) J. Educational Research 84
119-125
12
Kinds of feedback

• Peekability (Simmonds Cope, 1993)
• Pairs of students, aged 9-11
• Angle and rotation problems
• class 1 worked on paper
• class 2 worked on a computer, using Logo
• Class 1 outperformed class 2
• Scaffolding (Day Cordón, 1993)
• 2 grade 3 classes
• class 1 given scaffolded response
• class 2 given solution when stuck
• Class 1 outperformed class 2

13
Effects of feedback

• Kluger DeNisi (1996)
• Review of 3000 research reports
• Excluding those
• without adequate controls
• with poor design
• with fewer than 10 participants
• where performance was not measured
• without details of effect sizes
• left 131 reports, 607 effect sizes, involving
  12652 individuals
• Average effect size 0.4, but
• Effect sizes very variable
• 40 of effect sizes were negative

14
Feedback

• Formative assessment requires
• data on the actual level of some measurable
  attribute
• data on the reference level of that attribute
• a mechanism for comparing the two levels and
  generating information about the gap between
  the two levels
• a mechanism by which the information can be used
  to alter the gap.
• Feedback is therefore formative only if the
  information fed back is actually used in closing
  the gap.

15
Formative assessment

• Frequent feedback is not necessarily formative
• Feedback that causes improvement is not
  necessarily formative
• Assessment is formative only if the information
  fed back to the learner is used by the learner in
  making improvements
• To be formative, assessment must include a recipe
  for future action

16
How do students make sense of feedback?

• Attribution (Dweck, 2000)
• Personalization (internal v external)
• Permanence (stable v unstable)
• Essential that students attribute both failures
  and success to internal, unstable causes(its
  down to you, and you can do something about it)
• Views of ability
• fixed (IQ)
• incremental (untapped potential)
• Essential that teachers inculcate in their
  students a view that ability is incremental
  rather than fixed(by working, youre getting
  smarter)

17
Sharing criteria with learners

• 3 teachers each teaching 4 grade 7 science
  classes in two US schools
• 14 week experiment
• 7 two-week projects, scored 2-10
• All teaching the same, except
• For a part of each week
• Two of each teachers classes discusses their
  likes and dislikes about the teaching (control)
• The other two classes discusses how their work
  will be assessed

Frederiksen White, AERA conference, Chicago,
1997
18
Sharing criteria with learners
19
Self-assessment Portugal

• Teachers studying for MA in Education
• Group 1 do regular programme
• Group 2 work on self-assessment for 2 terms (20
  weeks)
• Teachers matched in age, qualifications and
  experience using the same curriculum scheme for
  the same amount of time
• Pupils tested at beginning of year, and again
  after two terms
• Group 1 pupils improve by 7.8 points
• Group 2 pupils improve by 15

Fontana Fernandez, Br. J. Educ. Psychol. 64
407-417
20
Self-assessment
My red folder in the fourth year wants me to be
positive about my grade E in English History the
heritage and glory of the British Empire in my
own words.
Sibani Raychaudhuri English in Education, (1988)
22(3) 12
My red folder in the fourth year wants me to be
clear and positive about what I achieve in
school in my own words which are foreign to me
My red folder in the fourth year suddenly out of
nowhere wants me to assert what I achieve in
school in my own words. How can I blow the
trumpet theyve taken from me?
In my own words in my own language (which has no
place here) how can I feel clear and positive?
21
Putting It into Practice
22
Learning intention

• After this session you will be able to
• Explain the important features of each of the
  five key strategies
• Evaluate the potential of questions as
• a) discussion items
• b) diagnostic items

23
Practical strategies questioning

• Key idea questioning should
• cause thinking
• provide data that informs teaching
• Improving teacher questioning
• generating questions with colleagues
• closed v open
• low-order v high-order
• appropriate wait-time
• Getting away from I-R-E
• basketball rather than serial table-tennis
• No hands up (except to ask a question)
• class polls to review current attitudes towards
  an issue
• Hot Seat questioning
• All-student response systems
• ABCD cards, Mini white-boards, Exit passes

24
Questioning in math diagnosis

• If f  g 8, then f  g h  ?
• 9
• 12
• 15
• 16
• 8  h

25
Questioning in math discussion

• Which is larger 0.33 or 1/3?
• 0.33 is larger than 1/3
• 1/3 is larger than 0.33
• They are the same
• You need more information to be sure

26
Questioning in science diagnosis

• The ball sitting on the table is not moving. It
  is not moving because
• A) no forces are pushing or pulling on the ball.
  
  
• B) gravity is pulling down, but the table is in
  the way.
• C) the table pushes up with the same force that
  gravity pulls down
• D) gravity is holding it onto the table.
• E) there is a force inside the ball keeping it
  from rolling off the table

Wilson Draney, 2004
27
Questioning in science discussion

• Ice-cubes are added to a glass of water. What
  happens to the level of the water as the
  ice-cubes melt?
• A) The level of the water drops
• B) The level of the water stays the same
• C) The level of the water increases
• D) You need more information to be sure

28
Questioning in English diagnosis

• Which of these is a good thesis statement?
• A) There are 9 violent incidents on TV per hour
• B) There is a lot of violence on TV
• C) The amount of violence on TV should be reduced
• D) Some programs are more violent than others
• E) Violence is included in programs to boost
  ratings

29
Questioning in English discussion

• Macbeth mad or bad?

30
Questioning in History diagnosis

• Why are historians concerned with bias when
  analyzing sources?
• A) People can never be trusted to tell the truth
• B) People deliberately leave out important
  details
• C) People are only able to provide meaningful
  information if they experienced an event
  firsthand
• D) People interpret the same event in different
  ways, according to their experience
• E) People are unaware of the motivations for
  their actions
• F) People get confused about sequences of events

31
Questioning in History discussion

• In what year did World War II begin?
• A) 1919
• B) 1937
• C) 1938
• D) 1939
• E) 1941

32
Practical strategies feedback

• Key idea feedback should
• cause thinking
• provide guidance on how to improve
• Comment-only grading
• Focused grading
• Explicit reference to rubrics
• Suggestions on how to improve
• Strategy cards ideas for improvement
• Not giving complete solutions
• Re-timing assessment
• (eg two-thirds-of-the-way-through-a-unit test)

33
Practical strategies sharing learning intentions

• Explaining learning intentions at start of
  lesson/unit
• Learning intentions
• Success criteria
• Intentions/criteria in students language
• Posters of key words to talk about learning
• eg describe, explain, evaluate
• Planning/writing frames
• Annotated examples of different standards to
  flesh out assessment rubrics (e.g. lab reports)
• Opportunities for students to design their own
  tests

34
Practical strategiespeer and self-assessment

• Students assessing their own/peers work
• with scoring guides
• post-it notes
• with rubrics
• with exemplars
• two stars and a wish
• Training students to pose questions/Identifying
  group weaknesses
• Self-assessment of understanding
• Traffic lights
• Smiley faces
• Red/green discs
• End-of-lesson students review

35
Concept cards

• On the colored index cards, write a sentence or
  two or give an example to explain each of the
  following six ideas (if youre not sure, ask a
  question instead)
• Questioning yellow
• Feedback orange
• Sharing learning intentions green
• Self-assessment red
• Peer-assessment blue

36
Expertise
1 Experts excel mainly in their own domain 2
Experts often develop automaticity for the
repetitive operations that are needed to
accomplish their goals 3 Experts are more
sensitive to the task demands and social
situation when solving problems. 4 Experts are
more opportunistic and flexible in their teaching
than novices 5 Experts represent problems in
qualitatively different ways than novices. 6
Experts have fast and accurate pattern
recognition capabilities. Novices cannot always
make sense of what they experience. 7 Experts
perceive meaningful patterns in the domain in
which they are experienced. 8 Experts begin to
solve problems slower, but bring richer and more
personal sources of information to bear on the
problem that they are trying to solve.
Berliner, 1994
37
Countdown
3
25
1
4
9
Target number 127
38
Klein Klein (1981)
Six video extracts of a person delivering
cardio-pulmonary resuscitation (CPR) 5 of the
video extracts are students1 of the video
extracts is an expert Videos shown to three
groups Students, experts, instructors Success
rate in identifying expert Experts 90 Students
50 Instructors 30
39
Chess (Newell Simon, 1973)
40
Sensory capacity (Nørretranders, 1998)
41
Three questions

• What formative assessment strategies do you use
  already?
• What new ideas do you want to add to your
  practice?
• What will you do less of to make time?
• During lessons
• Outside of lessons

42
What can you do?

• Write a memo to yourself
• To be opened in the New Year
• Commit to making 2 or 3 changes
• Hold yourself accountable
• Set up a study group
• Set up peer observations

43
Learning log

• Please use at least three of the following
  sentences to share your thoughts on todays
  sessions. Please write your responses on the
  lined NCR paper and leave the white copy.
• Today I learned
• I was surprised by
• I was interested in
• What I liked most about today was
• One thing Im not sure about is
• After these sessions I feel
• I might have got more from today if
• The main thing I want to find out more about is
  
• The most useful thing I will take from these
  sessions is