Implementing formative assessment with teacher learning communities

1
Implementing formative assessment with teacher
learning communities

• Dylan Wiliam
• Director, Learning and Teaching Research Center
  ETS
• Using data to improve instruction
• New strategies and tools
• Boulder, CO July 30-August 2, 2006

2
Overview of session

• Why raising achievement is important
• Why investing in teachers is the answer
• Why assessment for learning should be the focus
• How we can put this into practice

3
Raising achievement matters

• For individuals
• Increased lifetime salary
• Improved health
• For society
• Lower criminal justice costs
• Lower health-care costs
• Increased economic growth

4
Wheres the solution?

• Structure
• Small high schools
• K-8 schools
• Alignment
• Curriculum reform
• Textbook replacement
• Governance
• Charter schools
• Vouchers
• Technology

5
Its the classroom

• Variability at the classroom level is up to 4
  times greater than at school level
• Its not class size
• Its not the between-class grouping strategy
• Its not the within-class grouping strategy
• Its the teacher

6
Teacher quality

• A labor force issue with 2 solutions
• Replace existing teachers with better ones?
• No evidence that more pay brings in better
  teachers
• No evidence that there are better teachers out
  there deterred by certification requirements
• Improve the effectiveness of existing teachers
• The love the one youre with strategy
• It can be done
• We know how to do it, but at scale? Quickly?
  Sustainably?

7
Learning power environments

• Key concept
• Teachers do not create learning
• Learners create learning
• Teaching as engineering learning environments
• Key features
• Create student engagement (pedagogies of
  engagement)
• Well-regulated (pedagogies of contingency)

8
Why pedagogies of engagement?

• Intelligence is partly inherited
• So what?
• Intelligence is partly environmental
• Environment creates intelligence
• Intelligence creates environment
• Learning environments
• High cognitive demand
• Inclusive
• Obligatory

9
Why pedagogies of contingency?

• Several major reviews of the research
• Natriello (1987)
• Crooks (1988)
• Kluger DeNisi (1996)
• Black Wiliam (1998)
• Nyquist (2003)
• All find consistent, substantial effects

10
Data-push vs. decision-pull

• Data-Push
• Quality control at end of an instructional
  sequence
• Monitoring assessment
• Identifies that remediation is required, but not
  what
• Requires new routines to utilize the information
• Decision-Pull
• Starts with the decisions teacher make daily
• Supports teachers on-the-fly decisions

11
Types of formative assessment
12
A framework for analysis
13
Cost/effect comparisons
14
Aspects of formative assessment
15
Five key strategies

• Clarifying and understanding learning intentions
  and criteria for success
• Engineering effective classroom discussions,
  questions, and learning tasks that elicit
  evidence of learning
• Providing feedback that moves learners forward
• Activating students as instructional resources
  for each other
• Activating students as the owners of their own
  learning

16
and one big idea

• Use evidence about learning to adapt instruction
  to meet student needs

17
Keeping Learning on Track (KLT)

• A pilot guides a plane or boat toward its
  destination by planning a route, taking constant
  readings and making careful adjustments in
  response to wind, currents, weather, etc.
• A KLT teacher does the same
• Plans a carefully chosen (possibly
  differentiated) route ahead of time (in essence
  building the track)
• Takes readings along the way
• Changes course as conditions dictate

18
Techniques for embedding the strategies in
practice
19
Questioning

• What can we do to preserve the ozone layer?
• Reduce the amount of carbon dioxide produced by
  cars and factories
• Reduce the greenhouse effect
• Stop cutting down the rainforests
• Limit the numbers of cars that can be used when
  the level of ozone is high
• Properly dispose of air-conditioners and fridges

20
(No Transcript)
21
Practical techniques

• Feedback
• Not giving complete solutions
• Three-fourths-of-the-way-through-a-unit test
• Sharing learning intentions
• Annotated examples of different standards to
  flesh out assessment rubrics (e.g. lab reports)
• Opportunities for students to design their own
  tests
• Students as owners of their own learning
• Red/green discs
• Students as instructional resources for one
  another
• Pre-flight checklist

22
Force-field analysis

• What conditions, resources,requirements and goals
  in your school, district or state will constrain
  or support the use of short-cycle formative
  assessment/ assessment for learning?

23
Putting it into practice
24
The design challenge

• Key metric
• Cost of buying one standard deviation of
  increased student achievement
• Constraints
• Solution must be in principle scalable

25
Why research hasnt changed teaching

• What happens when learning takes place?
• depends on what it is that is being learned
• constructivist theories explain misconceptions
  better than associationist theories, but
• associationist theories explain acquisition of
  number facts better than constructivist theories
• each new theory provides better accounts of
  things the previous theory explained badly, but
• tends to explain poorly the things the previous
  theory explained well.
• Teacher expertise is multidimensional
• subject matter knowledge
• knowledge of schools as institutions
• knowledge of teacher practices
• How you change teaching depends on what you want
  to change

26
Changing teachers

• Improving subject-matter knowledge
• Direct instruction
• Improving summative assessment skills
• Expert-led workshops
• Improving minute-by-minute practice
• Teacher learning communities

27
Sensory capacity (Nørretranders, 1998)
28
Knowledge creation and transmission
After Nonaka Tageuchi, 1995
29
A model for teacher learning

• Content (what we want teachers to change)
• Evidence
• Ideas (strategies and techniques)
• Process (how to go about change)
• Choice
• Flexibility
• Small steps
• Accountability
• Support

30
Content strategies and techniques

• Distinction between strategies and techniques
• Strategies define the territory of AfL (no
  brainers)
• Teachers are responsible for choice of techniques
• Allows for customization/ caters for local
  context
• Creates ownership
• Shares responsibility
• Key requirements of techniques
• embodiment of deep cognitive/affective principles
  
• relevance
• feasibility
• acceptability

31
Design and intervention
Our design process
cognitive/affective insights
synergy/ comprehensiveness
set ofcomponents
Teachers implementation process
set of components
synergy/ comprehensiveness
cognitive/affective insights
32
Process Teacher Learning Communities

• Teacher as local expert
• Sustained over time
• Supportive forum for learning
• Embedded in day-to-day reality
• Domain-specific

33
A four-part model

• Initial workshop
• Monthly two-hour TLC meetings (16 in all)
• Collaboration time
• Peer observations
• Constructing questions
• Preparing work samples
• Collaborative marking
• Training for TLC leaders

34
Summary

• Raising achievement is important
• Raising achievement requires improving teacher
  quality
• Improving teacher quality requires teacher
  professional development
• To be effective, teacher professional development
  must address
• What teachers do in the classroom
• How teachers change what they do in the classroom
• AfL TLCs
• A point of (uniquely?) high leverage
• A Trojan Horse into wider issues of pedagogy,
  psychology, and curriculum

35
Questions?Comments?