Learning Progressions:

1
Learning Progressions Some Thoughts About What
we do With and About Them Jim Pellegrino Universi
ty of Illinois at Chicago
2
Why Learning Progressions?

• Learning is facilitated when new and existing
  knowledge is structured around big ideas or a
  conceptual framework rather than small, discrete
  bits of information.
• Learning develops as a continuous process with an
  individual continuously making links back and
  forth among ideas and not in linear, discrete
  steps.
• Learning difficult ideas takes time and often
  comes together as students work on a task that
  forces them to synthesize ideas.
• Yet, K 12 science curricula are generally not
  structured to build and cycle back on ideas.

3
Importance of Domain-Based Models
Assessment
Domain-Based Models of Learning Understanding
Curriculum
Instruction
4
Connecting Assessment toCurriculum Instruction
Observations
Interpretation
Assessment
Domain-Based Models of Learning Understanding
Curriculum
Instruction
5
Assessment as a Process of Reasoning from Evidence

• cognition
• model of how students represent knowledge
  develop competence in the domain
• observations
• tasks or situations that allow one to observe
  students performance
• interpretation
• method for making sense of the data

observation
interpretation
cognition
Must be coordinated!
6
Guiding Principles

• All models are wrong, some are useful
• George Box
• Better an approximate answer to the right
  question than an accurate answer to the wrong
  question
• John Tukey

7
Assessment Design Principles

• Assessment design should always be based upon a
  model of student learning and a clear sense of
  the inferences about student competence that are
  desired for the particular context of use.
• The model of student learning suggests the most
  important aspects of comptence that one would
  want to make inferences about and provides clues
  about the types of tasks that will elicit
  evidence to support those inferences.

8
Aspects of Student Models

• Domain specific and empirically based
• Identifies cognitive performances that
  differentiate expert and novice learners
• Lays out one or more typical progressions toward
  competence including milestones or landmark
  performances along the way.
• Can be at various levels of detail grain size
  depends on assessment purpose

9
Contrasting Complementary Perspectives on
Cognition
10
Assessment as a Process of Reasoning from Evidence
Observations
Interpretation Model
Assessment
Learning Progression
11
Interpretation -- Making Sense of the
Observational Data

• Some of the ways in which individuals typically
  try to make sense of the observational data
• Intuition -- gut reaction
• Counting number correct (4/8) generating percent
  correct
• Assigning points to answers or assigning letter
  grades
• Going deeper -- getting at student strengths
  weaknesses
• Focus on the nature of student thinking,
  including misunderstandings, not just correct
  responses
• Create interpretive scales and rubrics that
  provide detail about multiple features of student
  competence
• Assessment isnt just about assigning scores
• Its a meaning making process

12
Learning Progressions

• Description of successively more sophisticated
  ways of thinking about a big idea
• Provide a framework for long-term development
• Describes what it means to move towards more
  expert understanding in an area
• Gauge increasing competence over time
• A sequence of successively more complex ways of
  thinking about how an idea develops over time
• Consider how ideas build upon each other to form
  more complex practices or ideas

13
The Researchers Modelof Development/Change/Progr
ess

• A central feature of all this work resides in the
  researchers beliefs about the nature of
  individual development or change -- IN ONE OR
  MORE DOMAINS!!!!
• These beliefs affect the researchers choice of
  theoretical framing, choice of data, choice of
  measure, and choice of analysis tools
• Choices interact with each other affecting the
  way claims are warranted and inferences are drawn

14
Questions to Help Unpack Likely Assumptions

• Question 1
• Has change been framed as systematic or random?
• All assessments have error. How does your model
  of change adjust for this error, or does it?

15
Questions to Help Unpack Likely Assumptions

• Question 2 Is development reversible?
• Implications for the form of the measurement
• Implications for the functional form of the
  developmental trajectories
• The shape of the trajectory should be
  hypothesized before data collection
• The shape can be empirically tested

16
Questions to Help Unpack Likely Assumptions

• Question 3 Is the change unitary or multi-path?
• Your theory of learning should be able to
  identify and describe why certain groups of
  students follow different growth trajectories

17
Questions to Help Unpack Likely Assumptions

• Question 4 Can we consider change to
• be a continuous, gradual, quantitative
  phenomenon?
• have large magnitude shifts on a quantitative
  variable?
• Be a progression through as series of
  qualitatively distinct stages?

18
Questions to Help Unpack Likely Assumptions

• Question 5 Is change considered as differences
  in magnitude in
• an absolute sense?
• calibration?
• conceptualization?

19
Questions to Help Unpack Likely Assumptions

• Questions 6 Is change considered a shared
  characteristic of a group of individuals over
  time?
• Is it what occurs within individuals over time?
• Or both?
• Keep in mind that both kinds of change can occur
  simultaneously!

20
Questions to Help Unpack Likely Assumptions

• Question 7 If we assume that all individuals
  have trajectories of the same functional form are
  there systematic inter-individual differences in
  the values of the individual growth parameters?
• What would this mean theoretically?

21
Questions to Help Unpack Likely Assumptions

• Question 8 Are there cross domain relationships
  in change over time?
• Is the relationship between inter-individual
  differences in intra-individual change over time
  (and the predictors of those differences)
  invariant across learning domains?

22
Questions to Help Unpack Likely Assumptions

• Question 9 Finally, is there invariance across
  groups with respect to the facet of change over
  time under investigation?
• Put simply, are change patterns at the group
  level constant or invariant across groups?

23
Ravits Questions

• Nature of progression
• Path/ paths/ landscapes?
• Nature of movement -cycles, multiple states
• Context dependence
• Nature of learning performances
• Integrate big ideas and practices
• Quantifiable variables that measure learning
  outcomes
• Nature of evidence
• Can we really rely on short terms studies, will
  we (and if so when) need to actually follow
  student learning over grades?
• Wont instruction fundamentally change what
  students can do , and therefore the progression
• Challenges for teaching (instructional practice)
• Challenges for assessment
• Challenges for curriculum design