Exploring design features for effective teaching cases

1
Exploring design features for effective teaching
cases

• Matthew J. Koehler
• Michigan State University

2
Context for my interest

• A circuitous route to this research area
• Mathematics ? Computer Science ? Artificial
  Intelligence ? Cognitive Psychology ? Educational
  Psychology
• Work in mathematics and science classrooms
  exposed me to research projects interested in
  conveying models of student thinking to teachers
  so that they could become better teachers
  (Carpenter Fennema Lehrer Lehrer Schauble).
• Each of these projects wrestled with how to
  develop materials and professional development
  experiences that helped teachers accomplish this.

3
Context for my interest

• Some common elements across efforts
• Complex, but structured models to be conveyed.
• A belief that pieces of video of student
  reasoning, or classroom practices were the
  fertile ground for this learning to take place.
• What could I offer?
• Thinking about the ways that recent advances in
  technology (e.g. video and hypermedia) might be
  designed to scaffold this process of teacher
  development.

4
Ancient History (e.g., 1998)

• Koehler, M.J., Lehrer, R. (1998). Designing a
  hypermedia tool for learning about childrens
  mathematical cognition. Journal of Educational
  Computing Research, 18(2), pp. 123-145.
• My first work in this area was developing teacher
  resources for Cognitively Guided Instruction
  (Carpenter Fennema, 1992)
• They had a handbook for teachers that described a
  model of childrens thinking about simple
  arithmetic, rich with text-based examples of
  childrens solution strategies
• Could materials adapted to a hypermedia format,
  using video vignettes of childrens reasoning
  show any advantage in teacher learning?

5
Ancient History (e.g., 1998)

• ANSWER Yes
• The study used pre-service teachers studying with
  the text-only materials and hypermedia,
  video-rich materials.
• Results showed that the hypermedia / video
  materials were more effective in two areas of
  teacher learning
• A taxonomy of word problems for addition and
  subtraction that had implications for instruction
  and the development of childrens reasoning
• Identifying the developmental sequence of
  solution strategies that children bring to bear
  for each of the problem types.

6
Ancient History -- Summary

• At the very least, well-designed hypermedia that
  used video to exemplify childrens thinking has
  something to offer in the realm of teacher
  learning.
• Important here, is that the way in which video
  was used was only to exemplify childrens
  thinking. Exemplification, is of course,
  important for understanding in general, but
  seemed to be particularly useful here for
  developing teachers knowledge of childrens
  thinking (even though text described the same
  episodes of childrens thinking)

7
Broadening the questions

• Given that the approach of using video and
  hypermedia showed some potential, what were the
  next steps?
• Moved to a consideration of what else you could
  do with video and hypermedia. What about teacher
  performances? What about classroom video? What to
  put in and what to leave out?
• Enter Case and Case-methods
• One common definition suggests that a case is a
  descriptive research document, often presented in
  narrative form, that is based on a real-life
  situation or event. It attempts to convey a
  balanced, multidimensional representation of the
  context, participants, and reality of the
  situation (Merseth, 1994).

8
Broadening the questions

• Also enter a research project by Lehrer
  (geometry) and Lehrer Schauble (modeling in
  mathematics and science).
• Up to 40 practicing teachers at one point.
• Regular teacher meetings that use text, classroom
  video, examples of childrens work, and teacher
  writings.
• New curriculum, being design and implemented by
  teacher-researchers
• A LOT of video from a LOT of classrooms.
• QUESTION Given the complexity in these projects
  (student models, teaching models, curriculum,
  examples of student work, teacher writings)
• HOW to develop a case that would be effective at
  helping learning teachers see and understand
  all of this.

9
Recent History (2002)

• Koehler, M.J. (2002). Designing case-based
  hypermedia for developing understanding of
  childrens mathematical reasoning. Cognition and
  Instruction, 20(2), 151-195.
• Most of the remaining talk explains this paper in
  more detail.
• High level overview Tests the effectiveness of a
  number of decisions about how to use video (and
  cases) effectively.

10
Core Research Questions

• What makes for a good case?
• What elements should be present?
• How should elements be sequenced?
• Do you follow the teacher or students?
• How do you design a case that meets your goals?
• When to link to cases?
• When (and how) to link from cases?
• How to help teachers see what is important in
  the cases.
• Given that there are different ways to accomplish
  the same goals, which ways are most effective for
  teacher learning?

11
What makes for a good case?

• Some caveats
• Primarily talking about what makes a good case in
  K-6 mathematics teaching
• Keeping in mind the kind of materials that are
  used in the professional development communities
  that I described (video,childrens work, texts,
  etc.)

• Five general principles to guide development.
  Good cases are
• Situated in Practice
• Layered with Annotation
• Annotated with Big Ideas
• Criss-cross the domain
• Anchor exploration

12
Elements of good casesSituated in practice

• Since teaching is situated in classroom practice,
  cases of teaching should also be situated in
  classroom practices.
• Advocate use of classroom video
• Video is more engaging and facilitates
  remembering (CTGV).
• Video is more like being there than text
• Written accounts of a classroom assume that
  textual expression can completely express the
  dynamics of classroom activity.

13
Elements of good casesLayered with annotation

• Video does not speak for itself.
• Any two viewers of a classroom video are likely
  to see different things, especially if they
  differ in experience, perspective, or expertise.
• Classroom events are often subtle and difficult
  to interpret.
• Therefore, video cases should be layered with
  annotation that helps teachers interpret
  classroom situations, so that teachers understand
  what the video is a case of.

14
Elements of good casesAnnotated with Big Ideas

• Big ideas in mathematics are important landmarks
  in teaching based on models of student thinking
  (Schifter, 1996 Lehrer Schauble, in press).
• Accordingly, annotation should help teachers
  lift out and interpret the big ideas of the
  domain as they occur in the case.
• Like big mathematical ideas
• norms for argument (Yackel Cobb, 1996)
• general trajectories of student thinking
  (Carpenter Fennema, 1992).

15
Elements of good casesCriss-cross the domain

• Teaching and learning comprises a complex,
  ill-structured domain, cases often embody more
  than one big idea.
• The same episode can be related to the big
  ideas in mathematics, childrens thinking, the
  use of tools and notations, and the classroom
  norms of teaching.
• Good teaching requires not only understanding
  these ideas in isolation, but also orchestrating
  them to design effective classroom environments.
• Cognitive Flexibility Theory (Spiro, Coulson,
  Feltovich, Anderson, 1988) suggests that cases
  should criss-cross the conceptual landscape.

16
Elements of good casesAnchor Exploration

• Cases that portray complex, ill-structured
  classroom situations often raise several
  important issues
• For example, the same episode bring up big
  ideas in mathematics, childrens thinking, the
  use of tools and notations, and the classroom
  norms of teaching.
• Cases should situate, or anchor (CTGV, 1990),
  explorations into these important ideas by
  providing access to further information (e.g.,
  text, interpretation, related case, etc.) as
  issues arise in the case.
• In contrast, if cases only represent the main
  story line, teachers may come to understand
  childrens development as a fixed progression
  through stages.

17
The Domain of Measurement

• Tried to use these ideas in the development of a
  case-based tool for teachers about length and
  area Measurement (based on the work of Lehrer et.
  al).
• Often taught and understood procedurally
• Instead, instruction should help children to
  understand the mathematical ideas that underlie
  measurement (e.g., all the units are the same
  size)

• Goal was to build a case-based hypermedia that
  emphasized 6 strands of teaching and learning
• Key Mathematical Ideas (e.g., Identical Units)
• Classroom Norms (e.g., make thinking visible)
• Childrens thinking (e.g., measurement rulers)
• Connections to other Math ideas (e.g., fractions)
• Classroom activities (e.g., building tape
  measures)
• Tools and Notations (e.g., graph paper)

18
Candidates for good cases

• Given
• the domain to cover (The six strands of
  measurement)
• A set of design principles for good cases
• There a still a lot of possible designs to
  explore.
• This study focused on two possibilities

19
Two types of casesExemplification

• One emphasized exemplification (like in the
  previous work)
• Mini-Demo of this type of case
• Uses all five principles
• Situated in practice
• Layered with Annotation
• Annotated with Big Ideas
• Criss-crosses the domain
• Anchors exploration

20
Two types of casesNarrative

• The other type of case emphasizes narrative
  structure
• Cognitive reasons in general -
• Narrative structure facilitates remembering and
  structures knowledge organization (Mandler, 1984)
• Teacher learning reasons
• Has the same footage as the examples do, but
  shows how they are structured over time.
• How students ideas are initially raised,
  developed, and recalled in later classroom
  episodes.
• Shows how episodes of student thinking are
  sequences to form a lesson, how lessons are used
  to design activities, and how activities and
  lessons are sequence to develop big ideas in the
  curriculum.

21
Two types of casesNarrative

• Mini-Demo of this type of case
• Uses all five principles
• Situated in practice
• Layered with Annotation
• Annotated with Big Ideas
• Criss-crosses the domain
• Anchors exploration

22
An ExperimentRationale

• Wanted to contrast the type of learning afforded
  by these two types of cases
• Believed that the advantage of narrative cases
  lies in the causal structure that ties stories
  together (van den Broek Trabasso, 1998).
• The ability to apply knowledge relies, in part,
  on understanding the causal relationships between
  situations and actions that need to be taken
  (Eylon Reif, 1984).
• Therefore, I expected that narrative cases would
  be more likely to lead to knowledge that could be
  applied.

23
An ExperimentProcedure

• Made two versions of the hypermedia tool
• One version had exemplification cases only.
• The other version had exemplification AND
  narrative cases.
• Twenty-four pre-service teachers, randomly
  assigned to study with one version of the
  hypermedia tool.
• Measures before study, after study, and 6 weeks
  after study
• Speak aloud to video - participants saw short
  classroom segments. Following each clip,
  participants were asked to identify any
  important elements of teaching or learning about
  measurement that they saw. This was used to track
  the type of knowledge that participants acquired.
• Analysis of student work - Participants were
  asked to apply their knowledge to an analysis of
  student work. Interviews addressed what the
  sample student understood (or did not), what the
  student needed to understand, and what classroom
  activities would most likely help this student
  gain understanding. This measured the ability to
  apply knowledge.

24
An ExperimentResults

• Speak aloud video interviews showed that
• Both groups gained knowledge about the
  mathematics of measurement and about the teaching
  norms in place in the classes illustrated in the
  hypermedia.
• Before, after, and six-weeks after instruction,
  both groups were equally able to name the big
  ideas (e.g., identical units) when they saw
  them in the videos
• Conclusion There were no difference in knowledge
  acquisition
• Analyses of student work showed that
• The group who had access to narrative cases did
  better at applying their knowledge to their
  analyses of student work
• More about this ...

25
An ExperimentAnalysis of student work

• Measuring Application of Knowledge
• What does the following student understand about
  measurement?
• Before instruction
• Both groups tended to give procedural
  explanations
• After instruction
• Both groups improved more so for the narrative
  group

26
An ExperimentOther findings

• This trend towards better application of
  knowledge by the narrative group shows up in
  other questions of the student work interview
• Better at listing all the requisite knowledge a
  student would need to understand the problem
• Better at suggesting appropriate follow-up
  activities
• Have better memory for the classroom activities
  outlined in the hypermedia tool
• Analysis of their time allocation during study
  supports the view that the narrative cases were
  responsible for these differences
• Tended to read less text than their exemplar-only
  counterparts
• Tended to watch less exemplar cases
• The more time spent watching narrative cases was
  predictive of better analysis of student work (up
  to a point).

27
Take-away message

• The nature of cases, how they should be crafted,
  and the consequences of different knowledge
  structuring are all important questions to
  investigate.
• This work shows that even given some guiding
  principles for design (the five), competing
  designs have different affordances for learning
• Cases used for exemplification are pretty good at
  helping students acquire declarative knowledge.
• Cases organized around narrative of classroom
  events has some potential for fostering the
  application of that knowledge.

28
More questions / Future work ..

• Since then, my work has opened up considerably
• Most broadly, Im interested in how the many
  design decisions that go into making and
  delivering cases impact learners.
• Here are few specifics of things that Im working
  on

29
Cases for Different Domains

• Science Learning (with Lehrer and Schauble)
• Different way to highlight ideas in the video
  cases
• Different subject matter (science)
• http//mkoehler.educ.msu.edu/growth/
• Early literacy teaching (K-2) (Hopefully w/ Nell
  Duke)
• What elements are needed in a case about literacy
  instruction
• Overall, interested in learning what elements of
  effective design are domain dependent and which
  are general purpose.

30
Future work - Student-authored cases

• Working with a team at MSU on how to design a
  system that scaffolds student-authored cases of
  classroom video
• What a student might learn from crafting a case
• What kinds of video do they use as evidence for
  their claims?
• What types of sequences would students build?
• What would they learn?

Interface design by Charlie Ruggiero
31
Future Work - When is video advantageous? And for
what?

• Aimed at figuring out what the cognitive
  affordances are for video vs. text treatments for
  various types of information/stories/genres in
  the teacher ed population (those 18-21 yr olds)
• Currently collecting data
• Looking at text and video versions of the same
  material
• Video lectures vs. text transcripts
• Poetry vs. Poetry readings
• Stories from the nightly news.
• Trying to figure out what the effect of video
  treatments are in terms of motivation, interest,
  affect, fact learning, persuasion, recall, and
  gist learning.
• Measures

32
Future Work - Video vs. Text

• Preliminary results are interesting
• Poetry reading vs. text of poem
• Video has, if anything, a negative impact on
  students interest level
• Doesnt seem to help their understanding of poem
  at all (and perhaps hurt it)
• News story on college drinking (fact-laden story,
  very serious)
• Seems to be no difference in text vs. video
  treatments on students interest, their ability
  to recall facts from the story, or a change in
  their beliefs about the problems of student
  drinking.
• Story about AIBO, the robotic dog.
• Medium-size effects for peoples interest
  level, change in emotional state, change in
  belief that robotic dogs could make a good
  companion, and ability to recall information
• BIG differences in the quality of students
  written summary of the story.
• WHY???

33
Future work - Genres differences

• Looking at the same classroom video footage
  rendered in different story or TV genres
• Case-format (from before)
• News-story
• Documentary
• Drama
• Reality TV (e.g., MTVs The Real World no
  narrator)
• Looking for the different learning affordances of
  each for the teacher candidate population (e.g.,
  18-21 yr olds). Looking at factual recall,
  interest levels, gist learning, opinion
  changes, etc.
• Hopefully some correspondance to previous slide

34
Conclusions and Questions

• In general, Im keenly interested in unpacking
  the learning affordances of the elements of
  effective case design.
• Questions? Comments? Critiques?

35
Sample Data from speak aloud video
Lesson has good use of visual manipulatives Teache
s about the need for same-size units Teaches
about units that are not whole Teaches that
things that look different may be the same
size Teaches about breaking big measurements into
smaller ones Teaches about reusing units Activity
is hands on Teacher uses a variety of units Good
introduction to tools such as rulers Teacher uses
open-ended questions Students lead class
discussions Teacher takes input from the
students Using more than just standard
units Teachers build understanding by restating
students ideas Teachers bring up history Children
understand that you cannot have gaps between
units Children understand the need for identical
units Children account for fractional
units Teachers help students see consequences of
their ideas Children understand that different
objects can have the same measure Children
understand that units can be reused
Episode Only (Post-test)
Episode Plus Narrative (Post-test)
36
What would a student need to know to understand
the problem
Not using the diagram and 1x1 squares
correctly Doesnt understand all the units have
to be the same size Student would need to
know the diff between square and rectangle All
the units of a measurement need to be the same
size There can be no spaces between the units
Episode Only (6-weeks later)
Episode Plus Narrative (6-weeks later)
37
Follow up activities suggested
38
Very Early Results