Studying Online Social Networks

1
Productive Failure Manu Kapur Assistant
Professor of Learning Sciences
Technology National Institute of Education,
Singapore ICET, Nov 22, 2007
2
Agenda

• Set up the argument for productive failure
• Study 1 online setting (in Indian schools)
• Study 2 F2F setting (in a Spore school)
• Draw common patterns across the two studies
• Draw implications using a complexity theory
  perspective

3
Argument for Productive Failure
The situative, socio-constructivist perspective
learners need to be participate/collaborate in
authentic, ill-structured problem-solving
activities for meaningful learning to take place
Learners need to be scaffolded in their process
of engaging in ill-structured tasks or else they
may fail But does this mean there is little
efficacy embedded in un-scaffolded,
ill-structured problem-solving processes?
4
Argument for Productive Failure
1. A Logical fallacy A implies B does not mean
not-A implies not-B 2. Validity and reliability
of measures 3. Several research programs point to
the role of failure in learning and problem
solving (VanLehn, 2003 McNamara, 2001 Schwartz
Martin, 2004, etc.) 4. The role of persistence
vis-à-vis performance success
5
Study 1 Demonstrating an Existence Proof for
Productive Failure
6
Purpose
To examine whether or not there is a hidden
efficacy in un-scaffolded, ill-structured
problem-solving processes Context Synchronous
CSCL problem solving in Physics with N 309,
11th grade science students across 7 high schools
in India
7
The Study in Brief

• Ill-structured groups showed
• Struggle with defining solving the problem
  (MANOVA)
• Complex and chaotic patterns of interaction (LSA)
• Low convergence in their discussions
  (computational)
• Poor group performance (ANCOVA),
• BUT, better individual performance on both well-
  and ill-structured problems (Hierarchical Linear
  Modeling)

8
So?

• It seems that there is efficacy embedded in
  un-scaffolded, ill-structured problem-solving
  processes
• This efficacy can be extracted using a
  contrasting-case mechanism a delay of structure
• This efficacy seemed to be embedded in the
  chaotic, divergent, all-over-the-place
  interactional dynamics in the ill-structured
  groups

9
Implications

• Question the default pedagogical rush to scaffold
  ill-structured problem solving
• The ontology of learning problem solving
• Simple to complex incremental, or
• Complex to simple - emergent

10
Study 2 Exploring Productive Failure in a
Singapore Classroom
11
Purpose

• To test the productive failure hypothesis in a
  Singapore classroom, i.e., examine whether or not
  there is a hidden efficacy in un-scaffolded,
  ill-structured problem-solving and how it
  compares with traditional lecture practice
  instruction
• Context
• Clementi Town Sec School A mainstream school
• N 76 Two classes of Sec 1 express-stream math
  students taught by the same teacher
• Two curricula units, each lasting 7 lessons
  (about 2 weeks each)
• Estimation Approximation Rate Speed

12
The Design in Brief (N 76 Sec1 math students
from CTSS, Singapore)
13
Example of an Ill-Structured Problem
Gist of the Biking Problem (Speed Unit) Two
friends, Jasmine and Hady, had to get to an
exhibition by a certain time. They could walk or
ride a bike or both. The constraint was that they
had to reach the exhibition at the same time
despite having different walking and biking
speeds. Furthermore, a little while into their
journey, one of the bikes breaks down, requiring
re-strategizing for the rest of the journey.
14
Results

• Process Analysis
• Problem Representations
• Group Individual Solution Scores
• Self-report Confidence in their Solutions
• Self-report Lesson Engagement
• Rich interactional data remains to be analyzed
• Outcome Analysis
• Pre-Post-test scores on rate and speed items
  well-structured and ill-structured problem items

15
Process Analysis Group Problem Representations
16
Process Analysis Group Solution Scores
17
Process Analysis Individual Solution Scores
18
Process Analysis Confidence Engagement
Engagement
Confidence
19
Outcome Analysis

• Sample Well-structured Items
• The flight distance between Singapore and Japan
  is 5316 km. A plane takes 6 hours and 15 min to
  fly from Singapore to Japan. What is the average
  speed of the plane?
• David travels at an average speed of 4km/hr for 1
  hour. He then cycles 6km at an average speed of
  12 km/hr. Calculate his average speed for the
  entire journey in km/hr.

20
Outcome Analysis
Ill-structured item Hummingbirds are small birds
that are known for their ability to hover in
mid-air by rapidly flapping their wings. Each
year they migrate approximately 8583 km from
Canada to Chile. The Giant Hummingbird is the
largest member of the hummingbird family,
weighing 18-20 gm. It measures 23cm long and it
flaps its wings between 8-10 times per second.
For every 18 hours of flying it requires 6 hours
of rest. The Broad Tailed Hummingbird beats its
wings 18 times per second. It is approx 10-11 cm
and weighs approx 3.4 gm. For every 12 hours of
flying it requires 12 hours of rest. If both
birds can travel 1 km for every 550 wing flaps.
If they leave Canada at approximately the same
time, which hummingbird will get to Chile first?
21
Outcome Analysis Overall Gains
Controlling for the effect of prior knowledge as
measured by the pre-test
10, p .002, ES .75
22
Outcome Analysis
6, p .02, ES .42
23, p .004, ES .98
23
Going even further
We also wanted to know how the PF cycle prepares
students to learn and apply new concepts on their
own Extension Concept Relative Speed Half the
students in each condition (PF and LP) took a
scaffolded item on relative speed, the other
halves took an un-scaffolded version Then all
students took an unscaffolded, conceptually
difficult problem on relative speed.
24
Going even further

• Item 1 You and your friend start running at the
  same time from the same position but in opposite
  directions on a 400m running track. You run at
  5m/s whereas your friend runs at 3m/s.
• In 1 second, how many meters do you travel
  towards your friend?
• In 1 second, how many meters does your friend
  travel towards you?
• Therefore, in 1 second, how many meters do the
  two of you travel towards each other in total?
• How many seconds will it take for the two of you
  to first cross each other?
• Item 2 Two MRT trains on separate but parallel
  tracks are traveling towards each other. Train A
  is 100m long and is traveling at a speed of
  100km/hr. Train B is 200m long and is traveling
  at a speed of 50km/hr. How many seconds will it
  take from the time that the two trains first meet
  to the time they have completely gone past each
  other?

25
Results
26
Results
27
Discussion

• Productive Failure design seems tractable within
  local classroom context since the study was
  carried out within the timetable and curricula
  constraints
• It seems to suggest shorter-term inefficiencies
  and failure but longer-term gains on both
  standard, well-structured items and more
  higher-order, ill-structured problem-solving
  items
• The assessment experiment reveals that PF also
  prepares students to better use the structure
  provided for new concepts
• One of the reasons structuring from the outset
  may not work could be due to our assumption that
  learners are prepared to use the structure
  provided!

28
Patterns across the 2 studies

• Collaboration in small groups
• Engage students in complexity of solving complex,
  ill-structured problems
• Minimize a priori structure by not providing any
  external support or scaffolds
• Delay structure, be it in the form of a
  contrasting well-structured problem or a
  consolidation lecture
• Shorter-term inefficiency and failure but
  longer-term productivity

29
A Complexity Theory Perspective

• Structure imposes order on the learning
  performance space
• Short term efficient
• Long term may lack flexibility and adaptability
• The laws of self-organization and complexity is
  under certain conditions, as systems (biological,
  social, neural, etc.) comprising multiple
  interacting agents (genes, people, neurons, etc.)
  become increasingly complex over time, there
  comes a critical point where the system
  self-organizes and order emerges spontaneously
  from chaos.

30
A Complexity Theory Perspective
So, order is important! But, how does it come
about? Top-down vs. bottom-up order (efficiency)
(flexible, adaptive)
31
Self-Organization Complexity
Do we engage learners more in efficient or
innovative processes?
32
Implications for Adaptive Expertise
(Hatano Inagaki, 1986)
33
A Working Hypothesis underpinning Productive
Failure
In the longer run, an innovation-dominant
approach would be more optimal for the
development of adaptive expertise than a balanced
approach.
34
THANK YOU manu.kapur_at_nie.edu.sg