Integrated Collaborative Learning Environments with Dynamic Support

1
Integrated Collaborative Learning Environments
with Dynamic Support

• Carolyn Penstein Rosé
• Language Technologies Institute/
• Human-Computer Interaction Institute

2
Design PrincipleKnow what problem you are trying
to solve!!
3
Passivity in Class Discussions
4
Accountable Talk

• A codified set of moves that Ts and Ss can be
  taught that build effective language in the
  classroom

• Press individuals to explain and elaborate
• Require individuals to listen and respond to each
  other
• Reveal misunderstandings, incomplete ideas..
• Let Ss know how others are hearing them
• Spread participation through the group

5
Transactivity

• Students explicitly display their reasoning
• Students orient their contributions towards
  previous contributions
• May be competitive or non-competitive
• May be representational operational
• May be oriented towards self or other

Student 1 Alright, we put that the negative and
positive whole numbers. Teacher Alright, other
ideas you want to add on, yes Mario. Student 2
I put a um numbers that doesn't include decimal
points or fractions but they include, they can
include negative signs or positive signs.
6
Accountable Talk elicits Transactivity in group
discussion
Accountable Talk Move Corresponding Transactivity Move
Revoicing Clarification, Competitive Clarification, Refinement
Restate Request Paraphrase, Competitive Paraphrase
Reasoning Application Request Extension, Competitive Extension, Contradiction, Reasoning Critique
Prompting for Further Participation. Extension, Completion, Clarification
Request for Elaboration Extension, Completion
Challenge or Request for Counter-Example Competitive Paraphrase, Contradiction, Competitive Extension
7
Accountable Talk elicits Transactivity in group
discussion

• Eddie Well, i don't think it matters what order
  the numbers are in. You still get the same
  answer. But three times four and four times
  three seem like they could be talking about
  different things.
• Teacher Rebecca, do you agree or disagree with
  what Eddie is saying?
• Rebecca Well, I agree that it doesn't matter
  which number is first, because they both give you
  twelve. But I don't get what Eddie means about
  them saying different things.
• Teacher Eddie, would you explain what you mean?
• Eddie Well, I just think that like three times
  four can mean three groups of four things, like
  three bags of four apples. And four times three
  means four bags of three apples, and those don't
  seem like the same thing.
• Tiffany But you still have the same number of
  apples, so they are the same!
• Teacher OK, so we have two different ideas here
  to talk about. Eddie says the order does matter,
  because the two orders can be used to describe
  different situations. So Tiffany, are you saying
  that three times four and four times three can't
  be used to describe two different situations?

8
Unequal Participation
9
Wizard-of-Oz Setup
Tutor
Student
Student
10
Problem Solving Interface
Interface allows students to submit Incorrect
solutions
11
Reflection Interface
12
Scaffolding for Collaboration

• Prompts to encourage the instructionally
  beneficial behavior of students
• Explanations
• Equal contributions

13
Explanation Prompt
(Student 1 corrects the answer for the
confederate learner without offering any
explanation) Student1 c is 1 Auto-Prompt (to
student 1) Help the student understand your
correction. Student2 what does c mean?
Student1 constant Student2 ok Student1
the constant in front of the variable which in
our case is variable t Student2 ok Student1
now for the second part we use the constant
function
14
Equal Contribution Prompt
(After the participant makes 5 correct problem
actions without confederate students
contribution) Auto-Prompt It seems like the
other student has not contributed lately. Why
dont you see if they need help? Student1 you
want to do the other half? Student2 ok
15
Learning effect

• Significant benefit for prompting on student
  learning with pretest as a covariate
• F(1,39) 4.47, p lt .05, effect size .58

16
Process Losses
17
Another Domain Earth Sciences

• Collaborative idea generation in the Earth
  Sciences domain
• Chinese TagHelper learns hand-coded topic
  analysis
• Human agreement .84 Kappa
• TagHelper performance .7 Kappa
• Trained models used in follow-up study to trigger
  interventions and facilitate data analysis

18
Example Dialogue
Speaker Text
Student 1 People stole sand and stones to use for construction.
VIBRANT Yes, steeling sand and stones may destroy the balance and thus make mountain areas unstable. Thinking about development of mountain areas, can you think of a kind of development that may cause a problem?
Student 2 Development of mountain areas often causes problems.
Student 1 It is okay to develop, but there must be some constraints.
Feedback during idea generation increases
learning and idea generation productivity
(except during the first 5 minutes) (Wang et al.,
2007)
19
Process Analysis
IndividualsFeedback
IndividualsNoFeedback
PairsFeedback
PairsNoFeedback

• Process loss Pairs vs Individuals
• F(1,24)12.22, plt.005, 1 sigma

Process loss Pairs vs Individuals F(1,24)4.61,
plt.05, .61 sigma
Negative effect of Feedback F(1,24) 7.23,
plt.05, -1.03 sigma
Positive effect of feedback F(1,24)16.43,
plt.0005, 1.37 sigma
20
Deficient Help Exchange
21
Why study help?

• Offering of deep help and elaborated explanations
  predicts post test performance (e.g., Webb et
  al., 2002)
• Help behavior mediates learning (e.g., Gweon et
  al., 2006 Gweon et al., 2007)
• Exchanging help in the context of collaborative
  learning increases identification with a learning
  community and motivation, and improves race
  relations (Sharan, 1980)
• Providing help leads to feelings empowerment for
  low status students (Elbers Hann, 2004)

22
How can we prompt helping behavior?

• DIRECT Explicit prompts (Gweon et al., 2006)
• LESS DIRECT Manipulating availability of help
  from problem solving environment (Gweon et al.,
  2007)
• Girls offered more help with delayed feedback
  from environment
• Boys offered more help with immediate help from
  the environment
• SUBTLE Conversation agents inject humor (Kumar
  et al., 2007)
• Positive effects on perceived help exchange
  (plt.05), effective help exchange (plt.1), and
  Learning (p.06)

23
Collaborative Problem Solving Environment
24
Extraneous Entertainment?
25
Coding Scheme (Gweon et al., 2007)

• Help provision mediates learning (Gweon et al.,
  2006 Gweon et al., 2007)
• (R) Help Requests Help me, Im stuck, I
  dont get it.
• (P) Help Provisions Find the common
  denominator, Try the flip and multiply
  strategy
• (C) Cant help I dont know, Im stuck too
• (D) Deny help ask the teacher, youre an
  idiot, press the help button
• (N) Nothing

26
Examples

• R Student 1 What do we put on top of the
  fraction?
• P Student 2 Did you find a common denominator?
• ltstudent 1 correctly finds the common
  denominatorgt
• R Student 1 I dont get it
• D Student 2 hold on
• ltthen student 1 tried something and got negative
  feedback from the problem solving environment.gt
• ltfinally student 1 tried something else, which
  was correct, and got positive feedback from the
  problem solving environmentgt
• R Student 1 Why 16?
• C Student 2 I dont know.
• R Student 1 I dont get it
• ltstudent 2 tries something and gets negative
  feedback from the problem solving environmentgt
• ltstudent 2 tries something else and gets negative
  feedback from the problem solving environmentgt
• ltstudent 2 clicks on the help buttongt
• ltstudent 1 tries something that is correct and
  gets positive feedback from the problem solving
  environmentgt

27
Big findingsMore help related episodes in
experimental conditionMore episodes where
people got help, and then managed to solve the
problem themselves in the experimental condition
28
Vision
29
Vision

• Support for collaborative learning is like
  training wheels
• Effective support allows learners to achieve
  better collaboration
• Unnecessary support can be demotivating
• Fading support is ideal
• But too little support can be detrimental as well
• Trained human facilitators are able to achieve
  the right balance

30
Vision

• Support for collaborative learning is like
  training wheels
• Effective support allows learners to achieve
  better collaboration
• Unnecessary support can be demotivating
• Fading support is ideal
• But too little support can be detrimental as well
• Trained human facilitators are able to achieve
  the right balance

31
Vision

• Support for collaborative learning is like
  training wheels
• Effective support allows learners to achieve
  better collaboration
• Unnecessary support can be demotivating
• Fading support is ideal
• But too little support can be detrimental as well
• Trained human facilitators are able to achieve
  the right balance

32
Vision

• Support for collaborative learning is like
  training wheels
• Effective support allows learners to achieve
  better collaboration
• Unnecessary support can be demotivating
• Fading support is ideal
• But too little support can be detrimental as well
• Trained human facilitators are able to achieve
  the right balance

33
Vision

• Support for collaborative learning is like
  training wheels
• Effective support allows learners to achieve
  better collaboration
• Unnecessary support can be demotivating
• Fading support is ideal
• But too little support can be detrimental as well
• Trained human facilitators are able to achieve
  the right balance

34
Vision

• Support for collaborative learning is like
  training wheels
• Effective support allows learners to achieve
  better collaboration
• Unnecessary support can be demotivating
• Fading support is ideal
• But too little support can be detrimental as well
• Trained human facilitators are able to achieve
  the right balance

35
Architecture for Dynamic Collaboration Support
36
VMT-Basilica
Students learn up to 1.25 standard deviations
more when interactive support is provided in the
environment.
37
Basilica Architecture
A channel filter collects all of the events that
occur at the interface from all students involved
in the conversation
38
Basilica Architecture
Events are then channeled to special purpose
filters such as a text processing filter Other
filters may keep track of time or other factors
not related to student behavior
39
Basilica Architecture
Filters that are related to decision making for
specific types of interventions receive
notifications from the analysis filters and pass
notifications on to the Actors, which are
responsible to launching interventions
40
Basilica Architecture
Actors of different types produce events that are
sent to the Outgoing Message Spooling
Filter From there, events are sent through the
Presence Actor to the interface.
41
Issues to consider

• What problem are you trying to solve?
• Formulate analysis scheme
• When should you intervene?
• Use technologies like TagHelper and SIDE to track
  interaction and trigger support
• What should the intervention be?
• Technologies like TuTalk can be used to offer
  support

42
Questions?