Cognitive Modeling in A ProblemBased Learning Technology

1
Cognitive Modeling inA Problem-Based Learning
Technology

• Albert Corbett
• Human Computer Interaction Institute
• Carnegie Mellon University
• Pittsburgh, PA 15213
• corbett_at_cmu.edu
• Co-Director Pittsburgh Advanced Cognitive Tutor
  Center

2
Cognitive Tutor Technology

• Problem Solving Environments Support rich
  problem analysis, solution, communication
• Model Tracing Follow students solution
  step-by-step, with just-in-time help
• Knowledge Tracing Monitor students
  growing knowledge individualize the
  problem sequence
• Cognitive Model Problem solving knowledge
  

3
Genetics Tutor Lesson Mendelian Transmission
Experiment Design
Given Three Plant Strains Red1 Red2 White3 Goal
Determine genotype of each by crossing
them Student crosses Red1 x White 3 (among 6
possible crosses) Resulting offspring are 75
red, 25 white
2 Conclusions Red1 is heterozygous  White3 is
homozygous recessive 3 Reasonable Next Crosses
to reveal Red2 Genotype Red2 x Red1 Red2
x Red2 Red2 x White3 5 rules in cognitive
model
4
Cognitive Tutors A Technology to Support
Learning By Doing

• Cognitive Model Expert system solves
  mathematics problems in the many ways students
  do, including misconceptions
• Rule IF the goal is to solve a(bxc)
  d
• THEN rewrite this as bx c d/a
• Rule IF the goal is to solve a(bxc)
  d
• THEN rewrite this as abx ac
  d
• Bug rule IF the goal is to solve a(bxc) d
• THEN rewrite this as abx c
  d
• Model Tracing Follows student through their
  individual approach a problem -gt
  context-sensitive instruction. Problem solving
  ends in success.
• Knowledge Tracing Assesses student's knowledge
  growth -gt individualized activity selection and
  pacing

5
Algebra Tutor LessonLinear Systems

• A hot air balloon is at an altitude of 350 feet
  and rising 90 ft per minute. A blimp is at an
  altitude of 8500 feet and descending 250 ft per
  minute.
• Question 4 When will the blimp land?

Recognize landing translates to a given
height of 0. Graph blimp descent function
and read time of landing off graph Use
equation solver to set up and solve equation
0 8500-250X Unwind equation in
head and enter -8500/250 in worksheet
6
Model TracingInterpret Problem-Solving Actions
Just-in-Time Error Message
You have entered the given 0 in the wrong column
of the worksheet.
7
Model Tracing Context-Specific Advice
Level 1 Read the problem statement. What
information is given in question 1 about the
height of the blimp? Level 2 Important
information about the height of the blimp is
highlighted for you in the problem
statement Level 3 Type the height of the blimp
in feet into the selected box in the worksheet
8
Early Demonstrations of Model Tracing
Effectiveness
Students learn faster (Lisp
programming) Cognitive Tutor group finishes tutor
problems in 1/3 the time of control group and
scores higher on tests (0.75 effect)
Students learn better (Geometry Proof) Classroom
studies Learning time is constant Tutor group
scores one letter grade better on test (1.0
effect)
(Anderson, J.R., Corbett, A.T. Koedinger, K.R.
and Pelletier, R. (1995). Cognitive tutors
Lessons learned. The Journal of the Learning
Sciences, 4, 167-207).
9
Outline

• Cognitive Tutor Technology
• Model Tracing Context-Sensitive Help
• Knowledge Tracing Dynamic Assessment
• Impact
• Recent Directions

10
Knowledge TracingDynamic Assessment of Knowledge

• Knowledge Tracing Monitor students growing
• knowledge of problem solving rules in the
  cognitive model.
• Cognitive Mastery Individualize the
  problem sequence to foster
  student mastery (learning) of
  each of the problem solving rules.

• At each opportunity to apply a rule update the
  probability that
• the student knows the rule
• -Simple learning performance assumptions
• -Bayesian updates

11
Cognitive Mastery Validation

• Impact
• Knowledge Tracing in
• tutor accurately predicts
• student test performance
• Actual Mean 0.81
• Predicted Mean 0.86
• R 0.66
• MAE 0.10
• (Corbett, A. Anderson, J. (1995). Knowledge
  tracing Modeling the acquisition of procedural
  knowledge. User Modeling and User-Adapted
  Interaction, 4, 253-278.)

12
Cognitive Mastery Efficiency
Effect Size Cognitive Mastery vs. Fixed
Curriculum 0.65
(Corbett, A.T. (2001). Cognitive computer
tutors Solving the two-sigma problem. User
Modeling Proceedings of the Eighth International
Conference, UM 2001, 137-147.)
13
Outline

• Cognitive Tutor Technology
• Cognitive Modeling
• Model Tracing Dynamic
• Knowledge Tracing Dynamic Assessment
• Impact
• Algebra and Geometry Problem Solving
• Recent Directions

14
2003-2004 Cognitive Tutor Algebra/GeometryAbout
1800 schools, 200,000 students
15
Cognitive Tutor Math CoursesNational
Dissemination

• Technology Integral to Full Courses
• Text, Cognitive Tutors, Assignments, Assessments
• Cognitive Tutor 2 days a week
• Teachers interact with individual students
• Small Group collaborative learning
• Interactive Text problem solving and writing
• Professional Development
• Hotline Support - Technology and Curriculum
• Course Results
• Achievement Gains Classroom Impact

16
Field Study Results

• Full year classroom experiments with comparison
  classes
• Replicated over 3 years in urban schools
• In Pittsburgh Milwaukee
• Results
• 50-100 better on problem solving
  representation use.
• 15-25 better on standardized tests.

(Koedinger, K. Anderson, J. Hadley, W. Mark M.
(1997). Intelligent tutoring goes to school in
the big city. International Journal of Artificial
Intelligence in Education, 8, 30-43.
17
Outline

• Cognitive Tutor Technology
• Cognitive Modeling
• Model Tracing Dynamic
• Knowledge Tracing Dynamic Assessment
• Impact
• Recent Directions
• Cognitive Science Findings
• Integrating Student Explanations

18
Cognitive Science Foundations

• Active Student Processing Helps Prepares for
  Learning
• Crouch, Fagen, Callan, Mazur (2004)
• Schwartz Martin (2004)
• Worked Examples Complement Problem Solving
• Kalyuga, Chandler, Tuovinen Sweller (2001)
• Trafton Reiser (1993)
• Self-Explanations/Worked Examples
• Chi 2000
• Chi, Bassok, Lewis, Reimann Glaser (1989)
• Bielaczyc, Pirolli Brown (1995)

19
Strategies to Support Student Learning in
Cognitive tutors

• Strategies to Support Problem Solving
• Plan Scaffolding (Corbett Trask, 2000)
• Student Problem-Step Explanations (Aleven et al
  2003)
• Augmented Feedback (Corbett Trask, 2000)
• Debugging Support (Mathan Koedinger 2003)
• Interactive Explanations of Worked Examples
• Algebra Model Analysis tool (Corbett et al, 2003)
• Provide both situation and algebra model
• Student explains mapping between situation and
  algebra expression
• Make structure of algebra expression explicit

20
Worksheet LessonGeneralization of Numerical
Examples
You are saving to buy a bicycle. You have 20 to
start with. Each week, you save 10 of your
allowance. If you know the amount of time you
have been saving, then you can find the amount of
money you have saved.
1. How much will you save in 5 weeks? 10
5 20 2. How much will you save in 6 weeks?
10 6 20 3. How much will you save in 7
weeks? 10 7 20
10W 20
21
Interactive Explanations of Worked Examples
You are saving to buy a bicycle. You have 20 to
start with. Each week, you save 10 of your
allowance. If you know the amount of time you
have been saving, then you can find the amount of
money you have saved. Y 10X 20
22
Model Analysis Tool
You are saving to buy a bicycle. You have 20 to
start with. Each week, you save 10 of your
allowance. If you know the amount of time you
have been saving, then you can find the amount of
money you have saved. Y 20 10X
23
Model Analysis Tool
Paper and Pencil Test Results Pretest Posttes
t p(correct) p(correct) Describing Model
0.24 0.61 Components in Own
Words Generating 0.52
0.72 Algebraic Model Transfer of
worked-example/explanation-based learning to
explaining in own words and to algebraic model
generation
24
ALPS Cognitive Tutors Synthetic Interviews

• Synthetic Interviews (Stevens Marinelli, 1998)
• Natural Language Students Inputs
• Anthropomorphic Interface
• Model Analysis
• Students type
• descriptions in own
• words.

25
ALPS Cognitive Tutors and Synthetic Interviews

• Synthetic Interviews (Stevens Marinelli, 1998)
• Natural Language Students Inputs
• Anthropomorphic Interface
• Open-Ended
• Questions during
• problem solving

26
Conclusions

• Cognitive modeling deployed in cognitive tutor
  technology substantially improves learning
  outcomes, both through
• Model tracing,
• Knowledge tracing
• The fundamental challenge of education remains
• Designing interactive learning tasks that yield
  deeper understanding, and in turn, better
  transfer and retention.

27
References

• Aleven, V., Koedinger, K. Popescu, O. (2003). A
  tutorial dialogue system to support
  self-explanation Evaluation and open questions.
  Proceedings of AIED 2003 The 11th International
  Conference on Artificial Intelligence
  Education, 39-46.
• Bielacyzc, K., Pirolli, P. Brown, A. (1995).
  Training in self-explanation and
  self-regulation strategies Investigating the
  effects of knowledge acquisition strategies on
  problem solving. Cognition and Instruction, 13,
  221-252.
• Chi, M. (2000). Self-explaining explository
  texts The dual processes of generating
  inferences and repairing mental models. In
  Glaser, R (Ed.) Advances in instructional
  psychology. Mahwah, NJ Erlbaum, pp. 161-238.
• Chi, M., Bassok, M., Lewis, M, Reimann, P.
  Glaser, R. Self-explanations How students
  study and use examples in learning to solve
  problems. Cognitive Science, 13, 145-182.
• Corbett, A. (2001). Cognitive computer tutors
  Solving the two-sigma problem. User Modeling
  Proceedings of the Eighth International
  Conference, UM 2001, 137-147.
• Corbett, A. Trask, H. (2000). Instructional
  interventions in computer-based tutoring
  Differential impact on learning time and
  accuracy. Proceedings of ACM CHI2000 Conference
  on Human Factors in Computing, 97-104.
• Corbett, A.T. Wagner, A. Raspat, J. (2003). The
  impact of analyzing example solutions on problem
  solving in a pre-algebra tutor. Proceedings of
  AIED 2003 The 11th International Conference on
  Artificial Intelligence Eduction, 133-140.
• Crouch, C., Fagen, A., Callan, P. Mazur, E.
  (2004). Classroom demonstrations Learning Tools
  or Entertainment? Am. J. Phys., 72, 835-838.
• Kalyuga, S., Chandler, P., Tuovinen, J.
  Swelller, J. (2001). When problem solving is
  superior to studying worked examples. Journal of
  Educational Psychology, 93, 579-588.
• Mathan, S. Koedinger, K. (2003) .Recasting the
  feedback debate Benefits of tutoring error
  detection and correct skills. Proceedings of AIED
  2003 The 11th International Conference on
  Artificial Intelligence and Education, 13-20.
• Schwartz, D. L., Martin, T. (2004). Inventing
  to prepare forlearning The hidden efficiency of
  original student production instatistics
  instruction. Cognition Instruction, 22,
  129-184.
• Stevens, S. Marinelli, D. (1998). Synthetic
  Interviews the Art of Creating a Dyad Between
  Humans and Machine-based Characters, Proceedings
  of the 4th IEEE Workshop on Interactive Voice
  Technology for Telecommunications Applications.
• Trafton, J. Reiser, B. The contributions of
  studying examples and solving problem to skill
  acquisition. (1993). Proceedings of the 1993
  Conference of the Cognitive Science Society,
  1017-1022.