Modelling Student Uncertainty and Mental Models using Bayesian and Inductive Logic techniques

1
Modelling Student Uncertainty and Mental Models
using Bayesian and Inductive Logic techniques

• Kate TaylorNewnham CollegeUniversity of
  Cambridge, UK kate.taylor_at_cl.cam.ac.ukhttp//www
  .cl.cam.ac.uk/ksw1000

2
The Operating System Experiment

• The students do not see our ontology.
• Learn about scheduling using the questioner and
  to build up a mental map after each question.
• This is done using a Draw Your Own ontology tool
  that uses our ontology language.

3
The Bayesian Belief Network

• The Bayesian net is built from the ontology,
  assuming exactly the relationships that we use
  for the knowledge base.
• The probability of understanding the concept
  priority having understood the concept process is
  given by
• The likelihoods are calculated as the proportion
  of links to this concept that the student has
  drawn.

4
Updating the Belief Network

• Bayesian updating is then used each time a
  relationship is added and the probability
  recalculated as above to reflect the new
  likelihood or evidence of understanding.
• There are a number of complications
• Our ontology may be incomplete either in concept
  nodes or dependency links
• Our ontology may not be detailed enough, where a
  concept could usefully be broken down into
  sub-concepts
• The students mental model differs from our
  ontology

5
Deducing Misunderstanding

• We develop an Inductive Logic Programming (ILP)
  approach to data mining on the students
  ontology. ILP uses examples and background data
  to induce new rules.
• An alternative to finding all the routes using a
  complete search, infeasible for relatively small
  number of concepts in the ontology.
• Mining Technique
• the areas that are incorrect, subtracting the
  student model from ours to filter out the areas
  where the two models match.
• a model of the areas which they have not captured
  at all.
• what questions have been asked.

6
How Wrong am I? (Bruza et al)

• Manhattan distance how far apart the two
  concepts in the question are measured as distance
  across the lattice.
• Semantic difference how often we have moved up
  and down the taxonomic hierarchy of is_a_part_of
  and is_a_kind_of before matching the predicates
  and concepts used in the question. Hamming
  distance how near the incorrect string scheduler
  is to the one required scheduling (algorithm)
  the overlap is five characters and the Hamming
  distance is two.

7
Future Work Logical Inference

• In the next stage of development, we would like
  to add better explanation generation to correct a
  misunderstanding.
• However, as this is effectively a search for a
  path between two nodes in a densely connected
  graph of concepts, its performance decreases
  exponentially with the number of nodes.
• Add logical nodes to the Belief Net

8
Future Work Bayesian Inference

• A conjugate prior gives a better overall
  approximation than the small amount of real data
  we will collect in our experiment.
• However, we need to analyse whether the
  estimating done in the prior is gaining as much
  accuracy as we hope when compared to the simpler
  techniques as we are ultimately hoping to do
  these calculations in real time.
• An alternative approach is to use Monte Carlo
  Markov Chain (MCMC) to provide a sample
  probability for each concept

9
Thank you for listening. What do you think,
please?
Kate TaylorNewnham CollegeUniversity of
Cambridge, UK kate.taylor_at_cl.cam.ac.ukhttp//www
.cl.cam.ac.uk/ksw1000