CSCE 582: Bayesian Networks

1
CSCE 582 Bayesian Networks

• Paper Presentation conducted by
• Nick Stiffler
• Ben Fine

2
Bayesian networks A teachers view

• Russel G Almond
• Valerie J Shute
• Jody S. Underwood
• Juan-Diego Zapata-Rivera

3
ACED

• A Computer-Based-Assessment-for-Learning system
  covering the topic of sequences
• In this Paper it spans three sequence types
• Arithmetic
• Geometric
• Recursive

4
ACED

• A Prototype that explores
• Madigan and Almond Algorithms for selection of
  the next task in an assessment
• The use of targeted diagnostic feedback
• Tech solutions to make the assessment accessible
  to students with visual impairments

5
Geometric Sequence Model

• Proficiency Levels available to each node
• Low .
• Medium
• High .

6
Bayesian Network (SS)

• Individual task outcome variables
• -are entered as findings in task specific
  nodes where the results are propagated through
  the proficiency model
• Posterior Proficiency Model
• -gives the belief about the proficiency state
  for a particular student
• Note Any functional of the posterior
  distribution can be
• used as a sore

7
Terminology

• Si0, Si1,,Sik proficiency variables for
  student i
• Si0 special overall PV (Solve Geo.
  Problems)
• Xi Body of evidence
• P(SikXi)
• -conditional distribution of Sk given the
  observed outcomes

8
The Four Statistics (at least the ones we look
at)

• Margin
• Cut
• Mode
• EAP

9
Margin

• The Marginal Distribution of Proficiency
• P(SikXi)
• expected numbers of students in each proficiency
• Si P(SikXi)
• Average proficiency for the class
• Si P(SikXi)
• class size

10
Cut

• Identifier for a special state
• Ex. students medium are proficient
• P(Sik medium Xi)
• Average cut score is the expected proportion of
  proficient students in the class

11
Mode

• The value of m the produces
• maxP(Sik m Xi)
• Improvements
• If student is within a threshold should be
  identified as being on the boundary
• When the Marginal Distribution is evenly spread
  out the system should identify students who have
  the greatest uncertainty
• To get modal scores count the number of students
  assigned to each category

12
EAPExpected a Posteriori

• Assign numbers to states to get an expectation
  over posterior
• High 1
• Medium 0
• Low -1

1P(Sik high Xi) 0 P(Sik med Xi)
-1P(Sik lowXi) Reduces to
P(Sik high Xi) - P(Sik low Xi)
13
EAP (cont.)

• What it means
• The EAP would return the average ability level
  for each class
• Standard Deviation variability of proficiency

14
Scores coming out of the BN
15
Individual Level Plots
16
Comparing Groups
17
(No Transcript)
18
Comparing Groups
19
(No Transcript)
20
Reliability

• Observed Score True Score Error
• Signal to noise ration in signal processing
• Applying the Spearmen Brown formula

21
Spearmen Brown formula
      is the predicted reliability
N is the number of "tests" combined
is the reliability of the current "test"
predicts the reliability of a new test by
replicating the current test N times creating a
test with N parallel forms of the current exam.
Thus N  2 implies doubling the exam length by
adding items with the same properties as those
in the current exam.
22
Why BN Works Well

• Offers significant improvement over number right
  scoring
• Bayes network estimates stabilize sub scores by
  borrowing strength from the overall reliability
• Differs from other methods b/c it starts with an
  expert constructed model of how the proficiencies
  interact
• Other methods use observed correlations b/t the
  scores on subtest