Extracting Refined Rules From KBNN

1
Extracting Refined Rules From KBNN
Initial Symbolic Knowledge
Trained Neural Network
Initial Neural Network
Refined Symbolic Knowledge
Rules to network
Neural Learning
Network to rules
2
Translation of a knowledge base into a KNN
A- B, C.
B- not H. B- not F,
G. C- I, J.
A
B
C
F G H
I J
3
M of N Algorithm

• Clustering
• With each hidden and output unit, form groups of
  similarly weighted links.
• Averaging
• Set link weights of all group members to the
  average of the group.
• Eliminating
• Eliminate any groups that do not significantly
  affect whether the unit will be active or
  inactive.
• Optimizing
• Holding all link weights constant, optimize
  biases of all hidden and output units using the
  back propagation algorithm
• Extracting
• Form a single rule for each hidden and output
  unit. The rule consists of a threshold given by
  the bias and the weighted antecedents specified
  by the remaining links
• Simplifying
• Where possible simplify rules to eliminate
  superfluous weights and thresholds.

4
M of N example
INITIAL UNIT
AFTER STEPS 1 AND 2
5
M of N Example Contd/-
If 6.1numberTrue(A, C, F) gt 10.9 then
Z NumberTrue returns the number of true
antecedents.
AFTER STEP 4 and 5
If 2 of A C F then Z
AFTER STEP 3
AFTER STEP 6
6
Specification of SUBSET algorithm

• With each hidden and output unit
• Extract up to ßp subsets of the positively
  weighted incoming links whose summed weight is
  greater than bias on the unit.
• With each subset P of the ßp subsets found in
  step 1
• 2.1 Extract up to ßp minimal subsets of
  negatively weighted links
• whose summed weights is greater than the
  sum of P less the
• bias on unit
• 2.2 Let Z be a new predicate used
  nowhere else.
• 2.3 With each subset N of the ßp
  .subsets found in step 2.1 form the
• rule If N then Z
• 2.4 Form the rule If P and not Z then
  ltname of unitgt.

7
Example Of SUBSET Algorithm
If B, C and not (E) then A. If B, D and not
(E) then A If C, D and not (E) then A If B,C,D
then A
8
Initial KNN for Promoter Recognition
DNA Sequence
9
Some Initial Rules for Promoter Recognition

• promoter - contact, conformation.
• contact - minus-35, minus-10.
• minus-35 - _at_-37 CTTGAC-.
• minus-35 - _at_-37 -TTGACA.
• minus-10 - _at_-14 TATAAT--.
• minus-10 - _at_-14 -TA-A-T-.
• conformation - _at_-45 AA--A.
• conformation - _at_-45 A---A,
• _at_-28
  T---T-AAT-,
• _at_-04 T.

10
Some Promoter rules extracted by M of N

• promoter minus-35, minus-10.
• minus-35 -10 lt 4.0nt(_at_-35 TTGAT-)
• 1.5nt(_at_-35 TCC-) 0.5nt(_at_-35
  -HC---)- 1.5nt(_at_-35 GGAGG-).
• minus-10 _at_-13 TAWA-T.
• Minus-10 2 of _at_-12 -CA---T and
• not (1 of _at_-12
  -RB---S).

11
Some Promoter rules extracted by SUBSET Algorithm

• promoter - minus-35, minus-10.
• minus-35- minus-35b, minus-35d
• minus-10 - _at_-14 -ATA----.
• minus-10 - _at_-14 -TA-A-.
• minus-35d - _at_-37 -TTC-.
• minus-35b - _at_-37 ---GA.

12
M of N Results
 
 
 
13
SUBSET Algorithm Results