Learning and Inference for Extended Subsumption

1
Learning and Inferencefor Extended Subsumption

• Presented by Vasin Punyakanok

2
The General Inference Problem

• Given two structures (concept graphs)
• With External Knowledge
• Conclude if one can entail the other.

3
Entailment in Question Answering

• Q What museum is directed by Henry Hopkins?
• A Henry Hopkins is the director of the
  UCLA/Armand Hammer Museum.
• Q Some museum is directed by Henry Hopkins.
• Answer entails Question

4
Problem

• Q What museum is directed by Henry Hopkins?
• A Henry Hopkins is the director of the
  UCLA/Armand Hammer Museum.

Knowledge (Rules)
??
5
Extended Subsumption

• Exact Subsumption S1 ?e S2 S1 contains S2

?e
6
Extended Subsumption

• S1 ? S2 If there exists S1 such that S1 ? S1 ?e
  S2,

?
7
Extended Subsumption

• S1 ? S2 If there exists S1 such that S1 ? S1 ?e
  S2,

?e
8
Extended Subsumption

• We do not do any substitution, but only add
  information

?
?e
?
9
Extended Subsumption

• Q What are the sexual discrimination allegations
  Morgan Stanley will fight against on July 7th?
• A Wall Street brokerage Morgan Stanley will
  defend itself on Wednesday against accusations it
  denied women promotions, allowed sexual grouping,
  office strip shows, and other forms of sexual
  discrimination.

10
Extended Subsumption
11
Extended Subsumption

• S1 ? S2 if there exists a PROOF (sequence of rule
  applications) such that S1 PROOF(S1) ?e S2
• If there are several proofs, choose the optimal
  one

12
Combinatorial Optimization

• Given S1 and S2
• Given a set of rules, each alter the structure.
• Find set/sequence of rules PROOF altering S1
  such that PROOF(S1) ?e S2 optimizing some
  objective function
• minPROOF ?r?PROOF wrr

13
Integer Linear Programming

• A set of Boolean variables, v1, v2, ,vn
• Each variable vi is associated with a weight wi
• Find an assignment to variables minimizing
• ?i wivi
• Subject to linear constraints
• ?i aivi gt b

14
Integer Linear Programming Formulation

• Two types of variables
• Variables representing graphs

direct_N1 HenryHopkins_N2 museum_N3 subject_N1_N2
Object_N1_N3
N1
N2
N3
15
Integer Linear Programming Formulation

• Two types of variables
• Variables representing graphs
• Variables representing rule applications

?
Rule1N1_N2_N3_N4_N5N6
16
Integer Linear Programming Formulation

• Given S1 and S2, apply rules to S1 up to some
  limit ? S1.
• Set up variables
• Represent S1
• Represent the rules
• Let ILP find the assignment for these variables
• Constraints!

17
Integer Linear Programming Formulation

• Constraints
• Ensure the rule are applicable
• RuleiNNNN ? Preconditions
• RuleiNNNN ? Effects
• Ensure that attributes and roles that are not in
  S1 cannot be true arbitrarily, but by some rules
• Attribi_Nj ? ? RulekNNNN that causes Attribi_Nj
• Rolei_Nj_Nk ? ? RulelNNNN that causes
  Rolei_Nj_Nk
• Ensure that the solution contains S2
• A matching problem
• Additional Constraints can also be incorporated

18
Learning

• Positve Example (), S1 ? S2
• There exists a proof with cost lt ?
• Negative Example (-), S1 ? S2
• There is no proof with cost lt ?
• Ideal Case
• An example (S1, S2, -/ with optimal proof )

19
Learning Costs

• Online Learning Algorithm
• An example (S1, S2, -/ with optimal proof )
• For example
• Find the cost of the proof
• if cost gt ?, mistake. Update the costs.
• For - example
• Find the minimal proof
• if cost lt ?, mistake. Update the costs.

20
Learning Costs

• Realistic Case
• Example (S1, S2, /-)
• Truncated EM Style Learning Algorithm
• Find the minimal cost proof for S1 ? S2
• For example
• if cost gt ?, mistake. Update the weights.
• For - example
• if cost lt ?, mistake. Update the weights
• Difference from the ideal case is in the positive
  example as the proof we update might not be the
  optimal proof.

21
Conclusion

• Introduced the extended subsumption
• Proposed an approach to do the inference with ILP
• Formed a learning problem
• Suggested learning algorithms
• Investigate the proposed idea

22
Thank you
23
Extended Subsumption

• In general adding information to a concept graph
  makes the graph more specific.

?
24
Extended Subsumption

• But not with subsumption rules because we add
  more general information

?
?e
?
????
?
25
Integer Linear Programming Formulation

• Matching Problem
• Define additional matching variables Match_X_Y
  meaning node X in S1 is matched to Y in S2
• Contraints
• X can match Y if X has all attributes of Y
• Ensure the matching of roles
• If there is a role between two nodes in S2, there
  must also be similar role between two nodes they
  match in S1
• Each node in S2 must match one and only one node
  in S1