Tree-edit CRFs for RTE

1
Tree-edit CRFs for RTE

• Mengqiu Wang and Chris Manning

2
Tree-edit CRFs for RTE

• Extension to McCallum et al. UAI2005 work on CRFs
  for finite-state String Edit Distance
• Key attractions
• Models the transformation of dependency parse
  trees (thus directly models syntax), unlike
  McCallum et al. 05, which only models word
  strings
• Discriminatively trained

3
TE-CRFs model in details

• First of all, lets look at the correspondence
  between alignment (with constraints) and edit
  operations

4
root
root
Q
A
substitute
root
root
met VBD
is VB
substitute
subj
obj
subj
with
Bush NNP person
Jacques Chirac NNP person
who WP qword
leader NN
insert
det
of
Fancy substitute
nmod
president NN
the DT
France NNP location
substitute
delete
nmod
French JJ location
substitute
5
TE-CRFs model in details

• Each valid tree edit operation sequence that
  transforms one tree into the other corresponds to
  an alignment. A tree edit operation sequence is
  models as a transition sequence among a set of
  states in a FSM

D, S, I
D, S, I
D, S, I
S1
S2
D, E, I
D, S, I
S3
D, S, I
D, S, I
substitute
insert
substitute
delete
substitute
substitute







6
FSM
insert
substitute
substitute
delete
substitute
substitute







This is for one edit operation sequence
substitute
delete
substitute
substitute
substitute
insert
insert
substitute
delete
substitute
substitute
substitute
substitute
substitute
delete
substitute
insert
substitute
There are many other valid edit sequences
7
FSM cont.
e
e
Positive State Set
Start
Stop
e
e
Negative State Set
8
FSM transitions
Positive State Set

S1
S1
S2
S3
S2
S3

S3
S3
S2
S3
S1
S2







S1
S1
S2
S2
S2
S2

S2
S3
S3
S2

S1
S3
Stop
Start
Negative State Set

S1
S1
S2
S3
S2
S3

S3
S3
S2
S3
S1
S2







S1
S1
S2
S2
S2

S2
S2
S3
S3
S2

S1
S3
9
What is the semantic interpretation of the FSM
states?

• At this moment since all the states in the FSM
  are all fully-connected, its unclear what they
  mean. We fix the number of states to 3, and
  experiments shows that setting it to 1 or 6 hurts
  performance.
• We are running new experiments with more
  meaningfully designed FSM topologies, e.g., each
  states deterministically corresponds to a
  particular edit operation.

10
Parameterization
substitute
S1
S2
positive or negative
positive and negative
11
Training using EM
Jensens Inequality
E-step
M-step Using L-BFGS
12
Features for RTE

• Substitution
• Same --Word/WordWithNE/Lemma/NETag/Verb/Noun/Adj/A
  dv/Other
• Sub/MisSub -- Punct/Stopword/ModalWord
• Antonym/Hypernym/Synonym/Nombank/Country
• Different NE/Pos
• Unrelated words
• Delete
• Stopword/Punct/NE/Other/Polarity/Quantifier/Likeli
  hood/Conditional/If
• Insert
• Stopword/Punct/NE/Other/Polarity/Quantifier/Likeli
  hood/Conditional/If
• Tree
• RootAligned/RootAlignedSameWord
• Parent,Child,DepRel triple match/mismatch
• Date/Time/Numerical
• DateMismatch, hasNumDetMismatch,
  normalizedFormMismatch

13
Tree-edit CRFs for Textual Entailment

• Preliminary results
• Trained on RTE2 dev, tested on RTE2 test.
• model taken after 50 EM iterations
• acc0.6275, map0.6407
• SUM, acc0.675
• QA, acc0.64
• IR, acc0.615
• IE, acc0.58

14
Work in progress

• Implementing a unordered tree-edit algorithm,
  which would allow swapping of sub-trees
• Use Stanford Parser dependency structure. Need to
  getting rid of cycles in CollapsedDependencyGraph
  (almost there, only have a few self-loops now).
• Experiment with deterministic topologies
• More features!! ?
• Training a separate model for each sub-task (is
  task information given at test time?)