Seed Generation and Seeded Version Space Learning

1
Seed Generation and Seeded Version Space Learning

• Version 0.02
• Katharina Probst
• Feb 28,2002

2
Seed Generation
3
Clustering

• Seed rules are clustered into groups that
  warrant attempt to merge
• Clustering criteria POS sequences, Phrase
  information, Alignments
• Main reason for clustering divide the large
  version space into a number of smaller version
  spaces and run the algorithm on each version
  space separately
• Possible danger Rules that should be considered
  together (such as the man, men) will not be

4
The Version Space

• A set of seed rules in a cluster defines a
  version space as follows The seed rules form
  the specific boundary (S). A virtual rule with
  the same POS sequences, alignments, and phrase
  information, but no constraints forms the general
  boundary (G)

G boundary virtual rule with no constraints
Generalizations of seed rules, less specific
than rule in G
S boundary seed rules
5
The partial ordering of rules in the version space

• A rule TR2 is said to be strictly more general
  than another rule TR1 if the set of f-structures
  that satisfy TR2 are a superset of the set of
  f-structures that satisfy TR1. It is said to be
  equivalent to TR1 if the set of f-structures that
  satisfy TR1 is the same as the set of
  f-structures that satisfy TR2.
• We have defined three operations that move a
  transfer rule to a strictly more general rule

6
Generalization operations

• Operation 1 delete value constraint, e.g.
• ((X1 agr) 3pl) ? NULL
• Operation 2 delete agreement constraint, e.g.
• ((X1 agr) (X2 agr)) ? NULL
• Operation 3 merge two value constraints to an
  agreement constraint
• ((X1 agr) 3pl) , ((X2 agr) 3pl)
• ? ((X1 agr) (X2 agr))
• Note if the first index is an X index and the
  second a Y index, this operation should only be
  performed if the feature is in the list of
  projecting features

7
Merging two transfer rules

• At the heart of the seeded version space
  learning algorithm is the merging of two transfer
  rules (TR1 and TR2) to a more general rule (TR3)
• Insert into TR3 all constraints that are both in
  TR1 and TR2 and remove them from TR1 and TR2.
• Perform all instances of Operation 3 on TR1 and
  TR2 separately.
• Repeat step 1.
• Note Operation 1 and Operation 2 are executed
  implicitly.

8
Seeded Version Space Algorithm

• Remove duplicate rules from the S boundary
• Try to merge each pair of transfer rules
• A merge is successful only if the CSet of the
  merged rule is a superset of the union of the
  CSets of the two unmerged rules, where the CSet
  of a rule denotes the set of training sentences
  that are covered, i.e. translated correctly by
  the rule
• Pick the successful merge that optimizes an
  evaluation criterion
• Repeat until no more merges are found

9
Evaluating a set of transfer rules

• Initial thought evaluate a set based on
• The coverage of its rules, i.e. the union of its
  CSets
• The size of the rule set
• Goal maximize coverage and minimize set size
• Currently merges are only successful if there is
  no loss in coverage, so size of rule set only
  criterion used
• Future(1) Coverage should be measured on a test
  set
• Future(2) Relax the constraint that a successful
  merge cannot result in loss of coverage

10
Next steps

• Compositionality, integration with transfer
  engine
• Exploring the space below the seed rules
• Specializing we do not want a merge to be a
  final decision, want to allow for a rule to be
  lowered to a more specific rule
• What is the right inductive bias?