Discriminative Modeling extraction Sets for Machine Translation

1
Discriminative Modeling extraction Sets for
Machine Translation

• Author
• John DeNero and Dan Klein UC Berkeley
• Presenter
• Justin Chiu

2
Contribution

• Extraction set
• Nested collections of all the overlapping phrase
  pairs consistent with an underlying
  word-alignment
• Advantages over word-factored alignment model
• Can incorporate features on phrase pairs, more
  than word link
• Optimize a extraction-based loss function really
  direct to generating translation
• Perform better than both supervised and
  unsupervised baseline

3
Progress of Statistical MT

• Generate translated sentences word by word
• Using while fragments of training example,
  building translation rules
• Aligned at the word level
• Extract fragment-level rules from word aligned
  sentence pair
• Tree to string translation
• Extraction Set Models
• Set of all overlapping phrasal translation rule
  alignment

4
Outline

• Extraction Set Models
• Model Estimation
• Model Inference
• Experiments

5
Extraction set models
6
Extraction Set Models

• Input
• Unaligned sentence
• Output
• Extraction set of phrasal translation rules
• Word alignment

7
Extraction Sets from Word Alignments

•  

8
Extraction Sets from Word Alignments

•  

9
Extraction Sets from Word Alignments
10
Possible and Null Alignment Links

• Possible links has two types
• Function words that is unique in its language
• Short phrase that has no lexical equivalent
• Null alignment
• Express content that isabsent in its translation

11
Interpreting Possible and Null Alignment Links

•  

12
Interpreting Possible and Null Alignment Links
13
Linear Model for Extraction Set

•  

14
Scoring Extraction Sets

•  

15
Model Estimation
16
MIRA(Margin-infused Relaxed Algorithm)

•  

17
Extraction Set Loss Function

•  

18
Model Inference
19
Possible Decompositions
20
DP for Extraction Sets

•  

21
DP for Extraction Sets
22
Finding Pseudo-Gold ITG Alignment

•  

23
Experiments
24
Five systems for comparison

• Unsupervised baseline
• Giza
• Joint HMM
• Supervised baseline
• Block ITG
• Extraction Set Coarse Pass
• Does not score bispans that corss bracketing of
  ITG derivations
• Full Extraction Set Model

25
Data

• Discriminative training and alignment evaluation
• Trained baseline HMM on 11.3 million words of
  FBIS newswire data
• Hand-aligned portion of the NIST MT02 test set
• 150 training and 191 test sentences
• End-to-end translation experiments
• Trained on 22.1 million word prarllel corpus
  consisting of sentence up to 40 of newswire data
  from GALE program
• NIST MT04/MT05 test sets

26
Results
27
Discussion

• Syntax labels v.s words
• Word align to rule ? Rule to word align
• Information from two directions
• 65 of type 1 error