SMART Final Review Meeting Conclusions

1
SMART Final Review MeetingConclusions

• Nicola Cancedda
• November 2009

2
Additional highlights
3
Regression-based Translation

• New approach and prototype for translation
• Participation in WMT 07
• Zhuoran Wang, John Shawe-Taylor (UCL) and Sandor
  Szedmak (USOU)
• Addressing ST objective 1 Alternative SMT
  formalization with convex optimization
• Related to Bangalore et al. 2009 and to Sinuhe,
  but different learning algorithm and decoding
• Intrinsic adaptation by preliminary filtering of
  training sentence based on whole source sentence
• Deliverables and publications
• D 2.1 and D 2.2
• NAACL-HLT 2007 Wang et al. 2007
• ACL 2008 ws on SMT Wang and Shawe-Taylor 2008,
• In MIT Press NIPS series edited volume Wang and
  Shawe-Taylor 2009

4
Study of the Learning Curves of Moses

• Large-scale statistical analysis of the behaviour
  of Moses under many conditions
• Does Phrase-Based SMT provide a sufficiently
  large hypotheses class to learn good translators?
• Do we have enough data to accurately estimate
  parameters? If not, will we ever have them?
• How important is it to have accurate feature
  estimates of phrase probabilities vs. e.g. having
  the phrases in the table to start with? How
  robust to noise are established estimators and
  procedures?
• Marco Turchi and Nello Cristianini, UoB
• Contributes to T 2.1 Characterization of current
  approaches
• First study of this kind
• Publications
• ACL 2008 ws on SMT Turchi et al. 2008
• http//videolectures.net/smartdw09_turchi_ltt/

5
Self-training SMT

• Autonomous agent iteratively fetching data from
  the web and retraining
• Marco Turchi, Tijl De Bie and Nello Cristianini
  (UoB)
• Addressing ST Objective 3 on topic-model
  adaptation
• First system to propose an open-world, evolving
  source of training data (the web)
• Integrated in Found in Translation
• Publications
• WI-IAT 2009 Turchi et al. 2009

6
Discriminatively trained lexicalized reordering
model

• Handles phrase reordering as a multi-class
  classification task
• Yizhao Ni, Craig Saunders, Sandor Szedmak and
  Mahesan Niranjan (USOU)
• Improves significantly over existing lexicalised
  distortion model (contributes to ER 2.1)
• Software patch could be integrated in Moses
• Publications
• Paper at ACL-IJCNLP 2009 Ni et al. 2009

7
Phrase-Based decoding and the Generalized TSP

• Establishes isomorphism between decoding for
  phrase-based SMT and the Travelling Salesman
  Problem
• Mikhail Zaslavski, Marc Dymetman and Nicola
  Cancedda (Xerox)
• Decoding faster and/or more accurate than SoA
  using off-the-shelf solvers for TSP
• Publications
• Paper at ACL-IJCNLP 2009 Zaslavski et al. 2009

8
S T Objectives Revisited
9
S T Objectives Revisited

• Problem Mainstream two-layer approach to SMT
  results in tangled and opaque training
• Three new models with clear formalization and
  convex optimization proposed. Two (Sinuhe and
  MMBT) implemented and scaled to large datasets
• Problem SMT trained batch and keeps repeating
  the same mistakes
• Models for adaptive learning in SMT designed,
  implemented (Adaptive PORTAGE), and successfully
  tested

10
S T Objectives Revisited

• Problem SMT/CLTIA requires large amount of
  on-topic training data
• Methods for translation and language model
  combination and adaptation designed and tested
• Autonomous agent training by active learning
  designed and deployed
• Problem typical language models used in SMT are
  not trained directly to improve translations and
  are unable to capture morphological information
• Discriminative and factored language models
  designed and tested

11
S T Objectives Revisited

• Problem Latent Semantic methods for CLTIA do not
  scale up to benefit from large datasets, or from
  three or more aligned languages
• Novel approximations scaling up to large datasets
  and extensions allowing multiple (gt 2) views
  proposed
• Problem most CLTIA is limited to word-by-word
  translation and does not take context into
  account
• Query-adaptation methods developed and proved
  successful in international evaluations
• Problem surface-form methods and latent-semantic
  methods are used alternatively
• Several combinations attempted

12
Conclusions

• We set up an ambitious agenda, and delivered on
  most of it
• Enthusiastic researchers with machine-learning
  background succeeded in identifying interesting
  new approaches
• Experts in machine translation and CLTIA helped
  with channelling efforts and avoiding
  rediscovering the wheel
• Cutting-edge research was integrated in
  prototypes and tested in real-world environments
• In 2005 we wrote a proposal with exactly what we
  felt excited about
• Thanks to the EC for funding it and allowing us
  to do this work

13
The cast

• Xerox
• Pierre Mahé ? bioMérieux (pharma company, France)
• Lucia Specia ? U. of Wolverhampton, UK
• Francois Pacull ? Commisariat á lEnergie
  Atomique, France
• Nicola Cancedda, Isabelle Pené, Jean-Michel
  Renders, Stephane Clinchant, Marc Dymetman, Tamas
  Gaal, Vassilina Nikoulina
• Amebis
• Spela Arhar ? Trojina, Institute for Applied
  Slovene Studies
• Miro Romih and Peter Holozan
• Celer
• Roberto Silva and Sofia Rodrigues Garcia
• JSI
• Dunja Mladenic, Blaz fortuna, Jan Rupnik, Marko
  Grobelnik, Tomaz Erjavec, Simon Krek, Tina Anzic,
  Marko Bohanec, Blaz Novak
• NRC
• Cyril Goutte, George Foster, Pierre Isabelle,
  Roland Kuhn

14
The cast

• University of Bristol
• Marco Turchi ? EC JRC Ispra, Italy
• Nello Cristianini
• University of Milan
• Nicolò Cesa-Bianchi, Gabriele Reverberi
• University of Helsinki
• Kimmo Valtonen, Vladimir Poroshin? m-Brain
  (business intelligence, Finland)
• Matti Kääriäinen ? Nokia
• Juho Rousu, Esther Galbrun, Matti Vuorinen
• University of Southampton
• Craig Saunders ? Xerox
• Sandor Szedmak, Yizhao Ni, Niranjan Mahesan
• University College London
• David Hardoon ? Institute for Infocomm Research,
  Singapore
• John Shawe-Taylor, Zhuoran Wang, Zakria Husain

15
The End