Automatic generation of multimodal interaction models from behavioural data

1
Automatic generation of multimodal interaction
models from behavioural data

• Marie-Luce Bourguet
• Computer Science Dept.
• Queen Mary, University of London

2
Multimodal Interaction Models
Interaction Model
Application
Multimodal Engine

Speech recognition
Gesture recognition
3
Modeling Multimodal Interaction
remove
4
Modeling Multimodal Interaction Synchronisation
patterns
remove
5
Modeling Multimodal Interaction Mutual
disambiguation
6
Design Tool
7
Simulation Tool
8
Automatic Generation of Interaction Models
Interaction Model
Training Module

Speech recognition
Gesture recognition
9
Conclusion

• Building interaction models from user interaction
  data presents numerous advantages
• It bypasses the difficult task of designing
  multimodal interaction models
• The resulting interaction models are highly
  personalised
• They should enable natural and robust interaction
• The FSM based formalism is well adapted because
  very simple analysis of the data should permit
  the automatic generation of interaction models.