Multimodal Interfaces for Emergency Management Systems

1
Multimodal Interfaces for Emergency Management
Systems

• M. Gemo, T. Amouh, A. El Gariani, B. Macq, J.
  Vanderdonckt

2
ARTHUR prototype
A Web-based Clinical Information System

• Co-operation through the whiteboard
• Electronic Health Records Handling
• GEHR based emergency service concepts

3
Multimodal Applications
InfoWiz kiosk from SRI Cheyer99
QuickSet from OGI Cohen99

• Using natural input
• Speech, pen, gesture

eClassroom from Georgia Tech Abowd99

• Using rich output
• Visual, audio, augmented display,
• animated characters

CARS navigation assistant from SRI Julia99

• Using different devices

More choices more complexity
4
What for effective seamless Interaction?
User centred Design combining the right pieces
Avoid unnecessary Complexity!!
Data
UserTask
Environment
Device
Ergonomic,Usabilityproperties
5
ARTHUR scenario
UserTask

• Input Interaction
• WIMP,Tactile, Speech
• Output Interaction
• Visualisation

• 2/3 sec for interaction
• Doctors, nurses
• Patient take-care
• Cooperation

• Limited space
• Stress
• Crowded/overbusy
• Noisy
• Unbounded service

• Text
• Graphics
• Animations

Environment

• PDA/Tablet PC
• Desktop
• Wall Display

• Traceability
• Evolvibility

6
Multimodal Output Walldisplay Whiteboard
7
An issue Consultation Summary
Vital Parameters
Clinical Tests
8
Multimodal Input speech
Which Device? Which User? What Task?
Data entry on PDAs scrolling paging

• Speech as
• Aid in preselection
• Mixed intiative dialogues
• Complementary to text autocompletion
• Speech alone not enough
• Inter-modality switching mandatory
• Recognition techniques not 100 accurate

9
Speech Input tests
10
Conclusions

• Added value with Multimodality
• User centred design
• Speech ???