Interaction techniques that leverage implicit input

1
Context-Aware Computing

• Interaction techniques that leverage implicit
  input

2
Agenda

• Questions
• Introductions of visitors
• Relaxing of syllabus
• Video DiamondTouch
• Review of Pebbles
• Adding more input to handhelds
• Hinckley et al. UIST 2000 (video)
• Looking more generally at context
• Salber et al. CHI 99 Friday

3
Video

• DiamondTouch
• http//www.merl.com/projects/DiamondTouch/
• P. Dietz and Darren Leigh. DiamondTouch a
  multi-user touch technology. UIST 2001, pp.
  219-226.
• We have one of these units at the Aware Home.

4
Videos

• MERL Real-Time Audio Buffering for Telephone
  Applications
• UIST 2001
• Sensing techniques for mobile interaction
• UIST 2000
• Manipulable interaction from earlier CHI?

5
Pebbles http//www-2.cs.cmu.edu/pebbles/

• PDAs for Entry of Both Bytes and Locations from
  External Sources
• Relationship between handhelds and desktop
• MultiMachine User Interfaces
• See Jeff Pierce

6
How it Works

• See Figure 5 in paper

7
What it Can Do

• Sharing control/data between public and private
  displays
• SlideCommander
• Two-handed input
• More effective scrolling
• Groupware
• Electronic quizzes
• Remote control
• Personal Universal Controller (PUC), UIST 2002
• Semantic snarfing, Ubicomp 2001

8
Questions to Ponder

• How can you provide a variety of interactions on
  a handheld device with such limited input?
• How might your phone automatically switch to
  appropriate ringing modes?

9
Answer

• Add more input to handheld devices.
• Place simple sensors on a small device so that
  physical interactions with the device can be
  turned into input.

10
Examples

• What can be sensed?
• Tilt, touch (handed-ness), proximity, page flick,
  temp, light, sound, orientation
• Together, these kinds of inputs are referred to
  as context.
• What can be done?
• Menus, scrolling, alert to turn on, determine
  status of owner

11
Some key references

• J. Rekimoto. Tilting operations for small screen
  interfaces. UIST 96. TechNote, pp. 167-168.
• B. Harrison, K. Fishkin, A. Gujar, C. Mochon R.
  Want. Squeeze me, hold me, tilt me! An
  exploration of manipulative user interfaces. CHI
  98 pp. 17-24.
• A. Schmidt, K.A. Aidoo, A. Takaluoma, U. Tuomela,
  K. Van Laerhoven and W. Van de Velde. Advanced
  Interaction in Context. Proc. of First
  International Symposium on Handheld and
  Ubiquitous Computing (HUC99), Karlsruhe, Germany,
  September 1999, LNCS 1707, Springer-Verlag, pp.
  89-101.

12
Paper you read

• K. Hinckley, J. Pierce, M. Sinclair, E. Horvitz.
  Sensing techniques for mobile interaction. UIST
  2000, pp. 91-100.
• Best Paper for UIST 2000
• Video

13
Paper Organization

• Motivation
• Background
• Sensor platform
• Software architecture
• Interaction techniques

14
The Platform
15
What Can Be Sensed?
16
Software Architecture

• How is sensor data delivered to apps?

17
Interaction Techniques

• Voice memo detection
• Display mode detection
• Tilt scrolling
• contrast compensation
• interaction with display mode
• Power management

18
Beyond handhelds

• Handling context-awareness in the more general
  sense.
• D. Salber, A. Dey G. Abowd. The Context
  Toolkit Aiding the development of
  context-enabled applications. CHI 99, pp.
  434-441.