Studying The Use of Handhelds To Control Smart Appliances

1
Studying The Use of Handhelds To Control Smart
Appliances
Jeffrey Nichols Carnegie Mellon University May
19, 2003
2
The Problem

• Appliances are too complex

3
The Problem, cont.

• Each complex appliance has its own idiosyncratic
  interface!
• Home and Car Stereos
• Car Navigation Systems
• Answering Machines
• Increasingly Computerized
• Low Usability

4
Our Solution

• Separate the interface from the appliance!
• Key Features
• User interface-independent appliance
  specification
• Automatic generation of GUI and speech interfaces

5
Benefits of Our Approach

• Handheld has richer interface technology than
  appliance can afford
• Color LCD screen, touch screen, text entry
  technology
• More effort can be put into interface design
  technology
• Appliance manufacturers must weigh trade-offs
  between usability, cost, time-to-market, etc.
• Two-way communication channel
• Better feedback can be provided to the user
  regarding the appliances state.

6
Automatic Generation of UIs

• Benefits
• All interfaces consistent for the user
• With conventions of handheld
• Other applications and UI guidelines
• Even from multiple manufacturers
• Addresses idiosyncracy problem!
• Multiple modalities (GUI Speech UI)
• Can take into account user preferences
• Will work on special purpose devices (for
  disabled)

7
Outline

• A First Step
• User Studies
• Current Work

8
A First Step

• Build Reference Interfaces
• Remote control interfaces for various appliances
  that we design manually.
• Verify that better interfaces can be created on a
  handheld
• Used for understanding what functional knowledge
  is necessary to make a good interface.

9
Reference Interfaces

• Interfaces were hand-designed for two appliances
  and two handhelds
• Appliances
• AIWA CX-NMT70 Shelf Stereo
• ATT 1825 Telephone/Answering Machine
• Handhelds
• Palm
• Microsoft PocketPC

10
Palm Interfaces

• We initially designed paper-prototype interfaces
  for Palm

telephone
stereo
11
PocketPC Interfaces

• We later implemented interfaces for Microsofts
  PocketPC (simulated remote control).

telephone
stereo
12
Interface Quality?

• We iteratively improved the interfaces using
  heuristic analysis techniques.
• We conducted a think-aloud study with several
  Carnegie Mellon students to find problems in the
  interfaces.
• Lastly, we conducted a user study that compared
  our reference interfaces with the manufacturers
  interfaces.

13
Outline

• A First Step
• User Studies
• Current Work

14
User Studies

• Two Studies
• Study 1
• Paper-Prototype Palm vs. Actual Appliance
• Study 2
• Functional PocketPC vs. Actual Appliance

15
User Studies, cont.

• Procedure
• We did a between-subjects study.
• Each subject worked on two sets of tasks.
• In order to minimize subjects, each worked on
  both the stereo and the phone.
• We controlled for order and interface type.

16
Evaluation of Task Performance

• Three Metrics
• Time to complete all tasks
• Number of times help was requested
• How often did the subject need the manual or
  online help?
• Number of missteps
• Misstep the pressing of a button that does not
  advance the progress on the current task
• No missteps were counted after the user requested
  help.

17
User Study 1 PalmOS

• Compared paper prototype interfaces with the
  interfaces of the actual appliances
• Experimenter changed paperas subjects tapped
• Control of stereo and phone simulated verbally
• When the stereo started playing music, the
  experimenter said you now hear music from the
  stereo

18
User Study 1, cont.

• Participants
• 13 Carnegie Mellon Graduate Students
• Five female, Eight male
• Enrolled in School of Computer Science
• Volunteers (unpaid)
• Seven owned a Palm device
• One had no Palm experience
• Four owned Aiwa-brand stereo systems

19
User Study 1 Results

• Users made five times the errors and needed help
  twice as often with the actual appliances!
• All results significant (p lt 0.001
  for all)

20
User Study 2

• We implemented the interfaces on a handheld and
  simulated remote control of an actual appliance.
• Remote control applications built in Visual Basic
  on a PocketPC
• Control of stereo and phone simulated in software
• Feedback appeared to come from the actual
  appliance

21
User Study 2, cont.

• Participants
• Twelve students from Carnegie Mellon
• Four female, Eight male
• Volunteered in response to a newsgroup
  advertisement
• Paid 7 for their participation (30-45 minutes)
• All had limited handheld experience
• Half (6) owned Aiwa-brand stereos
• Two had ATT digital answering machines

22
User Study 2 Results

• All differences are significant (p lt 0.05)

• About ½ the time and ½ the errors!

23
Qualitative Results

• Why were the reference interfaces better?
• Clear feedback and explanation of the current
  state was possible.
• Elements could be disabled on the screen (graying
  out)
• Functions were separated across multiple screens.

24
Outline

• A First Step
• User Studies
• Current Work

25
Current Work

• Designed a XML-based Specification Lang
• Functions of Device
• State Variables and Commands
• Labeling
• Multiple labels are necessary
• Grouping
• Hierarchical groups
• Dependency Information
• For enabling and structure

26
Current Work, cont.

• Built multiple automatic interface generators
• PocketPC
• SmartPhone
• Tablet PC (desktop)
• Speech

27
Control of Real Appliances
28
Acknowledgements

• Funding
• National Science Foundation
• Microsoft
• General Motors
• Pittsburgh Digital Greenhouse
• Equipment Grants
• Mitsubishi (MERL)
• VividLogic
• Hewlett-Packard

• PUC Project Members
• Brad A. Myers
• Thomas K. Harris
• Roni Rosenfeld
• Michael Higgins
• Joseph Hughes
• Kevin Litwack
• Rajesh Seenichamy
• Mathilde Pignol
• Stefanie Shriver
• Jeffrey Stylos
• Peter Lucas

29
Thanks!

• For more information see
• http//www.cs.cmu.edu/jeffreyn/
• http//www.cs.cmu.edu/pebbles/puc/
• Or e-mail me at
• jeffreyn_at_cs.cmu.edu

30
(No Transcript)
31
Actual Appliance Interfaces

• Lots of Problems
• Poorly labeled and overloaded buttons
• Insufficient feedback
• Timer example
• Programming the speed-dial
• Phone has technical separation between phone and
  answering machine

32
Qualitative Results

• Grouping controls is important
• Groups define which elements are placed adjacent
  to each other and how elements are separated onto
  pages.
• Groupings vary between devices and interface
  styles.

33
Qualitative Results, cont.

• Dual-associated functions are hard to make
  obvious for users
• The record button is associated with both tapes
  (record onto) and each of the other modes
  (recorded from).
• Some users expected the first mapping to used,
  whereas the controller used the second mapping.

34
Qualitative Results, cont.

• Tree-based structures are not sufficient for
  fully describing an interface
• Some interface concepts, especially
  dual-associated functions, break the tree because
  they may interact with the children of several
  different elements within the tree
• The record button breaks the stereos tree
  structure because it is globally accessible but
  has different local effects.

35
Qualitative Results, cont.

• A single function may map to multiple interface
  widgets (and vice versa)
• Example One state variable could be used to
  represent all of the playback states of a tape
  player. The play, stop, fast-forward, and rewind
  buttons all act on this single variable.

36
Applying These Results

• We are actively applying these results to the
  design of the specification language
• A tree-grouping structure is augmented with a
  dependency graph to help describe dual-mapped
  functions
• Ranking relationships within groups using
  priorities
• We will also apply them in the design of the
  automatic layout engine

37
Future Work

• Build the specification language and automatic
  generation engine

38
A Hard Problem

• Automatically generating a good user interface is
  hard, but we think we can do it for several
  reasons
• Remote controls are a special class of user
  interface that use relatively simple interaction
  techniques.
• Buttons, text fields, and other standard widgets.
• Our approach differs from earlier work

39
The Approach

• Study Interfaces
• Functional knowledge of the appliance
• What must the appliance tell the handheld about
  itself so that a good interface can be
  constructed.
• Design and Layout
• How do we turn the knowledge about the appliance
  into a usable interface?
• Design a specification language
• Build an automatic interface generator

40
Our Progress

• Study Interfaces
• Functional knowledge of the appliance
• What must the appliance tell the handheld about
  itself so that a good interface can be
  constructed.
• Design and Layout
• How do we turn the knowledge about the appliance
  into a usable interface?
• Design a specification language (in progress)
• Build an automatic interface generator

41
Problems with User Study 1

• Paper-prototype study introduced a high
  possibility of experimenter interference.
• Solution
• Create an environment that completely simulates
  what one might experience using a personal
  universal controller
• Interfaces running on an actual handheld
• Interfaces should appear to control an actual
  appliance