Microsoft Research Faculty Summit 2003

1
Microsoft Research Faculty Summit 2003

• Brad A. Myers
• Mobility Handhelds for Universal Remote Control
• Pebbles Research Project
• Human Computer Interaction Institute
• Carnegie Mellon University
• http//www.pebbles.hcii.cmu.edu

2
Pebbles Project

• Use handheld devices with other technology
• Most other projects looking at handhelds to
  replace other technology
• Examples
• PocketPCs with a PC
• SmartPhone with appliances

3
Example Applications Meetings

• Remote Commander
• Use PocketPCs askeyboard and mouseso everyone
  canparticipate
• Mouse or Draw
• Laser Pointer for interaction at adistance

4
Demo 1Slide Show Commander

• Released 1999
• Remote Control of PowerPoint
• Two-way communication
• Picture of slide, notes, titles, other
  applications, preview
• Scribble, click on links

5
Slide Show Commander, 2

• List of applications
• Preview of other slides

6
Example Applications Individuals in an Office

• Shortcutter
• User-created panels of controls
• Create custom interfaces and extensions to PC
  applications
• Scrolling with left hand on PDA

7
Example Applications People with Disabilities

• Use handhelds as interface to PC and appliances
  for people withneuro-muscular disabilities
• New input methods

8
Example Applications In Classrooms

• Handhelds for testing and note taking

9
Example Applications In Homes

• Use handheld for remote control of appliances
• Personal Universal Controller (PUC)
• Help solve these problems

10
Personal Universal Controller

• Different from so-called universal remotes
• Two-way communication
• Appliance describes its functions
• Handheld PUC
• Automatically creates interface
• Controls the appliance
• Displays feedback about appliance status

Specifications
Control
Feedback
11
Research Challenges

• Automatic generation of high-quality user
  interfaces
• For PocketPCs, TabletPCs, SmartPhone
• Also for speech interfaces
• XML-based specification describes appliance
  functions and dependencies
• Rule-based automatic design
• Connecting with real appliances
• Custom and standard protocols

12
Demo 2Automatic Remote Control Generation

• Control of
• Axis Pan-Tilt-Zoom Camera
• UPnP interface
• Windows Media Player
• Using
• PocketPC
• Microsoft SmartPhone
• TabletPC
• Programmed using C with .NET Compact Framework

13
Example Screens

• Axis Camera Windows Media Player

14
See More About This Work

• At Microsoft Faculty Summit
• Use of Compact FrameworkTuesday, 1130-1230,
  St. Helens
• DemoFest lots more devices Tuesday, 100-230
• Download software and papers http//www.pebbles.
  hcii.cmu.edu