CG007: Advanced HCI

1
Mobile computing
www.wfu.edu/djanders/ labweb/PDA.htm

• CG007 Advanced HCI

2
Interaction with mobile devices
Early on in his journey, the ape-descended human
known as Arthur Dent was handed an electronic
device, The Hitchhiker's Guide to the Galaxy,
which allowed him to access all manner of
fascinating facts about the universe at the touch
of a few buttons. http//www.bbc.co.uk/dna/h2g2/
mobile-info
3
Interaction with mobile devices
What physical form did does the Hitchhikers
Guide take and how does Arthur interact with it?

• The Hitchhiker's Guide to the Galaxy
• Secondary Phase (Audio CD) Audiobook

Most of the time it seems to work by pressing a
few buttons and looking and listening the BBC
TV series is particularly clear on this.
Maybe the interface works by adapting to what
seems familiar to the contemporary culture of the
life form that is using it Arthur just thinks
hes tapping on keys and reading from a 2D
display.
4
Dilemma of key-based interaction paradigms
5
3G - Third Generation Wireless Technologies

• 1G
• analogue voice
• 2G
• digital voice, short messaging services
• 3G
• seeks to merge voice and data communications in
  mobile contexts.
• High bandwidth, direct IP access, short response
  times, ability to allow always on-line.
• seeks to merge two interaction paradigms
• Telecom the phone
• Personal computing the pc/ desktop metaphor
• raises issues of usefulness, usability and
  interaction design.

6
Phone interaction paradigm

• Predominantly real-time
• Limited number of keys needed for dialing
• You hold it near your cheek and talk
• you dont need to see the phone while you talk
• You can hardly see the phone while you talk
• Not much needs to be or can be stored
• Limited scope for manipulating stored information

7
Desktop pc interaction paradigm

• Predominantly not real-time
• Large range of complex applications
• Complex ways of manipulating data
• Large, complex file storage and manipulation
• Ubiquitous desktop metaphor GUI

8
Merging paradigms - questions

• Are interactions saved in files (pc) or do they
  only last until I hang up (phone)?
• Do they support an undo function?
• Can some problems by fixed through a restart?
• Can I buy new applications an software (pc) or do
  I have to buy new hardware (phone) ?

9
3G interaction paradigm

• Currently the practice is to switch modes from
  one paradigm to the other depending on how the
  device is being used.
• Advantages
• exploits existing, well-understood rules
• works while device is still seen as 2-in-one
• Disadvantage
• may not facilitate new uses.
• Theory of socio-technical evolution predicts
• new uses will lead to new interaction
  paradigm(s)
• new interaction paradigm(s) will lead to new
  uses.

10
Contemporary examples
11
some contemporary examples

• BBC London on the move http//www.bbc.co.uk/lon
  don/content/articles/2005/08/10/digitalguide_mobil
  e.shtml
• Information services on the move
  http//www.bbc.co.uk/mobile/web/index.shtml
• Data gathering on the move http//www.bbc.co.uk/n
  ature/animals/wildbritain/springwatch/record/mobil
  e.shtml
• Travel information on the move
  http//www.tfl.gov.uk/tfl/mobile/mobile.asp?cntex
  amples

12
New interaction paradigms and new uses

• Work-oriented perspective
• Ways of categorising mobility
• Mobility of the device
• Mobility of user (no specific destination,
  visiting pre-arranged locations)
• Mobility of the activity using while moving is
  the primary factor in influencing interaction
  design (Pascoe, J., Ryan, N. and Morse, D. (2000)
  Using while moving HCI issues in fieldwork
  environments. Transactions on Computer-Human
  Interaction. ACM. Vol.7, no.3, 417-437.)

13
New interaction paradigms and new uses

• Example ecologists observing giraffe in Kenya
• Characteristics
• Dynamic user configuration need to collect data
  as opportunity arises whether they are standing,
  walking or crawling.
• Limited attention capacity observation of
  wildlife may require concentration over extended
  periods, in these situations the user needs to
  spend as much time as possible observing and to
  minimize the time devoted to interacting with the
  recording mechanism.
• High speed interaction observation is
  event-driven, it may be necessary to enter high
  volumes of data in a sudden burst of activity.
• Context dependency observation is bound up with
  context, when and where an observation took place
  for example is of vital importance.

14
New interaction paradigms and new uses

• Design implications
• Automatic context-sensing global positioning,
  time-of day.
• Handwriting with stylus preferable to typing on
  keyboard, but it still requires two hands and
  attention to interface.
• Minimal Attention User Interface (MAUI)
• Tries to allow other senses to be exploited
• e.g. audible or tactile feedback
• Simplifies interaction task
• E.g. by allowing pre-definition of counting,
  which is then incremented by a single button.
• Touch-sensitive screen divided into quadrants,
  allows one-handed operation and can require only
  occasional visual attention.

15
Physical form of interactions
16
The latest mobile computing devices?

• Pager
• Receive only
• Tiny displays
• Simple text messages
• Personal Digital Assistants (PDAs)
• Simple graphical displays
• Character recognition
• Simplified WWW

• Mobile phones
• Voice, data, images
• Simple text or graphical displays
• Palmtop
• Tiny keyboard
• Downgraded versions of standard applications
• Laptop
• Functionally equivalent to desktop
• Standard applications

Things move quickly these illustrations look
very dated, but they were the latest gadgets only
a few years ago.
17
ultra-mobile pc

• Origami project defines form factors (the
  physical style) that characterise devices that
  Microsoft intend to call ultra-mobile pcs.
• Microsoft today unveiled details for
  Ultra-Mobile Personal Computers (UMPCs)
• a new category of mobile computing devices that
  features small, lightweight, carry-everywhere
  hardware designs
• coupled with the full functionality of a
  Microsoft Windows-based PC and a choice of input
  options, including enhanced touch-screen
  capabilities.
• http//www.microsoft.com/presspass/features/2006/m
  ar06/03-09Mobile.mspx

18
ultra-mobile pc form factors demonstrated
19
Design guidelines for ultra-mobile pc.

• Examples of guidelines issued by Microsoft for
  developers adhering to ultra-modile pc
  criteria
• Users may point with either a finger or the pen
  tip.
• Display resolution is small. In some cases items
  such as dialogs may be restricted by the screen
  size.
• The size of selection target areas on smaller
  devices is crucial.
• Scrolling up, down, left, and right makes an
  application hard to use. Most UMPC applications
  will probably run in full screen mode and be
  formatted to support minimal scrolling.
• The UMPC does not normally generate hover events.
  Because these are touch devices, you cannot tell
  when the pen is getting close to the screen, as
  you can with an electromagnetic digitizer common
  in most Tablet PCs today.
• The edges of the screen are difficult to target.
  The bottom of screen is especially hard to
  target. Make sure your customers can get to the
  UI they need to activate.

http//msdn.microsoft.com/mobility/tabletpc/umpc/d
efault.aspx
20
Multimodal applications

• one-handed and hands-free operation,
• pen input enables handwriting, gestures, drawings
  and specialized notations.

21
Multimodal interaction

• Speech might be an important alternative channel
  for interacting with smaller devices.
• Multimodal auto-fill assists you in entering form
  input information. By extending the auto-fill
  capabilities of a visual browser with voice
  interaction, a mobile user can easily add name,
  address, and other personal data, also known as
  user preferences, to form input fields.

http//www-128.ibm.com/developerworks/library/wi-m
obweb/
22
Emerging multimodal design principles example

• Multiple Modalities Need To Be Synchronized -
• Spoken interaction is highly temporal,
• whereas visual interaction is spatial.
• When combining these modes of interaction in a
  multimodal interface, synchronization is a key
  feature that determines overall usability of the
  interface.

More
23
Emerging multimodal design principles example

• Point And Talk - User points at a location on the
  map while speaking a question.
• Redundant Confirmation -The user interface
  supplements visual output with a spoken
  confirmation for example, a travel reservation
  system might visually highlight the user's
  selection while speaking an utterance of the form
  e.g Leaving from San Francisco
• Unless synchronized, such supplementary use of
  modalities can significantly increase the
  cognitive load experienced by the end-user and
  prove a source of confusion.
• Parallel Communication - The user interaction
  leverages the availability of multiple streams of
  output to increase the band-width of
  communication. For example, a travel reservation
  system might visually present a list of available
  flights and speak a prompt of the form There
  are 7 flights that match your request, and the
  flight at 830am appears to be the most
  convenient.