An Intrusive Evaluation of Peripheral Display

1
An Intrusive Evaluation of Peripheral Display

• Xiaobin Shen, Andrew Vande Moere, Peter Eades
• University of Sydney
• National ICT Australia

2
Overview

• Introduction
• Related Work
• Peripheral Display Systems
• Experimental Design
• Results
• Discussion
• Conclusion and Future Work

3
Peripheral Display

• Ambitious Goal
• Present information without distracting or
  overwhelming users
• Limitations
• Limitations of human understanding
• Limitations of current technologies
• Our Aim
• Systematic evaluation of peripheral display

4
Two Evaluation Methods

• Intrusive Evaluation
• Consciously aware of the evaluation experiment
• Focus on usability tests
• Short-term period of time
• Normally evaluated by questionnaires, interview
• Non-Intrusive Evaluation
• Not consciously aware of the evaluation
  experiment
• Focus on actual use in general environment (in
  situ)
• Long-term period of time
• Few successful methods can be applied

5
Advantages

• Present an intrusive evaluation
• Advantages
• Easily conducted
• Reveals results quickly and cheaply
• Easy to adopt existing evaluation methods and
  criteria

6
Related Work Design

• Visual sense
• InfoCanvas, uses a beach scene to depict multiple
  pieces of real-time information Plaue et al.
  2004

7
Related Work Design (Cont.)

• Audio sense
• Non-Speech audio glance represents important
  properties of a message into a concise sound
  Hudson and Smith 1996

8
Related Work Design (Cont.)

• Tactile sense
• VisPad, a new haptic design for visualizing data
  Weissgerber et al. 2004

9
Related Work Evaluation

• Heuristic Evaluation
• Adapted Nielsens heuristic to the domain of
  peripheral displays Mankoff et al. 2003
• Wizard of OZ
• Simulates a wide range of plausible sensors to
  build multiple models Hudson et al. 2003
• Limitations
• Suitability
• Ignore the real value

10
Peripheral Display Systems
11
Experimental Design

• Aims
• To adapt a measurement of efficiency theory from
  cognitive science to measure the efficiency level
  in peripheral display
• To discover the difference between large and
  small displays in the context of distraction,
  comprehension and efficiency
• Hypotheses
• H1 Large display performs better than small
  display in terms of comprehension
• H2 Large display achieves higher efficiency than
  small display
• H3 The more times subjects shift their focus to
  the peripheral screen, the more information they
  will get

12
Experimental Setup
13
Subjects and Data

• Subjects
• Sixteen(8 female) subjects participated
• Average age 28.6
• Background 5 master, 9 PhD, 2 Postdoc
• Data
• Two hours trading from Australian financial stock
• Randomly pick three companies (AMP, NCPDP, VCR)
• Data trend
• AMP smoothly increasing
• NCPDP smoothly decreasing
• VCR smoothly decreasing

14
Materials

• Software
• Three peripheral displays (showed before)
• OpenCV Face Detection
• Hardware
• Two large rear-projection screen/ two 19 inch
  monitor (with same resolution 1024768)
• Dell Precision Workstation360
• LogiTech QuickCam Pro 4000

15
Settings

• Data Setting
• Same data set for three peripheral systems
• Stock price change rate is -10, 10
• Image change rate is once per-minute
• Experiment Setting
• Within-subject experimental design
• Fixed-ordering of tasks
• Whole experimental time around 1hour

16
Tasks

• Primary Task
• Type words on the primary screen
• Lasting two minutes
• May keep the peripheral screen on
• Secondary Task
• Shift focus to the peripheral screen
• Get information

17
Experiment Methodology
Introduction Task
Practice Task
Actual Trials
18
Introduction Task

• Definition of peripheral display
• Brief introduction to peripheral display
• General information about three systems
• Sample questions and answers

19
Practice Task

• Subjects were exposed to what the actual trials
  would be like
• Experiment involves two parts
• Large display
• Small display
• Each part involves four sub-tests
• Primary task without peripheral display
• Primary task plus Digital Data
• Primary task plus MoneyTree
• Primary task plus Stock Chart

20
Results

• Methods
• A Paired students t-Test
• Z-Score
• Three parameters
• Mean Comprehension Error Rate (MCER) by the
  answers in questionnaire
• Mean Words Typed (MWT) by the number of words
  typed in the primary task.
• Mean Focus Shift (MFS) by the number of focus
  shifts to the peripheral screen.

21
Three Analysis

• Mean Comprehension Error Rate
• Use of MCER to measure the comprehension in each
  test
• Mean Focus Shift
• Use of MFS to measure the distraction.
• Efficiency
• Adapted from Pass Pass et. al 1993
• Combing consideration of the measurement of
  mental effort and performance
• All the data is calculated in Z-Score

22
Mean Comprehension Error Rate (MCER)

• Significant different in MCER between large and
  small displays
• Paired student t-test on MCER (significant
  different is plt0.05)

23
MCER

• Mean Comprehension Error Rate (MCER) increases
  from left to right on both large and small
  displays

24
Mean Word Typed

• Significant difference in MWT between large and
  small displays

Paired student t-test on MCER (significant
different is plt0.05)
25
Other Results

• Large display achieves lower efficiency than
  small display
• Large display get higher Mean Focus Shift (MFS)
  than small display
• Limitations
• But the result is not significant
• Needs further study

26
Discussion Comprehension

• Results Discover
• Comprehension affected by relative position of
  the information
• The farther from the primary, the higher MCER
• Theory Support
• Agreement with the theory of human vision
• The acuity outside of the fovea vision drops
  rapidly
• Conclusion1 the most important information in
  the peripheral display has to be in the close
  proximity to the primary screen.

27
Discussion Distraction

• Results Discover
• Display distraction/Self-Interruption
• Smooth animation can decrease display distraction
  but not affect self-interruption
• Display distraction
• Caused by display itself
• Can be further divided into different levels
• Self-interruption
• Caused by subjects themselves
• Affected by the characteristics of subjects
  (personality, curiosity)
• Conclusion2 slow, smooth animation can help to
  reduce the level of display distraction but not
  affected the self-interruption

28
Discussion Efficiency

• Purpose
• Quantify the efficiency of peripheral display
• Result Discover
• Keep previous visualization history system can
  achieve higher efficiency
• Conclusion3 Considering perviously known visual
  language and keeps previous visualization history
  can improve the efficiency of peripheral display

29
Conclusion and Future work

• Two overall results
• Put forward a measurement of efficiency from
  cognitive science into peripheral display
• Propose three guidelines for peripheral display
  design
• Future Work
• Conduct non-intrusive evaluation
• Further investigate in the non-intrusive
  evaluation