Cooltown User Studies

1
Cooltown User Studies
From Informing to Remembering Deploying a
Ubiquitous System in an Interactive Science
Museum

• Margaret Fleck, Marcos Frid, Tim Kindberg, Eamonn
  OBrien-Strain, Rakhi Rajani and Mirjana
  Spasojevic
• Mobile Systems and Services Lab
• Hewlett-Packard Labs

2
Reality Check User Studies

• What works with ordinary users?
• What is their experience like?

3
Research Focus Cooltown User Studies
Web pages
Virtual
Physical
Exhibits

• Study actual use how to interleave electronic
  functionality with physical, nomadic activities
• Inform design, stimulate design

4
Exploratorium science museum

• Exploratorium mission
• Public exhibits space
• Educational activities, teachers programs
• Center for media and communication-dissolve the
  walls of the building
• NSF funded Electronic Guidebook project
• Investigate use of handhelds, wireless networks
• Increase level of engagement and help learning
• Pre-visit, during and post-visit interaction
• Expand Exploratoriums educational mission

5
Motivation
stimulating
exploration
Rich web based content
Electronic Guidebook
Cooltown technologies
6
Challenges
vigorous, noisy
hands-on
hard to navigate
7
Proposed functionalities

• Informer
• Content delivered to the user next to the exhibit
• Suggester/Guider
• Things to do while at the exhibit, things to see
  elsewhere
• Communicator
• Communication among family members, field trip
  groups
• Experience sharing, virtual graffiti
• Rememberer
• Help remember exhibits/phenomena, increase
  post-visit engagement

8
Phase I

• Prototype hybrid Electronic Guidebook Informer,
  Suggester, Rememberer
• Variety of technologies (handhelds, sensing,)
• Diverse user population (staff, teachers,
  students, )
• User tasks (browse, bookmark, treasure hunt, )
• Measure interactions with physical virtual
  resources (web logs, observers, videos, ..)
• Qualitative evaluation (interviews, surveys)

9
Infrastructure overview
p
content
server
RFID
IR
802.11
Internet
real exhibit
Pi-station beacon (infrared), barcode, RFID
plus processing, I/O and networking
10
Electronic Guidebook Technologies
beacons
barcodes
Hitachi E-Plate
Jornada 690
Jornada 540
11
Electronic Guidebook User Experience
12
User Experience first step

• location sensing get exhibit URL
• first web page automatically downloaded

Web page
URL
beacon
13
User Experience top page, follow to information
nuggets
top page (image map)
info nuggets
14
Remembering content personal scrapbook

• press bookmark button to save web page
• capture experience and make it memorable

postcard
collage poster
booklet
personal online scrapbook
personal experience
personal artifacts
15
Phase I experiments (Summer 2001)

• 34 users teachers, kids, staff, developers (ages
  10 to 50)
• Data collected detailed web logs, observations,
  interviews

16
Phase I key findings

• Low-level generic
• Verified feasibility of basic components
• Beacons/barcodes acceptable for location sensing
• Wireless connectivity (802.11)
• Browser UI
• Web pages easily accessible
• Issues with stylus
• Battery life will continue to be an issue
• Potential for getting lost in cyberspace!

17
Phase I key findings

• Domain-specific (Exploratorium-related)
• Most appropriate user groups teachers,
  explainers, repeat visitors
• Easy adoption with younger visitors
• Personalization of content is important
• Audio needs earplugs
• Users reacted positively to the 'my scrapbook'
  idea
• Potential negative effects
• distraction from exhibits by on-line content
• carrying devices affects interaction at the
  exhibits
• Ongoing evaluation of content for self-directed
  informal learning

18
Choosing a tool
Web pages
Virtual
Physical

Exhibits

• Informer, Suggester
• Too obtrusive for many visitors
• Guider, Communicator
• Less frequent distractions
• Rememberer
• Minimizes attention shifts
• Extends engagement to post-visit
• User interest

19
What is Rememberer?

• Record of the exhibits to remember. User controls
  what is captured.
• Components
• "remember-this" technology for selecting objects
  during the visit
• the visit record (set of web
  pages)
• a physical token, which reminds the user of the
  visit

20
Phase II infrastructure Rememberer
personal artifact (web page, postcard, etc.)
content
server
RFID
IR
802.11
Internet
remember-this
real exhibit
21
Rememberer research questions

• Interference with exhibit manipulation?
• Does the tool help with recall?
• Influence on discussion and social interaction
• among people who visited together
• with people who are remote
• Does the tool help as a resource?
• in itself
• as a launching point for further exploration

22
Rememberer experiments

• Round 1 informally verified basic concept
• RFID tags, cameras, web pages for photos
• Users like having photos taken this way
• Web pages accessed after the visit and annotated
• Round 2 detailed evaluation (23
  individuals/groups)

23
Round 2 experiment description

• Experiment setup
• Mixture of exhibits non-instrumented, with
  beacon, with beacon camera
• Individuals and small groups
• Control no device, all exhibits non-instrumented
  
• Observations interviews
• Diverse Exploratorium-associated volunteers (!)
• Kids and adults, male female

24
Use of remember-this

• Propensity
• All instrumented exhibits - 74
• Exhibits w/ beacons - 67
• Exhibits w/ beacons and
• camera - 86
• Interference
• Beacon pickup 10 out of 17 said easy or very
  easy
• Observed beacon problems 10 of 140 uses
• Camera adjustment 25 of 63 uses
• 50 said wanted more camera control (what/when)

25
Time spent at exhibits
No instrumentation 1 min 5 sec
Beacons 1 min 49 sec
Beacons w/ camera 2 min 3 sec
All exhibits 1 min 29 sec
Control group 1 min 40 sec

• Camera visits 20 longer than control group

26
Home use

• 10 out of 17 revisited web pages
• 4 twice
• 3 of them about 10 days after the experiment

27
Moving forward

• Continue analysis of Rememberer at Exploratorium
• More widely applicable remember-this
• e.g. Jornada with camera

Exploratorium
Multiple domains
Feb 02
Electronic Guidebook
Rememberer
28
Conclusions

• Nomadic computing tools simple applications
  that enhance interaction with the physical world
• Cooltown physical hyperlinks are an effective
  mechanism for invoking web-based services
• Exploratorium findings are likely to apply to
  other domains that are dynamic with high demand
  on user attention

29
More information

• Electronic Guidebook project information
• www.exploratorium.edu/guidebook/
• Most recent TR HPL-2002-54

30
Backup slides
31
Sample content orientation page
32
Sample content exhibit instructions, information
nuggets
33
Research focus Physical vs. Virtual navigation
Web pages
Virtual
Physical
Exhibits




34
Phenomena under investigation
Higher order effects (informal learning,
engagement, social interaction, )
Paths through physical and virtual
space (attention switching exhibits, devices,
content, companions, )
complexity
Attention to artifacts (exhibits, online content,
)
Basic affordances (handhelds, beacons, web pages,
audio, )