Olympic Message System

1
Olympic Message System

• John Kelleher
• Material Source
• THE 1984 OLYMPIC MESSAGE SYSTEM
• A TEST OF BEHAVIORAL PRINCIPLES
• OF SYSTEM DESIGN
• GOULD, STEPHEN J. BOIES, STEPHEN LEVY,
• JOHN T. RICHARDS, and JIM SCHOONARD

2
Overview

• 1984 Olympic voice-mail system
• 15 behavioural methologies employed
• Three principles
• Early focus on users
• Empirical measurement
• Iterative design

3
J. Gould Principles

• Early focus on users and tasks
• This means first understanding who the users
  will be by directly studying their cognitive,
  behavioural, anthropomorphic, and attitudinal
  characteristics. This required observing users
  doing their normal tasks, studying the nature of
  those tasks, and then involving users in the
  design process
• Empirical measurement
• Early in development, the reactions and
  performance of intended users to printed
  scenarios, manuals, etc. is observed and
  measured. Later on, uses interact with
  simulations and prototypes and their performance
  and reactions are observed, recorded and
  analysed.
• Iterative Design
• When problems are found in user testing, they
  are fixed and then more tests and observations
  are carried out to see the effects of the fixes.
  This means that design and development is
  iterative, with cycles of design, test, measure,
  and redesign being repeated as often as
  necessary.

4
Reactions to 3 principles

• Theyre obvious everybody says that
• Gould study1 26 mentioned none 35 mentioned
  only one
• Everybody does these things.
• should be process-oriented not mile-stone
  oriented
• Human factors is just fine-tuning.
• Gloss does not fix design defects not peanut
  butter
• In real life, you cant follow them.
• You cant measure usability

1 Gould. J.D.. and Lewis. C. Designing for
usability Key principles and what designers
think. Commun ACM 28, 3 (Mar. 1985). 300-311.
5
Background

• High risk system
• Short project schedule
• 10,000 Olympic athletes and officials
• across two campuses
• extended family and friends (postcards provided)
• 6 networked mainframes
• 25 kiosks
• 12 different languages supported (a first)
• 24/7 system

6
Use of system

• Kiosk housing
• CRT of Olympians with new messages
• Push-button telephone
• Entertaining videodisk of a mime giving
  instructions on use (4 min.)
• Copies of OMS guide
• People from around the world send messages to
  athletes (gt18,000 sent)
• Olympians send messages to each other
• Olympians pick up their own messages (43,000
  sign-ons)

7
How principles were followed

• Printed scenarios
• first definition of system function
• tangible, identifiable usage of system available
  at a time when comments could have greatest
  impact
• several features dropped (E.g. call verification)
• later coding saved

8
How principles were followed

• Early iterative tests of user guides
• guides written before coding began
• Olympians guide 200 iterations
• Families guide 50 iterations
• needed to minimise long distance call times
• option to change language heard E.g. Canadians.
• Discouraged late changes
• proposals had to be proved to be better
  empirically

9
How principles were followed

• Early simulations
• Wizard of Oz prototyping on simpler-to-program
  IBM VM System
• Provided early feedback from users
• Lab personnel and visitors proved poor judges of
  usability
• Feedback helped form prompt messages
• Need for consistent escapes identified
• E.g. USA Olympian accidentally enters USSR

10
How principles were followed

• Early Demonstrations
• Prompted dropping of several functions to reduce
  audio prompting.
• Olympians on the design team
• Ghana ex-Olympian
• Tours of Olympic Villages
• Difficulty of access
• Classroom training for Olympians rejected due to
  campus layout, climate and availability of large
  rooms

11
How principles were followed

• Interviews with Olympians
• Insight into their routine and time schedules
• Overseas Tests of Family/Friends interface
• Added an example to OMS guide
• Hallway and Storefront Methodology
• Refined dimensions of kiosk and documentation
• E.g. Deutsch rather than German
• Asked people to find their name on the list

12
How principles were followed

• Yorktown Prototype test
• 100 participants
• Try-to-Destroy-it Tests
• Pre-Olympic Field Test
• Competitors from 65 countries
• 57 usability issues arose
• Embarrassed at exclusion of competitors from
  Oman, Columbia, Pakistan, Japan and Korea
• Yorktown Final Prototype Test
• 2800 participants

13
Sample Dialog

• User (Dial 8540.)
• OMS Olympic Message System.
• Please keypress your three-letter Olympic
  country code.
• Systeme de Message Olympique
• Tapez le numer de votre pays, sil vous plait.
• User USA
• OMS United States. Les Etats-Unis.
• Please keypress your last name.
• User goul
• OMS John Gould.
• Please keypress your password.
• User 319
• OMS Welcome to Olympic Message System.
• New Message from Stephen Boies.

14
Usability Findings 1/3

• Three-letter Olympic country code
• Interruptible initial prompt not obvious
• Time-out help messages tuned only for English.
• Name problems
• Middle/Far Eastern uses unsure of whether first
  or last name required
• Solved by referring to Olympic ID badge
• Indicated by change in all documentation

15
Usability Findings 2/3

• Name issues (contd.)
• Alphabet translations often inaccurate for 5
  languages
• Referred to badge for accuracy
• Spelling names of other Olympians
• Added reply option for messages received
• Sign-on required only enough chars to distinguish
  from other Olympians from same country
• smart sponge programming
• 80-90 of Olympians spoke 1 of 4 major languages
• Testing indicated need for 12 languages to be
  supported

16
Usability Findings 3/3

• Passwords
• Often did not know password
• Solved by initially assigning last 3 digits of
  badge number as password
• Some non-English-speaking users had difficulty
  answering some prompts by chars and others by
  digits
• We have to read the telephone keys differently
• Typography changed to emphasise distinction
• Upper case used and letters separated