IEEM 620b Industrial Applications of Virtual Reality Technology

1
IEEM552 - Human Computer Systems Week 2 - A
usability study February 23, 1999
Dr. Vincent Duffy - IEEM http//iesu5.ieem.ust.hk/
dfaculty/duffy/552 email vduffy_at_ust.hk
1
2
Today and next few weeks

• Week 4 - Tuesday, 23rd February
• in week 4 - select papers for oral presentations
• finalize groups of 2 or 3,
• present during wk.6,7,9,10, or 11
• first a practice exercise - homework due by noon
  Tuesday 23rd by email (no attachment)
• details in just a few minutes minute
• Week 5 - Monday, March 1
• OSHC seminar 9am-6pm (no Tuesday class)

2
3
For today - week 4

• Postpone demonstration until week 6
• Administrative
• Determine teams and select papers for oral
  presentations
• handout (references week due)
• Discuss trip Monday, March 1 (week 5)
• no class meeting Tuesday
• handout (who is registered, who needs travel
  arrangments, reg. code)
• what to look for at the conference?

3
4
Today continued

• Guidelines on VDT (video display terminal) use in
  the workplace
• Permission for the I.S. Lab
• user access card
• New content
• Economic benefit or Cost/benefits Analysis
  Assessing Usability (Ch.1-p.11-17)
• Mantei and Teori method
• For Discussion/lecture A usability study
• summary of Nielsen - thinking aloud method

4
5
Things to consider for presentation

• See 10 ways to look at a research paper
• also - 1. Objective of the research
• 2. Brief discussion of literature support
• 3. Development of hypotheses (if any) or why
  the research was done in this way
• 4. Methods
• 5. Results
• 6. Discussion/conclusions
• any practical application for the research?
• Theoretical contribution?
• 7. Future work any suggestions

5
6
10 ways to look at a research paper
systematically for critical analysis

• 1. research ideas/question -what is the purpose?
• 2. what background support? - literature review
• 3. theoretical basis for analyzing
  question/hypothesis?
• 4. applicability-practical contribution?
• 5. theoretical contribution?
• 6. appropriate methodologies for carrying out
  study? (determining variables, data
  collection, method used to test hypotheses,
  validity of measures and reliability)
• 7. appropriate statistical analyses and
  assumptions?
• 8. presentation of results what do they really
  mean?
• 9. conclusions drawn are they reasonable?
• 10. future work/research directions any
  possibilities?

6
7
For oral presentation

• Please determine groups of two
• choose the paper of your interest for
  presentation
• library catalog - images reserve

7
8

• Due Week 4 Today- Estimating the Number of
  Subjects Needed for a Thinking Aloud Test,
  J.Nielsen, International Journal of
  Human-Computer Studies, 41, 385-397, 1994. (no
  presentation)
• Week 5 - March 1 - A Study on Prevalence of
  Musculoskeletal Discomfort Among Visual Display
  Terminals (VDT) Users in Hong Kong, W.S. Tang,
  ERGON-AXIA, Proceedings of the First World
  Congress on Ergonomics for Global Quality and
  Productivity, Ed. Bishu, Karwowski and
  Goonetileke, p.6-30, 1998. (no presentation)
• Week 6 - March 9 - An Empirical Assessment of
  Task Complexity for Computerized Menu Systems,
  J.A. Jacko, International Journal of Cognitive
  Ergonomics, 1 (2), 137-147, 1997.
• Week 7 - March 16 - The Effect of Pictogram-Based
  Interface Design on Human-Computer Performance,
  L.A. Miller and K.M. Stanney, International
  Journal of Human-Computer Interaction, 9 (2),
  119-131, 1997.
• Week 8 - Mid-term Exam
• Week 9 - March 30 - Using Natural Language
  Conventions in the User Interface Design of
  Automatic Speech Recognition Systems, D.J. Brems,
  M.D.Rabin, J.L.Waggett, Human Factors, 265-282,
  1995.
• Week 10 - April 13 - Development and Evaluation
  of the Windows Computer Experience Questionnaire
  (WCEQ), L.A. Miller, K.M. Stanney, W.Wooten,
  International Journal of Human-Computer
  Interaction, 9 (3), 201-212, 1997.
• Week 11 - April 20 - Temporal Issues of Quality
  of Working Life and Stress in Human-Computer
  Interaction, P.Carayon, International Journal of
  Human-Computer Interaction, 9 (4), 325-342, 1997.
• Week 12 - April 27 - Persuasiveness of Expert
  Systems, J.J.Dijkstra, W.B.G.Liebrand,
  E.Timminga, Behaviour and Information Technology,
  17 (3), 155-163, 1998.

9
Homework due before next class meeting

• 1. Please read the following before going to the
  Symposium.
• Week 5 - March 1 - A Study on Prevalence of
  Musculoskeletal Discomfort Among Visual Display
  Terminals (VDT) Users in Hong Kong, W.S. Tang,
  ERGON-AXIA, Proceedings of the First World
  Congress on Ergonomics for Global Quality and
  Productivity, Ed. Bishu, Karwowski and
  Goonetileke, p.6-30, 1998. (no presentation)
• 2. While at the Symposium. Please plan to
  discuss your knowledge with someone there and ask
  someone an intelligent question about it.
• 3. Please tell the name and affiliation of the
  person you spoke with and summarize your question
  and answer/discussion (up to 1 paragraph)..
• Please submit it by noon Tuesday of Week 6 (can
  be done in group of 2).
• 4. Please read the following and be prepared to
  ask a question during/after the
  discussion/presentation)
• Week 6 - March 9 - An Empirical Assessment of
  Task Complexity for Computerized Menu Systems,
  J.A. Jacko, International Journal of Cognitive
  Ergonomics, 1 (2), 137-147, 1997.

9
10
Recall H.W. For week 4

• 2. Economic Benefits of User Interface design
• read chapter 1-Eberts-User Interface Design
• consider the Mantei and Teori method.
• Q. Could it have been used by Microsoft (for Word
  for DOS) for determining the economic benefits of
  increased usability?
• Yes/No and 1 sentence is sufficient.

10
11
Economic benefits of Usability analysis

• assessing user interfaces
• benefits?
• faster task completion times, less errors,
  reduced costs for training (help required)
• Bullinger, Burmester, Vossen, 1995
• learning times, user error, cost of maintenance
• Eberts, 1994 Mantei and Teory, 1988 Karat 1992
• costs?
• focus groups, building mock-ups, prototype,
  design changes, running the studies, making test
  environment, conducting the user survey

11
12
Economic benefits of Usability analysis

• total savings
• learning time user errors maintenance costs
• learning time
• turnover, training time saved, wages
• cost of errors per year
• of employees, searches/hr, hr/yr, P(error),
  time to fix error, wages
• maintenance costs (design change savings)
• of changes (25?), hrs to change (8?), wages
• late change 4x early cost (late cost minus early
  cost)

12
13
Recall H.W. For week 4

• 2. Economic Benefits of User Interface design
• read chapter 1-Eberts-User Interface Design
• consider the Mantei and Teori method.
• Q. Could it have been used by Microsoft (for Word
  for DOS) for determining the economic benefits of
  increased usability?
• Yes/No and 1 sentence is sufficient.

13
14
Economic benefits of Usability analysis

• Assumption of this method?
• Mantei and Teory - mostly concerned with
  in-house software use
• if company is selling software, then how
  different?
• improvement increased revenue
• cost percent of increase sale of old units
• Why not for Microsoft case?
• It is about whether or not the software will sell
  better than that already on the market.

14
15
Recall H.W. - week 4

• 1. research paper for next week
• Nielsen, J. (1994) Estimating the number of
  subjects needed for a thinking aloud test Int. J.
  of Human Computer Studies 385-397.
• using the 10 ways to look at a research paper
• choose 3 subheadings in the 10 ways and write 3
  sentences about each.
• Choose 1 subheading and write 1 sentence about
  why you didnt choose it.

15
16
Nielsen paper

• Thinking aloudhow does it work
• see video
• Do you agree or disagree w/the following
• Thinking aloud method was not widely used because
  of its complexity and high cost.
• True/false?
• Actuallyusability studies where many subjects
  are needed to show quantitative results are of
  high cost.
• Nielsen is pointing out that a large of
  problems can be found with a small of subjects.

16
17
Remember homework due before next class meeting

• 1. Please read the following before going to the
  Symposium.
• Week 5 - March 1 - A Study on Prevalence of
  Musculoskeletal Discomfort Among Visual Display
  Terminals (VDT) Users in Hong Kong, W.S. Tang,
  ERGON-AXIA, Proceedings of the First World
  Congress on Ergonomics for Global Quality and
  Productivity, Ed. Bishu, Karwowski and
  Goonetileke, p.6-30, 1998. (no presentation)
• 2. While at the Symposium. Please plan to
  discuss your knowledge with someone there and ask
  someone an intelligent question about it.
• 3. Please tell the name and affiliation of the
  person you spoke with and summarize your question
  and answer/discussion (up to 1 paragraph)..
• Please submit it by noon Tuesday of Week 6 (can
  be done in group of 2).
• 4. Please read the following and be prepared to
  ask a question during/after the
  discussion/presentation)
• Week 6 - March 9 - An Empirical Assessment of
  Task Complexity for Computerized Menu Systems,
  J.A. Jacko, International Journal of Cognitive
  Ergonomics, 1 (2), 137-147, 1997.

17
18
for next class meeting - week 6

• in class demonstration - March 9
• postponed from this week
• all groups will participate in collecting data
• compare two interfaces
• analyze the data, interpret the results
• estimating the benefits -
• we will compare library cat for dos/www
• meet in the I.S. Lab
• first half of class
• 3rd floor - near 3203

18