Human Abilities and Models

1
Human Abilities and Models

• Sensory and cognitive abilities and models,
  models of human performance

2
Outline

• Human capabilities and disabilities
• Senses
• Motor systems
• Memory
• Cognitive Processes
• Selective attention, learning, problem solving,
  language
• Predictive models
• Contextual models

3
Typical Person

• Do we really have limited memory capacity?

4
Basic Human Capabilities

• Do not change very rapidly
• Not like Moores law!
• Have limits, which are important to understand
• Why do we care?
• Better design!
• Want to improve user performance
• Universal design designing for all people,
  including those with disabilities

5
But were all disabled sometimes

• Environment
• Fatigue
• Injury
• Aging
• Changing role of information technology

6
Usable Senses

• The 5 senses (sight, sound, touch, taste and
  smell) are used by us every day
• each is important on its own
• together, they provide a fuller interaction with
  the natural world
• Computers rarely offer such a rich interaction
• Can we use all the available senses?
• ideally, yes
• practically no
• We can use sight sound touch (sometimes)
• We cannot (yet) use taste smell

7
Vision Fundamentals

• Retina has
• 6.5 M cones (color vision), mostly at fovea
  (1/3)
• About 150,000 cones per square millimeter
• Fewer blue sensing cones than red and green at
  fovea
• 100 M rods (night vision), spread over retina,
  none at fovea
• Adaptation
• Switching between dark and light causes fatigue

8
Vision implications (more to come in visual
design)

• Color
• Distinguishable hues
• optical illusions
• Acuity
• Determines smallest size we can see
• Less for blue and yellow than for red and green

9
Color/Intensity Discrimination

• The 9 hues most people can identify are
• Color Wavelength
• Red 629
• Red-Orange 596
• Yellow-Orange 582
• Green-Yellow 571
• Yellow-Green 538
• Green 510
• Blue-Green 491
• Blue 481
• Violet-Blue 460

10
Color Surround Effect

• Our perception of a color is affected by the
  surrounding color

11
Vision Difficulties

• Color blindness
• About 9 of males are red-green colorblind!
• See http//colorlab.wickline.org/colorblind/colorl
  ab/
• Low-vision
• The vast majority of visually disabled people
  have some sight
• Blindness
• Rely on other senses to receive information
• Specialized hardware and software
• Screen readers
• Braille printers, etc.

12
Myopia and Hypermetropia

• Myopia Hypermetropia
• (short-sighted) (far-sighted)

13
Macular degeneration
14
Diabetic retinopathy
15
Cataracts
16
Tunnel vision
17
Accommodating Partial Sight

• Large monitor, high resolution, glare protection
• Control of color and contrast
• Control of font size everywhere
• Keyboard orientation aids

18
Accommodating Blind Users

• Screen Readers
• Full-featured
• Cursor-tracking, routing
• Dialogue focus
• View areas
• Auditory or tactile output

http//www.freedomscientific.com/fs_products/softw
are_jaws.asp
http//www.webaim.org/simulations/screenreader
19
Audition (Hearing)

• Capabilities (best-case scenario)
• pitch - frequency (20 - 20,000 Hz)
• loudness - amplitude (30 - 100dB)
• location (5 source stream separation)
• timbre - type of sound (lots of instruments)
• Often take for granted how good it is(disk
  whirring)
• Implications ?

20
Hearing uses

• Redundant output
• Email beep icon, IM sound popup message, etc.
• Output when screen not available
• Multimedia systems

21
Hearing problems or deafness

• An increasing problem?
• Population
• Phone interfaces
• Various technologies used
• Communication aids
• Automated software (speech to text, etc.)

22
Touch

• Three main sensations handled by different types
  of receptors
• Pressure (normal)
• Intense pressure (heat/pain)
• Temperature (hot/cold)
• Where important?
• Mouse, Other I/O, VR, surgery

23
Motor System

• Capabilities
• Range of movement, reach, speed,strength,
  dexterity, accuracy
• Workstation design, device design
• Often cause of errors
• Wrong button
• Double-click vs. single click
• Principles
• Feedback is important
• Minimize eye movement
• See Handbooks for data

24
Work Station Ergonomics to Facilitate I/O
25
Large Range of Physical Impairments

• Complete lack of function
• absence of a limb
• paralysis usually due to spinal injury, the
  higher the damage the greater the degree of
  paralysis
• Lack of strength
• Tremor/lack of accuracy
• Slowness

26
Implications

• Try to minimize movement and strain
• Alternative input devices
• Keyboard hardware and software
• Speech input
• Other input switches for more severe needs
• Eye gaze, sip and puff, etc.
• Acceleration techniques
• Word completion, macros, etc.

27
The Mind

• And now on to memory and cognition

28
The Model Human Processor

• A true classic - see Card, Moran and Newell, The
  Psychology of Human-Computer Interaction,
  Erlbaum, 1983
• Microprocessor-human analogue using results from
  experimental psychology
• Provides a view of the human that fits much
  experimental data
• But is a partial model
• Focus is on a single user interacting with some
  entity (computer, environment, tool)
• Neglects effect of other people

29
Memory

• Perceptual buffers
• Brief impressions
• Short-term (working) memory
• Conscious thought, calculations
• Long-term memory
• Permanent, remember everything that ever happened
  to us

30
LONG-TERM MEMORY
R Semantic D Infinite S Infinite
SHORT-TERM (WORKING) MEMORY
AUDITORY IMAGE STORE
VISUAL IMAGE STORE
R Acoustic or Visual D (one chunk) 73 73-226
s D (3 chunks) 7 5-34 s S 7 5-9 chunks
R Acoustic D 1.5 0.9-3.5 s S 5 4.4-6.2
letters
R Visual D 200 70-1000 ms S 17 7-17
letters
PERCEPTUAL PROCESSOR C 100 5-200 ms
COGNITIVE PROCESSOR C 70 27-170 ms
MOTOR PROCESSOR C 70 30-100 MS
R Representation D Decay Time S Size C
Cycle Time
Eye movement (Saccade) 230 70-700 ms
31
Sensory Stores

• Very brief, but accurate representation of what
  was perceived
• Physically encoded
• Details decay quickly (70 - 1000 ms visual 0.9 -
  3.5 sec auditory)
• Limited capacity
• Iconic visual
• 7 - 17 letters 70 - 1000 ms decay
• Echoic auditory
• 4 - 6 auditory 0.9 - 3.5 sec auditory
• Haptic - touch
• Attention filters information into short term
  memory and beyond for more processing

32
Short Term Memory

• Symbolic, nonphysical acoustic or visual coding
• Decay 5-226 sec, rehearsal prevents decay
• Another task prevents rehearsal interference
• Use chunks 7 - 2 units of information

33
About Chunks

• A chunk is a meaningful grouping of information
  allows assistance from LTM
• 4793619049 vs. 704 687 8376
• NSAFBICIANASA vs. NSA FBI CIA NASA
• My chunk may not be your chunk
• User and task dependent

34
Long-Term Memory

• Seemingly permanent unlimited
• Access is harder, slower
• -gt Activity helps (we have a cache)
• Retrieval depends on network of associations
• How information is perceived, understood and
  encoded determines likelihood of retrieval
• Effected by emotion, previous memory

File system full
35
LT Memory Structure

• Episodic memory
• Events experiences in serial form
• Helps us recall what occurred
• Semantic memory
• Structured record of facts, concepts skills
• Semantic network theory
• Or theory of frames scripts (like record
  structs)

36
Memory Characteristics

• Things move from STM to LTM by rehearsal
  practice and by use in context
• Do we ever lose memory? Or just lose the link?
• What are effects of lack of use?
• We forget things due to decay and interference
• Similar gets in the way

37
Recognition over Recall

• We recognize information easier than we can
  recall information
• Examples?
• Implications?

38
Processes

• Four main processes of cognitive system
• Selective Attention
• Learning
• Problem Solving
• Language

39
Selective Attention

• We can focus on one particular thing
• Cocktail party chit-chat
• Salient visual cues can facilitate selective
  attention
• Examples?

40
Learning

• Two types
• Procedural How to do something
• Declarative Facts about something
• Involves
• Understanding concepts rules
• Memorization
• Acquiring motor skills
• Automotization
• Tennis
• Driving to work
• Even when dont want to
• Swimming, Bike riding, Typing, Writing

41
Learning

• Facilitated
• By structure organization
• By similar knowledge, as in consistency in UI
  design
• By analogy
• If presented in incremental units
• Repetition
• Hindered
• By previous knowledge
• Try moving from Mac to Windows
• gt Consider users previous knowledge in your
  interface design

42
Observations

• Users focus on getting job done, not learning to
  effectively use system
• Users apply analogy even when it doesnt apply
• Or extend it too far - which is a design problem
• Dragging floppy disk icon to Macs trash can does
  NOT erase the disk, it ejects disk!

43
Problem Solving

• Storage in LTM, then application
• Reasoning
• Deductive -
• Inductive -
• Abductive -
• Goal in UI design - facilitate problem solving!
• How??

If A, then B
Generalizing from previouscases to learn about
new ones
Reasons from a fact to theaction or state that
caused it
44
Observations

• We are more heuristic than algorithmic
• We try a few quick shots rather than plan
• Resources simply not available
• We often choose suboptimal strategies for low
  priority problems
• We learn better strategies with practice

45
People

• Good
• xxx
• yyy
• zzz

• Bad
• aaa
• bbb
• ccc

Fill in the columns - what are people good at and
what are people bad at?
46
People

• Good
• Infinite capacity LTM
• LTM duration complexity
• High-learning capability
• Powerful attention mechanism
• Powerful pattern recognition

• Bad
• Limited capacity STM
• Limited duration STM
• Unreliable access to LTM
• Error-prone processing
• Slow processing

47
Models

• Translating empirical evidence into theories and
  models that influence design.
• Performance measures
• Quantitative
• Time prediction
• Working memory constraints
• Competence measures
• Focus on certain details, others obscured

48
Fitts Law

• Models movement times for selection tasks
• Paul Fitts war-time human factors pioneer
• Basic idea Movement time for a well-rehearsed
  selection task
• Increases as the distance to the target increases
• Decreases as the size of the target increases

49
Moving

• Move from START to STOP

Index of Difficulty ID log2 ( 2D/W ) (in
unitless bits)
width of target
distance
50
Movement Time

• MT a bID
• or
• MT a b log2 (2D/W)

• Empirical measurement establishes constants a and
  b
• Different for different devices and different
  ways the same device is used.

51
Questions

• What do you do in 2D?
• h x l rectone way is ID log2(d/min(w, l) 1)
• Should take into account direction of approach

52
Applications

• When does it apply?
• How used in interface design?

53
GOMS

• Goals, Operators, Methods, Selection Rules Card,
  Moran, Newell (1983)
• Assumptions
• Human activity is problem solving
• Decompose into subproblems
• Determine goals to attack problem
• Know sequence of operations used to achieve the
  goals
• Timing values for each operation

54
GOMS Components

• Goals
• State to be achieved
• Operators
• Elementary perceptual, cognitive, motor acts
• Not so fine-gained as Model Human Processor
• Methods
• Procedures for accomplishing a (sub)goal
• e.g., move cursor via mouse or keys
• Selection Rules
• if-then rules that determine which method to use

55
GOMS Limitations

• GOMS is not so well suited for
• Tasks where steps are not well understood
• Inexperienced users
• Why?

56
GOMS Application

• NYNEX telephone operation system
• GOMS analysis used to determine critical path,
  time to complete typical task
• Determined that new system would actually be
  slower
• Abandoned, saving millions of dollars

57
Keystroke Level Model (KLM)

• Low-level GOMS variant
• Also developed by Card, Moran, and Newell (1983)
• Skilled users performing routine tasks
• Assumes error-free performance
• Analyze only observable behaviors
• Keystrokes, mouse movements
• Assigns times to basic human operations -
  experimentally verified

58
KSLM Accounts for

• Keystroking TK
• Mouse button press TB
• Pointing (typically with mouse) TP
• Hand movement betweenkeyboard and mouse TH
• Drawing straight line segments TD
• Mental preparation TM
• System Response time TR

59
Step One MS Word Find Command

• Use Find Command to locate a six character word
• H (Home on mouse)
• P (Edit)
• B (click on mouse button - press/release)
• P (Find)
• B (click on mouse button)
• H (Home on keyboard)
• 6K (Type six characters into Find dialogue box)
• K (Return key on dialogue box starts the find)

60
Using KSLM - Step Two

• Place M operators
• Rule 0a. In front of all Ks that are NOT part of
  argument strings (ie, not part of text or
  numbers)
• Rule 0b. In front of all Ps that select commands
  (not arguments)

61
Step Two MS Word Find Command

• H (Home on mouse)
• MP (Edit)
• B (click on mouse button)
• MP (Find)
• B (click on mouse button)
• H (Home on keyboard)
• 6K (Type six characters)
• MK (Return key on dialogue box starts the find)

Rule 0b Pselects command
Rule 0b Pselects command
Rule 0a Kis argument
62
Using KSLM - Step 3

• Remove Ms according to heuristic rules
• (Rules relate to chunking of
  actions)
• Rule 1. Anticipated by prior operation
• PMK -gtPK (point and then click is a chunk)
• Rule 2. If string of MKs is a single cognitive
  unit (such as a command name), delete all but
  first
• MKMKMK -gt MKKK (same as M3K) (type run rtn is a
  chunk)
• Rule 3. Redundant terminator, such as )) or rtn
  rtn
• Rule 4. If K terminates a constant string, such
  as command-rtn, then delete M
• M2K(ls)MK(rtn) -gt M2K(ls)K(rtn) (typing ls
  command in Unix followed by rtn is a chunk)

63
Step 3 MS Word Find Command
H (Home on mouse) MP (Edit) B (click on mouse
button) MP (Find) B (click on mouse button) H
(Home on keyboard) 6K (Type six characters) MK
(Return key on dialogue box starts the find)
Rule 1 delete M H anticipates P
Rule 4 Keep M
64
Using KSLM - Step 4

• Plug in real numbers from experiments
• K .08 sec for best typists, .28 average, 1.2 if
  unfamiliar with keyboard
• B down or up - 0.1 secs click - 0.2 secs
• P 1.1 secs
• H 0.4 secs
• M 1.35 secs
• R depends on system often less than .05 secs

65
Step 4 MS Word Find Command

• H (Home on mouse)
• P (Edit)
• B (click on mouse button - press/release)
• MP (Find)
• B (click on mouse button)
• H (Home on keyboard)
• 6K (Type six characters into Find dialogue box)
• MK (Return key on dialogue box starts the find)
• Timings
• H 0.40, P 1.10, B 0.20, M 1.35, K 0.28
• 2H, 2P, 2B, 2M, 7K
• Predicted time 8.06 secs

66
Example MS Windows Menu Selection

• Get hands on mouse
• Select from menu bar with click of mouse button
• The pull down menu appears
• Select desired item from the pull down menu

67
Step 1 MS Windows Menu

• H (Home on mouse)
• P (point to menu bar item)
• B (left-click with mouse button)
• P (point to menu item)
• B (left-click with mouse button)

68
Step 2 MS Windows Menu - Add Ms

• H (get hand on mouse)
• MP (point to menu bar item)
• B (left-click with mouse button)
• MP (point to menu item)
• B (left-click with mouse button)

Rule 0b Pselects command
Rule 0b Pselects command
69
Step 3 MS Windows Menu - Delete Ms

• H (get hand on mouse)
• MP (point to menu bar item)
• B (left-click with mouse button)
• MP (point to menu item)
• B (left-click with mouse button)

Rule 1 Manticipated by P
Keep M
70
Step 4 MS Windows Menu Calculate Time

• H (get hand on mouse)
• P (point to menu bar item)
• B (left-click with mouse button)
• MP (point to menu item)
• B (left-click with mouse button)
• Textbook timings (all in seconds)
• H 0.40, P 1.10, B 0.20, M 1.35
• H, 2P, 2B, 1 M
• Total predicted time 4.35 sec

71
Alternative Menu Selection

• Operator sequence
• H(mouse)P(to menu item)B(down)PB(up)
• Now place Ms
• H(mouse)MP(to menu item)B(down)MPB(up)
• Selectively remove Ms
• H(mouse)MP(to menu item)B(down)MPB(up)
• Textbook timings (all in seconds)
• H 0.40, P 1.10, B 0.10 for up or down, M
  1.35
• H, 2P, 2 B, 1 M
• Total predicted time 4.15 sec
• Alternative is predicted to be .2 secs faster
  than typical, about 5

Rule 0b
Rule 0b
Rule 1 Delete H anticipates P
72
KSLM Comparison Problem

• Are keyboard accelerators always faster than menu
  selection?
• Use MS Windows to compare
• Menu selection of File/Print (previous example
  estimated 4.35 secs.)
• Keyboard accelerator
• ALT-F to open the File pull down menu
• P key to select the Print menu item
• Assume hands start on keyboard

73
KSLM ComparisonKeyboard Accelerator for Print

• Use Keyboard for ALT-F P (hands already there)
• K(ALT)K(F)K(P)
• MK(ALT)MK(F)MK(P)
• MK(ALT)K(F)MK(P)
• 2M 3K 2.7 3K
• Times for K based on typing speed
• Good typist, K 0.12 s, total time 3.06 s
• Poor typist, K 0.28 s, total time 3.54 s
• Non-typist, K 1.20 s, total time 6.30 s
• Time with mouse was 4.35 sec
• Conclusion Accelerator keys not necessarily
  faster than mouse for all users!

First K anticipates second K
74
One more practice

• Draw through text and make it bold
• By pointing to BOLD icon in floating palette
• By selecting BOLD from pull-down menu
• Hierarchical menu selection combine with Fitts
• One big menu
• Several smaller menus?

75
Context and Cognition

• Human information processor models all involve
  unaided individual
• In reality, people work with other people and
  other artifacts
• Other models of human cognition
• Situation action
• Activity theory
• Distributed cognition

76
Distributed Cognition (DCog)

• HCI Proponent Ed Hutchins
• Distributed collection of interacting people and
  artifacts

77
Distributed Cognition

• Unit of analysis is cognitive system composed of
  individuals and the artifacts they use
• Studies the coordination and cooperation between
  people and artifacts in a distributed process

like activity
78
D.C. Principles

• Individual agents
• Distributed collection of interacting people and
  artifacts
• Functional system is what matters, not individual
  thoughts in peoples heads

79
Activity Theory

• Long history
• HCI proponent Bonnie Nardi

80
Activity Theory

• Unit of analysis is an activity
• Components

subject, object, actions, operations
Noun
Held by subject, motivates activity object of
game
Goal-directedprocesses tasks
How actionis carried out
81
A.T. Principles

• Key idea Notion of mediation by artifacts
• Our work is a computer-mediated activity
• Starring role goes to activity
• In regular HCI, stars are person and machine
• Context is not out there. It is generated by
  people in activities

82
Situated Action

• Noted proponent Lucy Suchman
• Much of the theory that underlies ethnography

83
Situated Action

• Studies situated activity or practice
• Activity grows out of the particulars of a
  situation
• Improvisation is important
• Basic unit of analysis is the activity of
  persons acting in a setting

84
Example

• Need 3/4 of 2/3 of cup of cottage cheese
• Just has a simple measuring cup available
• Person solves problem by
• Measuring 2/3 cup
• Pouring out into a circle
• Divide into quadrants
• Take away one
• One time solution to one time problem

85
Other comments on S.A.

• Emergent property of moment-by-moment
  interactions
• Improvisation
• Detailed temporal accounts
• De-emphasizes rigid plans and rational problem
  solving

86
S.A. Principles

• Structuring of an activity grows out of immediacy
  of the situation
• People engage in opportunistic, flexible ways to
  solve problems
• Formulaic plans
• Rational problem solving

87
Comparing Models

• The role of goals or intentions
• S.A. retrospective reconstructions
• A.T. D.C central
• Persistent structures
• S.A. emphasize emergent/ contingent/
  improvisatory over routine/predictable
• A.T. our activity assimilates experience of
  humanity
• D.C. much focus on transformation of artifacts
  over time

88
Comparing Models

• People and things
• MHP model each as a machine, study the diad of
  H-C
• S.A. qualitatively different, but mostly
  reactive
• A.T. computers and people not equals, a moral
  stance
• D.C. both are agents, study multiagent system

89
Some Commentary

• Take the reading with a grain of salt.
• How does this influence design?