Some History of Integrated Human Performance Models

1
Some History of IntegratedHuman Performance
Models

• Richard W. Pew
• BBN Technologies
• March 2, 2005

2
Overview HPMs Three Major Threads

• Manual Control
• Models of human control in closed-loop systems
• Task Network Models
• Models that fundamentally predict probability of
  success and performance time
• Models derived from Cognitive Architectures
• Typically models that capture theories of human
  performance capacities and limitations

Since most of you cut your teeth on Cognitive
Architectures I will focus on the first two, and
a couple of hybreds
3
Manual Control Thread
McRuer Krendel (1965) Quasi-Linear Control
Models
McRuer Krendel (1957) Cross-Over Model
Jagacinski Flack (2003) Control Theory for
Humans
Tustin (1947) Russell (1951) Elkind
(1956) Servomechanisms
Kleinman Baron (1970) Optimal Control Model
1940/50s Tracking Research
Chronology
4
Early Human Tracking Research 1940-50s

• During World War II and immediately after, a
  number of psychologists were funded to study
  human motor performance primarily in support of
  selection and training of pilots
• The most interesting and analytic work was done
  in Great Britain, especially at the Applied
  Psychology Research Unit by J.K.W. Craik, E.R.F.
  Crossman, Chris Poulton and Margaret Vince
• My first introduction came from a paper in The
  Journal of the Institute of Radio Engineers by
  Birmingham and Taylor entitled, Man-operated
  continuous control systems in 1955

5
The Manual Tracking Paradigm
6
The Early Days of Human Feedback Control

• Servomechanism theory required analysis in the
  frequency domain rather than the time domain
• Human frequency response 0.1-1.0 hz.
• Modern control theory not invented yet
• Fast Fourier Transform not rediscovered until
  Cooley/Tukey (1962)
• Daunting task to calculate power spectra and
  cross-power spectra for low frequencies from
  noisy data
• Paper and Pencil
• Watt-Hour meters
• Photometric correlator
• Analog computers

7
Estimating Human Response Requires Calculating
Power Spectra
8
First Citation attributed to Arnold Tustin (1947)

• Well-known British control engineer
• During WWII he was concerned with design of
  massive gun turrets and wanted to make their
  servomechanism response compatible with human
  control.
• Through laboratory experiments and tedious paper
  and pencil analysis he demonstrated an
  approximate linear law plus remnant
  consistent with a describing function analysis.
• Explored various aided gun-laying schemes to
  improve aiming performance.

9
What is a Describing Function?

• A Describing Function is a control engineering
  approximation that characterizes a non-linear
  system as a sum of a linear transfer function
  combined with additive noise (referred to as
  remnant).
• If the linear function accounts for more than 75
  or so of the output, the linear representation is
  considered useful.
• Think about it in correlational terms. R2
  0.75

10
Lindsay Russell (1951)

• MIT Masters Thesis
• Measured human response to sum of 4 different
  frequency sine waves combined to form random
  appearing wave form.
• Ingenious measurement accomplished with Watt-Hour
  meters
• Simulated different physical system dynamics
• Found that humans could differentially adapt
  their response to compensate for system dynamics

11
Jerry Elkind (1956)

• MIT Ms PhD theses with J.C.R. Licklider
• Programmed a very sophisticated analogue (sic)
  computer spectrum analyzer
• Measured human response to random appearing
  inputs made up of a sum of 40 to 144 sine waves
  having amplitudes and frequencies to simulate
  various filter behaviors
• Computed human transfer functions and adjustment
  rules for wide range of amplitude, bandwidth and
  shape of input signal characteristics
• The effort was tremendous! Probably worth 3 PhDs

12
Enter Duane MacRuer and Ezra Krendel

• Comprehensive review of work up to 1957
  summarized in Dynamic Response of Human
  Operators for the Air Force
• Formulation of Quasi-Linear Describing Function
  model of human response, based on classical
  control theory
• Characterized the Cross-over Model to explain
  human adaptation to changing physical dynamics
• Introduced the Precognitive Model to explain
  programmed behavior such as response to
  predictable signals.

13
The Quasi-Linear Operator Model
Noise
(TL s 1)
Yp Kp
E-t s
(TI s 1)
Human Operator
14
Cross-Over Model Why it Works

• Considers the human controller and the physical
  plant together in the aggregate
• Human adapts performance to compensate for the
  dynamics of the physical plant
• The nature of the adaptation produces a stable
  system in the region of wc, the crossover
  frequency where the the system is most sensitive
  to changes in gain, K.

15
The Cross-Over Model
16
Optimal Control Model

• In the late 1960s Modern Control Theory was
  sufficiently developed that David Kleinman and
  Shelly Baron, at BBN, applied it to the manual
  control problem
• Treats the control problem as a matrix of state
  space variables
• Defines a quantitative performance metric to be
  optimized, such as the mean square error
• Derives the ideal or optimal controller that
  minimizes this metric, subject to the constraints
  faced by human operators
• Assumes that a well-trained operator behaves like
  an optimal controller faced with the same
  constraints and performance limitations, i.e. it
  is a normative model like the signal detection
  model

17
Optimal Control Model
MotorNoise
Observation Noise
Kalman Estimator
Control Gains
Predictor
Delay
Filter
InputSignal
18
Quasi-Linear Models
Optimal Control Models

• Can be related to human information processing
  behavior
• Deals coherently with multi-variable control
• Derived automatically by computer program
• Requires definition of quantitative performance
  metric

• Provide good intuition about behavior of system
• System analysis is a trial and error process
  based on many interactive criteria
• Much subjective adjustment required for
  multi-variable, multi-loop systems

19
Network/Reliability Modeling Thread
Pert NetworksPritzger
20
Reliability Models

• The American Institutes for Research (AIR)
  Database collected reliability statistics for
  elemental human actions, i.e. operating
  controls, reading instruments for the US Navy
• Technique for Human Error Prediction (THERP)
  applies standard reliability equations to data
  from databases like that from AIR and invokes
  Performance Shaping Factors to account for
  human individual differences and environmental
  variables, etc.
• Human Reliability models are still used in human
  error analyses in complex process control system

21
Siegel Wolf Network Simulation

• Program a task network that captures the
  operations of a man-machine system
• Each action unit has a probability of success
  and a distribution of completion times moderated
  by a series of Performance Shaping Factors (PSFs)
  or Moderator Functions. PSFs were implemented as
  scale factors applied to the action units
• Aggregate probability of success and performance
  times are estimated by Monte Carlo Simulation of
  the overall network
• Significant effort devoted to developing PSF
  equations that map the effects of Psycho-social
  behavior e.g. performance stress, team
  cohesiveness and goal aspiration on to
  performance

22
SAINT/MicroSaint

• In 1974 the Air Force contracted with Pritzger
  Asssoc. to create SAINT, a general purpose
  discrete simulation language (in FORTRAN) to
  include the kinds of features in Siegel Wolfs
  work.
• Ron Laughery worked on SAINT for Pritzger. In
  1985 Ron, then having formed MicroAnalysis and
  Design, developed MicroSaint to run on a PC.

23
Human Operator Simulator (HOS)

• In 1969 Robert Wherry Jr., then with the Navy at
  Point Magu, conceived of a Human Operator
  Simulator (HOS)
• The idea was to have an easy-to-use procedure
  language (HOPROC) for programming task execution
  together with a collection of micro-models that
  could be called to represent the individual
  human performance processes
• This spawned the original version of HOS from
  Analytics, Inc. and later versions (HOS V) from
  Micro Analysis and Design

24
HOS Friends and Relations

• Chi Systems COGNET, Later COGGEN
• GOMS
• IPME

25
Cognitive Process Modeling Thread
Newell Simon (1963) General Problem Solver
Laird, Rosenbloom (1983) Soar
Meyer Kieras, (1997) EPIC
Card, Moran Newell (1983) GOMS
Broadbent(1958) Perception Communication
Anderson, Et Al. (1980) ACT/ACT-R
Neisser (1967) Cognitive Psychology
Modular Psychological Models
Chronology
26
Hybred Control/Procedure Models
27
PROCREW (1980)

• Simulation of aircrew during approach and landing
  of Boeing 727 type aircraft (before Flight
  Management Computers and glass cockpits)
• Shelly Baron generalized the Optimal Control
  Model to include crew decision making and
  procedural activities
• Decisions among competing activities based on
  probabilistic assessments and a gain function
  derived from mission impact
• Each procedural activity represented as a
  sub-model that may effect aircraft response, crew
  information state, and/or attentional distraction
  from flying the aircraft

28
Integrated Driver Model (1995)

• Prepared in connection with the Intelligent
  Transportation Systems Program to assess impact
  of driver information systems on driving behavior
• Combined cognitive information processing model
  (a Pre-D-OMAR BBN model), including Wickens
  multiple resource theory, with the optimal
  control model (Not yet a Cognitive Architecture)
• Considerable effort devoted to modeling attention
  sharing
• Compared favorably with simulator data on
  in-vehicle telephone use collected by Paul Green
  at UMTRI
• Predicted counter-intuitive empirical result of
  Noy (1991), that performance was better when the
  driver was looking away from the road

29
Current Status - Manual Control

• There has not been much innovation in Manual
  Control models since about 1980. The
  quasi-linear Model of MacRuer, et. al., and the
  Optimal Control Model are still pretty much the
  state-of-the art.
• Automation has been introduced into many systems
  where manual control was critical reducing the
  need for inner loop control
• There continue to be applications, mainly to
  aviation (fighter, civil aircraft), vehicle
  driving.

30
Current Status Network Models
31
GOMS variants and tool sets appear regularly

• CMN GOMS,
• NGOMSL,
• CPM GOMS,
• CRITIQUE

32
Current Status - Cognitive Architectures

• This workshop provides a good assessment of the
  state-of-the-art
• Enough said!

33
Current Status Hybred Models

• I know of no recent examples of control/procedure
  hybreds
• They are very relevant, particularly to
  controller/vehicle/information system
  interactions where precision manual control is an
  integral part of the task.
• Safety of driver information systems
• Some tasks of unmanned aerial vehicles
• Civil aviation

34
The Future

• I hope that some of you might consider
  integrating a model using your cognitive
  architecture with a control theory representation
  of vehicle control
• If so, this talk will have accomplished one of
  its goals .

35
Now for the Commercial
36
Some Genealogy