The influence of hierarchy on probability judgment

1
The influence of hierarchy on probability
judgment

• David A. Lagnado
• David R. Shanks
• University College London

2
Level of hierarchy can modulate judgment

• Consider two statements about the next World Cup
• It is most likely that Brazil will win
• It is most likely that a European team will win
• These appear to support opposing predictions, but
  both may be true
• Shows the importance of the level at which
  probabilistic information is represented

3
Hierarchical structure

• Pervasive feature of how we represent the world
• Reflects pre-existing physical and social
  hierarchies
• Readily generated through conceptual combination
• Category hierarchies serve both to organize our
  knowledge, and to structure our inferences

4
Inference using a hierarchy

• One powerful feature of a category hierarchy is
  that given information about categories at one
  level, you can make inferences about categories
  at another level.
• This allows you to exclude alternatives, or
  reduce the number you need to consider

5
Probabilistic Inference using a hierarchy

• In many real-world situations we must base our
  initial category judgments on imperfect cues,
  degraded stimuli, or statistical data.
• What effect do such probabilistic contexts have
  on the hierarchical inferences that we are
  licensed to make?

6
Commitment heuristic

• Commitment heuristic - When people select the
  most probable category at the superordinate
  level, they assume that it contains the most
  probable subordinate category.
• This leads to the neglect of subordinates from
  the less probable superordinate.

7
How adaptive is this heuristic?

• The efficacy of such a heuristic depends on the
  precise structure of the environment.
• In certain environments it confers considerable
  advantages
• increases inferential power by focus on
  appropriate subcategories
• reduces computational demands by avoiding complex
  Bayesian calculations.
• But in some environments it can lead to anomalous
  judgments and inferences.

8
Non-aligned hierarchy
Tabloid 60
Broadsheet 40
Times 5
Guardian 35
Mirror 30
Sun 30

• In the above sample the most frequently read type
  of paper is a Tabloid, but the most frequently
  read paper is a Broadsheet (the Guardian).
• Non-aligned hierarchy the most probable
  superordinate category does not contain the most
  probable subordinate category.

9
Real world examples

• Word frequencies the superordinate BE- is more
  frequent than BU-, but the subordinate BUT is
  more frequent than any of the other subordinates
  (BET, BEDetc.)
• NHS statistics on survival rate for operations
  for different areas sub-areas
• You are more likely to survive a hip operation in
  Surrey rather than Essex, but the best sub-area
  for survival is Colchester (in Essex).

10
Experiments 1 and 2

• Learning phase - participants exposed to a
  non-aligned hierarchical environment in which
  they learn to predict voting behavior from
  newspaper readership.
• 100 trials reading/voting profiles

11
Screen during learning phase
Broadsheet
Tabloid
Chronicle
Herald
Reporter
Globe
? Liberal
? Progressive
12
Screen during learning phase
Reading profile for J. K.
Broadsheet
Tabloid
Chronicle
Herald
Reporter
Globe
? Liberal
? Progressive
13
Screen during learning phase
Reading profile for J. K.
Broadsheet
Tabloid
Chronicle
Herald
Reporter
Globe
? Liberal
Outcome feedback
? Progressive
14
Structure of environment
Tabloid 60
Broadsheet 40
Times 5
Guardian 35
Mirror 30
Sun 30
Party A Party B
50
50
15
Judgment phase
What is the probability that X votes for one
party rather than the other?
Baseline
X is selected at random
Which type of paper is X most likely to read?
Type
Which paper is X most likely to read?
Paper
16
Results of Experiment 1

• Probability ratings for Party B rather than Party
  A with judgments divided into those based on
  aligned and non-aligned choices

17
Experiment 2

• Replication of Experiment 1, with frequency as
  well as probability response formats
• Frequentist hypothesis that probability biases
  reduced with frequency format

18
Results of Experiment 2

• Mean ratings for Party B rather than Party A
  collapsed across probability and frequency ratings

19
Summary of Results

• Participants allow their initial probability
  judgment about category membership (newspaper
  readership) to shift their rating of the
  probability of a related outcome (voting
  preference), even though all judgments are made
  on the basis of the same statistical data.
• When their prior choices were non-aligned this
  led to a switch in predictions about the outcome
  category

20
Conclusions

• These biases are explicable by the Commitment
  heuristic
• The priming question commits people to just one
  inferential path, leading them to compute an
  erroneous estimate for the final probability.
• This is understandable given the complexity of
  the normative Bayesian computation.

21
Comparison of Bayesian and commitment heuristic
computations (just type level inference)
Type of paper? Type of paper?
0.4
0.6
0.6
Tabloid Broadsheet
Tabloid
0.23
0.1
0.9
0.77
0.77
Party A Party B
Party A
P(A) (0.6 . 0.77) (0.4 . 0.1) 0.46
0.04 0.5
P(A) 0.77
Bayesian computation
Simplified heuristic computation
22
Conclusions

• Simplifying heuristic that assumes that
  environment is aligned
• Empowers inference when hierarchical structure is
  aligned, otherwise can lead to error
• Suggests tendency to reason as if a probable
  conclusion is true

23
Process level accounts

• Associative model
• People learn predictive relations between
  category options (at both levels of hierarchy)
  and outcome. At test responses to category
  questions prime the appropriate associations and
  lead to a biased rating of the outcome.
• Frequency-based model
• People encode event frequencies in the learning
  phase. At test responses to the category question
  serves as the reference class for subsequent
  conditional probability judgments about voting
  preferences.

24
Implications

• Importance of the level at which probabilistic
  data is represented to (or by) a decision maker
• E.g., using NHS statistics to decide on hospital
• How do people search through hierarchical
  statistical data?
• Peoples judgments can be manipulated by the
  level at which statistical information is
  represented
• More generally, in multi-step inferences people
  are susceptible to biased probability judgments