INTELLIGENCE, THINKING AND PERSONALITY

1
INTELLIGENCE, THINKING AND PERSONALITY

• Statistical Reasoning, Prediction, and Biases

2
OVERVIEW

• Kahneman and Tversky heuristics for statistical
  judgement
• Availability
• Representativeness
• Conjunction Fallacy
• Anchoring and Adjustment
• Illusory Correlation
• Overconfidence
• Hindsight Bias

3
KAHNEMAN AND TVERSKY (1973) HEURISTICS

• Availability
• Representativeness
• Anchoring and adjustment

4
AVAILABILITY HEURISTIC

• Heuristic for estimating probability
• Relies on
• Structure of memory
• Meta-cognitive ability

5
AVAILABILITY DEFINITION

• A person is said to employ the availability
  heuristic whenever he estimates frequency or
  probability by the ease with which instances or
  associations could be brought to mind

6
AVAILABILTY IN USE

• Examples
• Do more English words have R as the first or as
  the third letter?
• Famous name demonstration. Subjects believe that
  gender-balanced lists contain more women (men) if
  the women (men) are more famous.
• Ease of recall is often good clue to probability
  - but not always.

7
CAUSES OF DEATH STUDY - SLOVIC, FISCHHOFF
LICHTENSTEIN (1976)

• Subjects asked to estimate frequency of various
  causes of death.
• Cause S. estimate Truth
• Tornado 564 90
• Fireworks 160 6
• Asthma 506 1886
• Drowning 1684 7380
• (rates per 200m US residents per year)

8
WHY DO WE USE THE AVAILABILITY HEURISTIC?

• Availability is based on fundamental aspect of
  memory search
• We generally fail to get necessary feedback to
  correct availability judgements.
• Hard to cure, even if we are aware of it
• Similar in this respect to overconfidence (see
  later).
• Availability also affects our perception of
  everyday lives Ross Sicoly's study of members
  of couple's perceived responsibility for
  activities

9
REPRESENTATIVENESS HEURISTIC (KAHNEMAN TVERSKY,
1972)

• Heuristic for estimating probability based on
  similarity judgements.
• Similarity is another basic cognitive process
  (like structure of memory).

10
REPRESENTATIVENESS HEURISTIC - DEFINITION

• A person using the representativeness heuristic
  evaluates the probability of an uncertain event,
  or a sample, by the degree to which it
• (i) is similar in essential properties to its
  parent population
• (ii) reflects the salient features of the process
  by which it is generated

11
REPRESENTATIVENESS HEURISTIC JUSTIFICATION

• Similarity and probability are often highly
  related, so representativeness is a good
  heuristic most of time.
• But, like availability, it leads to systematic,
  predictable biases for certain tasks.

12
THE TOM W EXPERIMENTS KAHNEMAN AND TVERSKY
(1972)

• Subjects read a description of Tom W. Written
  by a psychologist when Tom was in high school.
• "Tom W. is of high intelligence although lacking
  in true creativity. He has a need for order and
  clarity, and for neat, tidy systems in which
  every detail fits in the appropriate place. His
  writing is rather dull and mechanical,
  occasionally enlivened by corny puns and flashes
  of the imagination of the sci-fi type. He has a
  strong drive for competence. He seems to have
  little feeling or sympathy for other people and
  does not enjoy interacting with others.

13
THE TOM W EXPERIMENTS - cont

• Question
• How likely is it that Tom is a graduate student
  in
• Humanities
• Computer Science
• 95 say Computer Science more probable

14
THE TOM W EXPERIMENTS - cont

• BUT
• there are 3 times as many graduate students in
  humanities as in CS (base rate)
• information is likely to be unreliable (because
  old, etc.)
• When information is unreliable, we should not
  revise belief much away from base rate (normative
  model Bayes theorem).

15
THE TOM W EXPERIMENTS - cont

• Subjects show general tendency to ignore base
  rates
• Subjects use representativeness (descriptive
  model).
• Tom W. is highly representative of CS graduate
  students (parent distribution 1) - not
  representative of Humanities graduate students
  (parent distribution 2).
• Thus subjects believe Tom is a CS graduate
  student.
• Representativeness ignores base rates.

16
REPRESENTATIVENESS HEURISTIC DEFINITION REVISITED

• The second part of the definition of the
  representativeness heuristic refers to the
  process by which an event or a sample is
  generated.

17
REPRESENTATIVENESS PROCESSES AND OUTCOMES

• Problem
• On each round of a game, 20 1 coins are
  distributed at random between 5 students
• Will there be more rounds of Type 1 or Type 2
  after playing the game 100 times?
• Person Type 1 Type 2
• Jim 3 coins 4 coins
• Sue 4 coins 4 coins
• Mary 5 coins 4 coins
• Pat 4 coins 4 coins
• Chris 4 coins 4 coins

18
REPRESENTATIVENESS PROCESSES AND OUTCOMES

• Type 2 is more probable, but Type 1 chosen much
  more often
• Reason We expect randomness to produce
  perturbations. Type 1 sample is more
  representative of this process than Type 2.

19
THE CONJUNCTION FALLACY TVERSKY KAHNEMAN
(1982)

• Linda is 31 years old, single, outspoken, and
  very bright. She majored in philosophy. As a
  student she was deeply concerned with issues of
  discrimination and social justice, and also
  participated in anti-nuclear demonstrations.
• Which of the following statements about Linda is
  more probable?
• She is a bank teller
• She is a bank teller who is active in the
  feminist movement.

20
THE CONJUNCTION FALLACY - WHY IS IT A FALLACY?

• Anyone who is a bank teller who is active in the
  feminist movement is also a bank teller.
• So, if Linda is a bank who is active in the
  feminist movement, she is also a bank teller.
• But, she could also be a bank teller but not
  active in the feminist movement.
• So, it is more likely that she is a bank teller
  than an bank teller who is active in the feminist
  movement

21
MATERNITY HOSPITAL PROBLEM

• A certain town is served by 2 hospitals. In the
  larger hospital about 45 babies are born each
  day. In the smaller hospital about 15 babies are
  born each day. As you know, about 50 of all
  babies are boys. The exact percentage of baby
  boys varies from day to day, however. Sometimes
  it will be higher than 50, sometimes lower. For
  a period of a year, each hospital recorded the
  days on which more than 60 of the babies born
  were boys.
• Which hospital do you think recorded more such
  days? Why?

22
MATERNITY HOSPITAL PROBLEM - RESULTS
23
JUDGEMENTS BY AND OF REPRESENTATIVENESS

• Judgements by representativeness
• Tom W, Linda
• People are judged to be members of groups because
  they seem representative of them
• Judgements of representativeness
• Maternity hospital problem
• Small samples are taken to be representative of
  the population from which they are drawn.
• These two uses of representativeness are
  logically independent of one another

24
REPRESENTATIVENESS AND THE GAMBLERS FALACY

• Representativeness can also explain the Gambler's
  Fallacy (the belief that an event - e.g., red on
  a roulette table- is likely to come up now
  because it is due e.g., after a run of black).

25
ANCHORING AND ADJUSTMENT

• Final heuristic for estimating probabilities but
  also applies to any quantitative estimate
• Stage 1 Person starts with initial idea of
  answer (anchor)
• Ball park estimate.
• Anchor may be suggested by memory, or by
  something in environment.
• Stage 2 Person adjusts away from initial anchor
  to arrive at final judgement.

26
WHY ANCHORING AND ADJUSTMENT MIGHT BE A BAD IDEA

• Problem Adjustments are generally inadequate.
  Final estimate is too closely tied to anchor
• Suggests that you can bias persons estimate if
  you provide the initial anchor

27
ANCHORING AND ADJUSTMENT AN EXPERIMENTAL STUDY

• Kahneman Tversky 1974
• Task Suppose you randomly pick the name of one
  of the countries in the UN. What is the
  probability that this country will be an African
  country?

28
ANCHORING AND ADJUSTMENT AN EXPERIMENTAL STUDY -
cont

• Stage 1 A wheel-of-fortune is spun and yields a
  random number, 1 - 100.
• Stage 2 The subject is asked whether the actual
  percentage of African countries in UN is higher
  or lower than number in Stage 1 (Supplies anchor)
• Stage 3 The subject is asked to arrive at final
  estimate by moving up or down from Stage 1
  number.

29
ANCHORING AND ADJUSTMENT AN EXPERIMENTAL STUDY -
cont

• Results
• When Stage 1 number was 65, mean estimate was 45
• When Stage 1 number was 10, mean estimate was 25
• Subjects are inappropriately swayed by random
  anchor.

30
ANCHORING AND ADJUSTMENT - AN EVERYDAY EXAMPLE

• Car dealer attempts to anchor you to windscreen
  price on car
• Combat by anchoring on price dealership paid
• Problem with using anchoring and adjustment
  heuristic is sticking too close to bad anchor.

31
CURING ANCHORING AND ADJUSTMENT

• Be aware of the problem - try to choose different
  anchor and see effect on solution
• Anchor, or be anchored!
• Get good feedback constantly modify your
  predictions with feedback from environment. Will
  help eliminate effect of bad anchor.

32
KAHNEMAN AND TVERSKY HEURISTICS - CONCLUSIONS

• There is a large literature on hypothesis testing
  and prediction showing subjects are make errors
• They deviate from normative model (e.g. logic,
  probability theory)
• Errors are not random - they all show the same
  biases - suggests common causes (heuristics).
• Does use of heuristics mean subjects are stupid?
  NO!

33
KAHNEMAN AND TVERSKY HEURISTICS - CONCLUSIONS

• We use heuristics because they are generally
  useful, but they predictably get us into trouble
  on certain (well-studied) tasks
• Heuristics help us avoid
• information processing limitations (e.g., lack of
  STM capacity, lack of processing power)
• (some) information processing biases (e.g., bias
  in recall, storage), but they produce other
  biases
• time limitations
• Some errors can be avoided by education,
  feedback, seeking multiple perspectives. It is
  worth avoiding these biases when correct results
  are important.

34
OTHER HEURISTICS AND BIASES

• Illusory Correlation
• Overconfidence
• Hindsight Bias

35
ILLUSORY CORRELATION

• First studied by Chapman Chapman (1969)
• Finding subjects who believe that two
  events/properties etc. are correlated will
  bolster this belief when exposed to either
  neutral or mildly disconfirmatory, data

36
EXPERIMENT ON ILLUSORY CORRELATION

• Clinical psychologists exposed to sets of stimuli
  from Draw a Person Test each labelled with
  diagnosis e.g. depression
• Clinicians asked to evaluate the relationship for
  that data set between features of drawing and
  diagnoses
• Example Is there a correlation between big eyes,
  and a diagnosis of paranoia?
• Results Clincians indicated strong positive
  relationship as present, if they had strong prior
  belief in relationship.
• Replicates even with weak negative correlation in
  data.

37
WHAT CAUSES ILLUSORY CORRELATION?

• MANY CANDIDATES
• Confirmation bias actively seek only confirming
  instances in retrospect
• Availability confirming instances may be more
  available in memory (either through storage or
  retrieval).
• Social factors unwillingness to admit to having
  been wrong.
• Laziness/cognitive miser account, Hypothesis
  error implies hypothesis revision, which is
  effortful,
• Kuhn (Theory of Scientific Revolutions) can only
  replace an existing theory with a new, better
  theory. (Says Popperian falsification is naive).
• How can illusory correlation be cured?

38
HINDSIGHT BIAS

• People are overconfident in their ability to have
  been able to predict an event in the future, once
  they know that it has occurred. (Fischhoff,
  1975).
• Example Subjects read historical passage about
  war between British and Ghurkas.
• Experimental groups
• told British won
• told Ghurkas won
• told stalemate
• Control group
• not told outcome

39
HINDSIGHT BIAS - cont

• Each group asked to say how likely they would
  have thought each outcome before knowing result.
• Result Ss grossly overestimate their ability to
  predict the future.
• Ss think that they would have estimated P(British
  win) much higher if they have been told British
  won.

40
HINDSIGHT FOR REAL EVENTS

• Hindsight effect also replicates for actual
  events.
• Fischhoff Beyth (1975) asked US subjects to
  make predictions about Nixon's visit to China
  before it occurred.
• e.g., What is the probability that Nixon will
  meet Mao?
• Phase 2 asked same subjects to recall these
  probabilities after the event.

41
FISCHHOFF BEYTH - RESULTS

• For events that actually occurred recalled
  probability tended to be higher than predicted
  probability.
• For events that did not occur, recalled
  probability tended to be lower than predicted
  probability,

42
CURING HINDSIGHT BIAS

• Hindsight bias appears to be caused by lack of
  availability in memory of alternative hypotheses
  to known event outcome.
• Remedy encourage subject to think of
  alternatives.
• e.g. Fischhoff (1975) reduced hindsight bias by
  asking subjects to think of reasons that the
  favoured party might not have won the war.

43
OVERCONFIDENCE

• Confidence in an events happening should ideally
  reflect its probability of happening.
• Example - on days when a weather forecaster says
  there is a 70 chance of rain the next day, it
  should rain on 7 out of 10 of those next days.

44
WEATHER FORECASTERS

• Good weather forecasters are almost perfectly
  calibrated.
• Novices (in any domain) tend to be overconfident
• Except where judgements are near chance when they
  tend to be slightly underconfident

45
NOVICE CALIBRATION CURVE
100
diagonal represents perfect calibration
actual probability
100
chance
judgement
46
OTHER EXPERTS

• Bridge players are well calibrated (on
  probability that a contract will be made)
• Medical experts tend to be overconfident in the
  correctness of their diagnoses

47
WHAT CHARACTERISES SITUATIONS IN WHICH EXPERTISE
AIDS CALIBRATION

• FEEDBACK
• EXPLICIT TRAINING
• Weather forecasters collect outcome data and
  actually plot calibration curves (and their pay
  may depend on their accuracy).
• Bridge players get immediate feedback (win or
  lose contract).
• Learning can only occur when there is a mismatch
  between outcome and expectation.

48
WHAT CHARACTERISES SITUATIONS IN WHICH EXPERTISE
AIDS CALIBRATION (Cont)

• Doctors often get no feedback (patient is
  transferred patient dies patient gets better
  anyway diagnosis is never disconfirmed). No
  opportunity to learn.
• Until recently, they did not make probabilistic
  estimates of the accuracy of their diagnoses.
• Confirmation bias in search can lead to increased
  confidence but not accuracy.

49
CURING OVERCONFIDENCE

• Encouraging people to think of alternatives also
  works as a general technique against
  overconfidence
• Hoch (unpub) asked MBA students to predict their
  chances on job market.
• Group 1 Give reasons supporting
• Group 2 Give reasons pro and con
• Group 2 significantly less overconfident