Outline

1
Outline

• Case Studies Testimonials
• Case studies as evidence
• The placebo effect
• The vividness issue
• Correlation Causation
• Third variable problem
• Directionality
• Selection bias

2
Case studies testimonials

• Case studies
• in-depth analyses of particular individuals
• cannot be used to establish causal relations
• Testimonials
• positive reviews of a person or product

3
Case studies as evidence

• Historically important in Psychology
• Used by Freud
• Psychophysics
• Complex system can go wrong in complex ways
  adequate description may require such lengthy
  observation that case study is only appropriate
  approach, at least initially

4
Case studies as evidence

• Most useful in early stages of research in any
  area
• Useful for development of theories hypotheses
• Observation may lead to identification of
  relevant variables that are controlling or
  influencing behavior - leading to theory building

5
Case studies as evidence

• Less useful for testing theories
• Case studies cannot rule out alternative
  hypotheses
• Testing theories requires a manipulation
  changing conditions under which people perform a
  task to see if variation in performance predicted
  by a theory occurs
• That process lets you pit one theory against
  another, and see which (if either) wins

6
Case studies as evidence

• The critical idea here is that there can be
  competing accounts of why something happened.
• Our task as scientists is to compare competing
  accounts and select one.

7
Case studies as evidence

• If we are to adjudicate between competing
  theories, we have to put them in jeopardy.
• Jeopardy means that we give all the competing
  theories a chance to be wrong.
• If one of them isnt wrong in that situation,
  but the others are, then we reject those others
  and adopt the one that survived the test.
• Manipulation based on theoretical predictions
  creates the required situation

8
Case studies as evidence

• We have to discriminate between alternative
  accounts
• If there isnt a manipulation that lets you
  adjudicate between two theories that is, if
  they dont make different predictions about
  performance then you dont have two theories,
  you only have one.
• So, whenever you truly have two theories, you
  must be able to test them. But not with case
  studies.

9
Case studies as evidence

• The problem with case studies is that they dont
  let you create the jeopardy situation
• One reasonable account of any improvement in the
  condition of a case study subject is the placebo
  effect.
• A second issue is that observations in case
  studies can be biased by the vividness effect.

10
Placebo Effect

• Placebo effect tendency to report that a
  treatment has helped, regardless of whether it
  has a real therapeutic effect.
• Control group gets a fake treatment
• Experimental group gets the real treatment

11
Placebo Effect

• If the two groups show the same amount of
  improvement, then the improvement is due to
  placebo effect.
• If control group shows any improvement, part of
  treatment effect is placebo effect.

12
Placebo effect

• This is about the power of belief e.g.,
  -physicians say, when a new drug is available,
  Hurry up and use it while it still works
• Hawthorne Effect peoples performance got
  better when other people were being observed
  possibly because someone cares
• Turing Test a computer program is intelligent
  if you cant tell whether youre talking to human
  or computer problem people see meaning

13
Placebo effects

• Bower (1996) Placebo effect of Prozac twice as
  large as effect of the drug itself.

• Greenberg Fisher
• The limits of biological treatments for
  psychological distress comparisons with
  psychotherapy panacea (1989)
• From placebo to panacea Putting psychiatric
  drugs to the test (1997)

14
Placebo effects

• Benefits of psychoactive drugs are largely due to
  placebo effects

• Brown (SciAm 1998) expectations have real,
  measurable effects prescribe placebos!

15
Placebo effects- conclusions

• Placebo effects are widespread, so they are
  always present as an alternative account of why
  someone got better when they got a therapy.

• You cannot rule out placebo effects in case
  studies.
• People who got the benefit of placebo effects may
  offer testimonials, not knowing that it wasnt
  their therapy that made the difference.

16
The Vividness Issue

• Retrieval from memory is influenced by vividness.
• E.g. which is safer, travel by car or travel by
  airplane?
• Statistically, travel by plane is much safer, but
  many people think it is more dangerous. Why?
• One account reports of airplane crashes are
  vivid, so easily retrieved from memory ease
  biases estimate of whether flying is safe.

17
The Vividness Issue

• Stanovich
• lt680 car, truck, motorcycle accident deaths/wk
  in the U.S.
• But (Stanovich says) people dont call for
  action because they dont recognize the problem
• Lots of reports of one or two deaths arent as
  vivid as one report of an airplane crash with
  hundreds of deaths

18
Criticism

• In the case of plane vs. car travel, Stanovich
  doesnt know that the traffic safety problem is
  not recognized thats just his opinion.
• He only knows that people dont respond as
  dramatically to traffic deaths as they would to
  similar numbers of plane crash deaths
• But theres a difference between not calling for
  investigations and not responding

19
Criticism

• Gerald Wilde, Queens University homeostatic
  theory of risk people set an acceptable risk
  rate for activities like driving
• If driving conditions change in such a way as to
  move drivers away from their preferred rate, they
  adjust their behavior to move back to that rate.

20
Criticism

• e.g., better brakes? Drive faster.
• speed limit reduced? Pay less attention
• people are sensitive to driving risks their
  behavior does vary in ways that are sensitive to
  risks
• but they may not be aware of that variation in
  their behavior or be able to articulate those
  risks as statistics.

21
Criticism

• Perhaps professors (including Stanovich)
  over-value the ability to articulate knowledge
  and undervalue the ability to modulate behavior
  without such knowledge.

22
The Vividness Issue

• In the new (8th) edition of his book, Stanovich
  has added a qualification to his comments about
  vividness, acknowledging that he uses vivid
  examples himself.

• He now says that he uses vivid examples to make
  his points memorable, but he also provides
  citations to scientific evidence for those
  points. The latter constitute proof of those
  points.

23
The Vividness Issue

• Stanovich says that he uses vivid examples only
  because they are memorable so in his mind,
  presumably, the reader will be persuaded by the
  cited scientific studies, not the vivid examples

• But will this be the case? Will his distinction
  between illustration and proof be as
  important to readers as it is to him?

24
The Vividness Issue

• Lets distinguish between the ideas involved
• 1. Do vivid examples prove claims?
• 2. Are ordinary people unsophisticated consumers
  of evidence?
• 3. Does vividness play any useful role in our
  thinking?

25
1. Do vivid examples prove claims?

• No. As a technical point, Stanovich is quite
  right on this score.

26
2. Are ordinary people unsophisticated consumers
of evidence?

• No. As Wildes homeostatic theory of risk, and
  the evidence he reports, suggest, ordinary people
  respond rationally to changes in conditions,
  though they may not be able to articulate those
  changes or the basis for their responses.
• The wisdom of crowds

27
3. Does vividness play any useful role in our
thinking?

• Our liking for vivid examples which Stanovich
  himself takes advantage of reflects a basic
  fact about human cognition
• Individual examples help humans to comprehend
  events or ideas that are otherwise beyond
  understanding. As Stalin said, 1 death is a
  tragedy. 1 million deaths is a statistic.

28
3. Does vividness play any useful role in our
thinking?

• As Stalin said, 1 death is a tragedy. 1 million
  deaths is a statistic.
• To illustrate this point, lets use as an example
  the Holocaust

29
The Holocaust

• 6 million Jews, and many other people (Gypsies,
  gays) were murdered in Germany and
  German-occupied territories during World War II.
• A terrible thing but how terrible?
• We dont have any measuring scale that lets us
  answer that question.
• Does it help to learn about one victim?

30
Irène Némirovsky

• Russian-born Jewish writer
• Moved to France as a child, in 1919, following
  Russian Revolution
• Published first novel in 1929 celebrated as a
  brilliant author
• She was refused French citizenship in 1938.
• Left Paris after German occupation of France in
  1940, went to live in a rural village with
  husband 2 daughters
• Continued writing

31
Irène Némirovsky

• Paper was scarce, so she wrote in tiny letters,
  with a magnifying glass
• Wrote a novel about the German occupation of
  France

• Arrested by French police in July 1942 in front
  of her two young daughters.
• Told the daughters she was going away on a trip

32
Irène Némirovsky

• Died of typhus in a German concentration camp one
  month later
• Husband arrested, died in Auschwitz gas chamber
  in November 1942

• Daughters guarded by Catholic nuns priests
  through the war, and provided for by Irènes
  publisher a friend during and after the war

33
Irène Némirovsky

• Daughters survived the war, kept a suitcase
  containing their mothers last writings.
• 30 years later, daughter Dianne spent 20 years
  transcribing Irènes novel Suite Francaise,
  working with a magnifying glass.

• Book was published in France in 2004, and a
  tremendous success though Némirovsky had been
  long forgotten.
• Published in Canada in April 2007

34
Vividness

• So, which helps you understand the horror of the
  Holocaust better the statistic, 6 million
  murdered or the story of Irène Némirovsky?
• Both are important, but in different ways

• So, vividness can be an important aid to thought
• And if vivid examples are filtered through many
  peoples evaluations, perhaps that process
  produces a useful conclusion

35
Correlation and Causation

• Basic point Correlation does not imply causation
• Goldberger example cause of pellagra
• Goldberger used a manipulation to test causal
  accounts

• He put the germ theory in jeopardy and showed
  that it was wrong
• He also put his diet theory in jeopardy and
  showed that it was right.

36
Correlation and causation

• Third variable problem
• Directionality
• Selection bias

37
Third variable problem

• Relationship between two variables A and B might
  be produced by a third variable which influences
  both.

• E.g., correlation between of births in German
  cities and of storks in German cities, in 19th
  century.
• Families ? houses ? chimneys ? storks

38
Correlation and Causation

• But note the example Stanovich uses the
  relation between SES and going to university.
• Correlation is still there even when academic
  aptitude effects have been removed statistically.
• S says this finding strikes at the heart of the
  merit-based goals of our society.
• So, hes concluding that SES controls university
  attendance, even though he admits that we dont
  know the reason for the correlation.

39
Directionality

• Poor reading example researchers once thought
  that poor reading caused by improper
  eye-movements.

• We now know that causation works in the other
  direction poor reading skill produces impaired
  eye-movements.

40
Directionality

• Self-esteem and academic achievement lots
  people have claimed that low self-esteem leads to
  low academic achievement.

• In self-esteem case, there is evidence that poor
  academic achievement leads to low SE and good
  academic achievement leads to high SE (other
  things being equal)

41
Selection bias

• relationship between subject and environmental
  variables arises because people with different
  characteristics select different types of
  environments.

• E.g., Arizona and respiratory illness.
• SAT scores teacher salaries example