Sources of error: Information bias

1
Sources of error Information bias
Principles of Epidemiology for Public Health
(EPID600)
Victor J. Schoenbach, PhD home page Department of
EpidemiologyGillings School of Global Public
HealthUniversity of North Carolina at Chapel
Hill www.unc.edu/epid600/
2
Abort, Retry, Fail

• Tips for safer drives Never turn off a PC or
  accessories while the computer is on or the disk
  is active.
• USA Today
• PC Magazine, 10/3/1996

3
Chapter 1 THE HISTORIAN'S TASKInsight into the
future

•   History, a record of things left behind by past
  generations, started in 1815. Thus we should try
  to view historical times as the behind of the
  present.
• Anders Henriiksson (ed), Non Campus Mentis, NY,
  Workman Publishing Co., 2003

4
Non Campus Mentis

• History, as we know, is always bias, because
  human beings have to be studied by other human
  beings, not by independent observers of another
  species.
• Anders Henriiksson (ed), Non Campus Mentis, NY,
  Workman Publishing Co., 2003, chapter 1

5
Information bias

• Information bias a systematic distortion or
  error that arises from the procedures used for
  classification or measurement of the disease, the
  exposure, or other relevant variables.

6
Information bias

• Classification or measurement
• Differential or nondifferential bias
• Direction of bias
• Misclassification of covariables

7
Classification or measurement

• Data for epidemiologic studies consist of
  classifications (e.g., hypertensive vs.
  normotensive) or measurements (e.g., 120 mmHg
  systolic BP).
• Possible sources of measurement or classification
  error include instrumentation, laboratories,
  records, respondents data collectors, managers,
  analysts, and interpreters.

8
Sources of measurement error

• Respondent (interview, questionnaire)
• inability to understand, recall, articulate
• unwillingness to disclose
• social desirability influences
• Can be influenced by wording of questions and how
  they are asked.

9
Example of misunderstanding

• Medico Não consigo encontrar o motivo das suas
  dores, meu caro. Só pode ser por causa da
  bebida.
• Paciente Não tem importãncia, doutor. Eu volto
  outro dia que o senhor estiver sóbrio.
• De Luciana V. Paiva, Osasco - SP, en Bom Humor
  Nosso E Dos Leitores, Almanaque Brasil de
  Cultura Popular. Maio 20013(26)
  (almanaquebrasil_at_uol.com.br). Exemplar de quem
  viaja TAM.

10
How not to ask questions

• Has anyone ever tried to give you the mistaken
  idea that sex intercourse is necessary for the
  health of the young man?
• (from a survey by the NC state health officer,
  circa 1926, summarized in Kinsey et al., 1948)
• Can you guess the right answer?

11
Respondent cognitive processes

• Respondent cognitive processes interpretation,
  recall, judgment formation, response formatting,
  editing
• Qualitative research on response processes, e.g.
• What types of physical activity or exercise did
  you perform during the past month?
• What did you think we meant when we said
  physical activity?
• Which, if any, of the following would you (also)
  consider to be physical activity?
• www.minority.unc.edu/institute/2000/materials/slid
  es/RichardWarnecke-2000-06-08.ppt

12
Cognitive testing - 2

• Recalling and retrieving Retrieval probes
• Recall strategy
• Recall interval
• Search strategy (proximal, distal, anywhere)
• Long term recall - link to events to help
  remember
• Recall frame of reference--what kinds of things
  helped you remember?
• www.minority.unc.edu/institute/2000/materials/slid
  es/RichardWarnecke-2000-06-08.ppt

13
Surveys and Questionnaires

• Survey validation
• Pretesting (wording, item sequence, time)
• Pilot testing (all steps - procedure, item
  performance)
• Translation validation

Sources of error Information bias
13
14
Sources of measurement error

• Data collector unclear or ambiguous questions,
  lack of a neutral demeanor, insufficiently
  conscientious, inaccurate transcription, fraud

15
Sources of measurement error

• Data managers inaccurate transcription,
  mis-reading, miscoding, programming errors
• Data analysts variable coding and programming
  errors
• Data interpreters inadequate appreciation of
  the characteristics of the measure or of the
  relations being studied

16
Techniques for avoiding data collection errors

• Precise operational definitions of variables
• Detailed measurement protocols
• Repeated measurements on key variables
• Training, certification, and re-certification
• Data audits (of interviewers, of data centers)
• Data cleaning visual, computer
• Re-running all analyses prior to publication

17
Validation and agreement

• Sensitivity and specificity used to evaluate
  classifications
• When no validation standard, we measure agreement
• Measures of agreement often correct for chance

11/5/2009
Sources of error Information bias
17
18
Information bias differential or
non-differential

• Important question for any kind of bias are
  error processes different for groups being
  compared
• If no, non-differential
• If yes, differential
• Has implications for direction of bias
• In general, non-differential is safer

19
Direction of bias

• Upward
• Downward
• Towards the null
• Away from the null
• Null 0 (for differences) 1 (for
  ratios)

20
Direction of bias

• In simple situation, information bias is towards
  the null IF
• 1. Dichotomous exposure and disease
• 2. Non-differential misclassification with both
  sensitivity and specificity each greater than
  0.5 AND
• 3. Errors in one variable are independent of
  errors in the other

21
Errors in covariables

• It is almost always important to control for
  other variables (e.g., age)
• Errors in measurement of these variables hamper
  attempts to control for them
• Direction of bias is generally unpredictable

22
Classic studies 6 degrees of separation?

• Classic experiment by Yale psychologist Stanley
  Milgram asked people in Kansas to forward a
  letter to a target person in Massachusetts
• If did not know target person, then send it to
  someone they thought might know him
• Milgrams 1967 paper reported that it only took 5
  jumps, on average, for letters to arrive

23
Selection bias in a classic study

• According to Judith Kleinfeld, psychologist at
  the University of Alaska, Fairbanks, archives
  reveal that only 30 of the letters actually
  reached their destination!
• (Gewolb, Josh. Random samples. Science 26
  October 2001294777. See Kleinfeld, Judith S.
  Society. Jan/Feb2002 39(2)61-66)

24
Dr. Kinsey and the Institute for Sex Research

• Alfred S. Kinsey (photograph from Wardell B.
  Pomeroy, Dr. Kinsey and the Institute for Sex
  Research)

25
Sex research in the mid-20th century
Alfred S. Kinsey (photograph from Wardell B.
Pomeroy, Dr. Kinsey and the Institute for Sex
Research)
26
Kinsey et al. on selection and information bias

• (Alfred C. Kinsey, Wardell B. Pomeroy, Clyde E.
  Martin. Sexual behavior in the human male, Phila,
  W.B. Saunders, 1948)

Alfred S. Kinsey (photograph from Wardell B.
Pomeroy, Dr. Kinsey and the Institute for Sex
Research)
27
(No Transcript)
28
Real-life example Quit for Life

• Randomized trial of smoking cessation
  interventions
• Self-reported In the past 7 days, have you
  smoked a cigarette, even a puff?
• Attempted (unsuccessfully) to validate with
  saliva cotinine
• People who did not give a time to be called for
  validation had very high quit rates!

29
Kinsey biography by Wardell Pomeroy

• Pomeroy, Wardell B. Dr. Kinsey and the Institute
  for Sex Research NY, Signet / New American
  Library, 1972 p136

30
Non Campus Mentis

• Hindsight, after all, is caused by a lack of
  foresight.
• Anders Henriiksson (ed), Non Campus Mentis, NY,
  Workman Publishing Co., 2003, chapter 1

31