Observational Studies in Clinical Research

1
Observational Studies in Clinical Research

• 10 October 2005
• Viktor E. Bovbjerg, PhD MPH
• Department of Health Evaluation Sciences

Life is short, science is long opportunity is
elusive, experiment is dangerous, judgment is
difficultHippocrates
2
Study designs
3
Classic RCT design
Comparison of outcome rate between treatment and
control group provides estimate of effect
4
Relative merits of RCTs

• Advantages
• strong claims for causality
• control of confounding, many biases,
• tight control on exposure/treatment
• high internal validity
• possible to examine multiple outcomes

• Disadvantages
• time consuming
• expensive, resource intensive
• compliance, drop-out
• sometimes severe ethical constraints
• may not mirror practice
• generalizability may be limited (i.e. selection
  bias)

5
Observational studies

• Goal obtain results that would have occurred if
  RCT had been conducted
• Observational studies must
• minimize random error (precision, statisical
  significance)like RCTs
• control mixing of effects (confounding)
• minimize systematic error in participation and
  data (bias)
• Extent to which confounding and bias are
  controlled determine credibility

6
(Some) roles of descriptive, observational studies

• Describing patient, at risk populations, health
  care use
• Documenting natural history of disease, treatment
• Identifying etiological factors in disease
  development, outcomes
• Evaluating therapy in practice, post-marketing
  surveillance
• Setting the stage for intervention trials

7
Descriptive studies

• Used to describe disease patterns and generate
  hypotheses
• extremely limited in ability to shed light on
  causes no attempt to test causal hypotheses
• Often the first step in learning about disease
  etiology, patient outcomes
• Relatively easy, rapid, and inexpensive
• Use diverse, often existing, sources of data

8
(No Transcript)
9
Cohort studies
Crucial comparison outcomes in persons with vs.
without predictor variable (or with different
levels of predictor)

10
Relative merits cohort studies

• Advantages
• Clear temporal relationship
• Least susceptible to some forms of bias
• Can examine multiple predictors of outcome
• Efficient for rare exposures
• Useful when RCT infeasible, unethical

• Disadvantages
• No control over predictor (vs. RCT)
• confounding
• Inefficient for rare or long-latent diseases
• Loss to follow-up threatens validity
• Potential bias in outcome ascertainment
• Relatively resource- and time-intensive

11
Case-control studies
Hypothetical study population
no investigator control
ascertainment
Controls
Cases
classification on outcome
previous exposure of interest
time
Comparison of exposure rates between cases and
controls provides estimate of effect (usually
odds ratio OR)
12
Relative merits case-control studies

• Advantages
• Efficient use of time, resources
• Efficient for rare outcomes
• Efficient for outcomes with long latency
• Can assess multiple exposures
• Best when cohort study infeasible, RCT unethical
  (e.g. harm)

• Disadvantages
• Inefficient for rare exposures
• Difficult to identify appropriate controls,
  identify study base
• Ascertaining previous exposure often difficult
• records, recall
• Potential bias in predictor ascertainment,
  confounding

13
Challenges for observational studies
14
Bias

• Deviation of results or inferences from the
  truth, or processes leading to such deviation.
  Any trend in the collection, analysis,
  interpretation, publication or review of data
  that can lead to conclusions that are
  systematically different from the truth.
• Last JM. A dictionary of epidemiology. New York
  Oxford University Press, 1995.
• Bias is a systematic deviation from the truth
  that distorts the results of research.
• Sitthi-amorn C, Poshyachinda V. Bias. Lancet
  1993 342 286-288.
• Not dealt with by traditional statistical
  methods, reporting (e.g. p-values)

15
Selection bias

• Participant selection procedures that result in
  an effect estimate different from that which
  would have been obtained from the entire
  population the study sought to characterize
• Potential exists when factors influencing
  participation in studies are also associated with
  outcomes of the study

16
Selection bias?Fluticasone vs. prednisone in
pediatric asthma

• Study period, 10/95 - 4/99
• 8001 pediatric asthma ED visits
• Exclusions
• no study nurse present 2084
• too young (lt5 years) 5196
• unable to perform spirometry 264
• already on study tx 236
• already enrolled 73
• Parent refusal 45 (30.4 of eligible)
• Randomized 103

N Engl J Med 2000 343 689-94.
17
Information bias

• Systematic failure in data collection that
  results in data which differ in quality or
  accuracy by predictor, outcome, or other
  important study factor
• Results from differential misclassification or
  systematic error in data collection, where errors
  are non-random

18
Lead time bias
Example of lead time bias breast cancer survival
with and without mammogram
First cancerous cells
Mammogram
M M M- M-
apparent survival time
Clinical exam
apparent survival time
Time
19
Confounding
Both the predictor and confounder are associated
independently with the outcome, but the predictor
and confounder are also associated
20
Dealing with confounding

• Randomization factors should be evenly
  distributed among study groups
• Restriction limiting study participants to have
  like values on confounding factors
• Stratified analysiseasily understood, difficulty
  with multiple confounders
• Modeling/statistical adjustment
• Confounding can not be assessed or compensated
  for if data are not collected or not collected
  accurately

21
Deciding on data sources
Accuracy and practicability of data collection
methods are often inversely correlated. A method
providing more satisfactory information will
often be a more elaborate expensive, or
inconvenient one Accuracy must be balanced
against practical considerations and that method
chosen which will provide the maximal accuracy
within the bounds of the investigators resources
and other practical limitations. Abramson JH.
Survey methods in community medicine. Edinburgh
Churchill Livingstone , 1984.
22
Observational studies in support of experimental
design

• Provide size of potential participant pool
• Provide description of potential study
  participants
• Provide estimates of effect/outcomes in various
  treatments, usual care

23
Why care?
Preliminary Studies. For new applications, use
this section to provide an account of the
principal investigator/program directors
preliminary studies pertinent to the application
information that will also help to establish the
experience and competence of the investigator to
pursue the proposed project. Peer review
committees generally view preliminary data as an
essential part of a research grant application.
Preliminary data often aid the reviewers in
assessing the likelihood of the success of the
proposed project. U.S. Department of Health and
Human Services Public Health Service Grant
Application (PHS 398)
24
Identifying participant pool
A search of our institutional tumor registry
indicates that 1340 breast cancers were diagnosed
between 1990 and 1999. We anticipate 180
additional cases will have been identified in
2000. Thus, we anticipate a total of 1520 women
diagnosed with breast cancer between 1990 and
2000. Based on our prior work, we anticipate that
70 of women will be postmenopausal, 40 of these
will be using HRT at the time of diagnosis, and
40 of these will be using both E P Harvey
JA, Santen RJ, Petroni GR, Bovbjerg VE, Williams
MB, 2000
25
Description of study participants
60.4 are women, 20.9 are non-Caucasian,
38.2 have baccalaureate or advanced degrees, and
the mean age is 53.3. years Halfwere diagnosed
with type 2 diabetes prior to 1995. Mean values
for BMI (36.9), weight (235.4 pounds/107 kilos),
waist circumference (46.3 inches115.8 cm), HbA1c
(7.72), total cholesterol (181.2 mg/dL) LDL
(104.2 mg/dL), and triglyceride (179.2 mg/dL)
suggestsomewhat increased risk of poor health
outcomes due to overweight, glycemic control, and
elevated lipids an average of 2 diabetes
medications at baseline, and substantial
proportions of participants were taking
medication for lipid lowering (46) and blood
pressure control (76). Bovbjerg VE, Wolf
AM, and the ICAN investigators, 2002
26
Estimation of participation
There are approximately 163,000 health plan
members in the four study regions 20,000 in
Roanoke, 15,000 in Harrisonburg/Valley, 65,000 in
Charlottesville, and 63,000 (50,000 commercial
and 13,000 Medicaid) in Richmond Based on the
ICAN pilot study, we expect that 489 members will
contact the study, be eligible, and wish to
participate among all four sites, a total of 390
will be enrolled we expect approximately 60 of
participants to be female, and approximately 30
African-American, with a mean age of
approximately 50 years. Approximately 8 will be
covered by Medicaid. Bovbjerg VE, Wolf AM, and
the ICAN investigators, 2002
27
Estimating effect size
The risk of chronic renal failure in our study
was higher among recipients of liver transplants
who were treated with cyclosporin than among
those who were treated with tacrolimus RR1.25
The results of studies comparing
cyclosporin-based immunosuppressive regimens
withtacrolimus-based regimens have been
contradictory. Most comparative evaluationsare
of limited validity Ojo et al. Chronic Renal
Failure after Transplantation of a Nonrenal
Organ. N Engl J Med 2003 349 931-40. Should
you propose a clinical trial to answer the
question?
28
Estimating sample size
Given ?0.05, power0.80 From observational
study incidence of renal failure in
tacrolimus0.16, relative risk of renal failure
in cyclosporin1.25 From study design random
assignment to equal sized groups Sample size
needed c 1500 in each group (Conversely--if one
could enroll only 200 patients, a relative risk
of over 2.5 would be needed to rule out chance.)
29
With an initial enrollment of 195 subjects per
group, and allowing for a 10 annual dropout
rate, the two-sample t-test has 80 power, with a
two-sided significance level of 5, to detect
differences between the treatment groups, when
the true mean difference is equal to 0.62.
sufficient power to detect slightly larger,
clinically significant group differences in HbA1c
(1.0 absolute difference) when stratified by
gender or race, both of which are important
potential effect modifiers. The use of the
two-sample t-test to compare the mean change from
the month 12 HbA1c level at a single point in
time (42 months) is a conservative estimate
does not represent the complexity of the analyses
These analyses will use repeated measures
models and all the HbA1c values collected at
months 12, 18, 24, 30, 36 and 42 to compare the
mean HbA1c profiles between the treatment groups
over time. Bovbjerg VE, Wolf AM, Conaway MR,
and the ICAN investigators, 2002
30
Observational studies for

• questions inherently not testable by RCT
  (feasibility, ethics)
• getting answers faster than RCTs
• identifying risk factors for disease, outcomes
• providing support for RCT planning
• identifying outcomes in usual practice
• investigating the potential for harm

31
real creativity in medicine lies in the hands
of the world community of innovative
investigators who together carry out hundreds of
thousands of small research studies confirmatory
large randomised controlled trials become the
task of burnt-out leaders who are only fit enough
for administration and organization. Lancet
editorial, May 12, 1990