WHAT CAN WE LEARN FROM DESIGN FAULTS IN THE

1
WHAT CAN WE LEARN FROM DESIGN FAULTS IN
THE WOMEN'S HEALTH INITIATIVE RANDOMIZED CLINICAL
TRIAL?

• F. NAFTOLIN MD, PHD
• DEPARTMENT OF OBSTETRICS AND GYNECOLOGY
• NYU SCHOOL OF MEDICINE

2
SUMMARY

• WHAT IS THE WHI?
• CRITICAL DESIGN FAULTS OF THE WHI
• ACCEPTING TO STUDY A DIFFERENT POPULATION THAN IN
  THE OBSERVATIONAL TRIALS IN ORDER TO AVOID
  DROPOUTS AND TO HAVE SUFFICIENT POWER
• ACCEPTING BIOLOGICAL IMPLAUSIBILITY
• STUDYING OUTCOMES RATHER THAN PROGRESS OF DISEASE
• APPARENTLY MISSING PLACEBO DATA IN YEAR FIVE
• ASSESSING QUALITY OF LIFE IN WRONG SUBJECTS
• SUBSET ANALYSIS OF WOMEN CHOSEN FROM THE LARGER
  GROUP OF RECRUITS BECAUSE OF A SINGLE FACTOR
  (AGE)
• STUDY OF DEMENTIA WITHOUT PRIOR NEUROLOGICAL
  EXAMINATION
• INTERPRETATIVE ERRORS BIOLOGICAL PLAUSIBILITY
• DID THE WHI MEET ITS OBJECTIVES NO
• WHAT HAS THIS COST?
• REDOING THE WHI KRONOS EARLY ESTROGEN
  PREVENTION STUDY (KEEPS)

3
WHAT IS THE WHI?
THE WHI IS A STUDY TO DETERMINE WHETHER ESTROGEN
(E) OR E PROGESTIN (P) REPLACEMENT IS
CARDIOPROTECTIVE
4
Design of the Womens Health Initiative Clinical
Trial and Observational Study Controlled
Clin Trials 19981961109 by The Womens Health
Initiative Study Group ABSTRACT The Womens
Health Initiative (WHI) is a large and complex
clinical investigation of strategies for the
prevention and control of some of the most common
causes of morbidity and mortality among
postmenopausal women, including cancer,
cardiovascular disease, and osteoporotic
fractures. The WHI was initiated in 1992, with a
planned completion date of 2007. Postmenopausal
women ranging in age from 50 to 79 are enrolled
at one of 40 WHI clinical centers nationwide into
either a clinical trial (CT) that will include
about 64,500 women or an observational study (OS)
that will include about 100,000 women. The CT is
designed to allow randomized controlled
evaluation of three distinct interventions a
low-fat eating pattern, hypothesized to prevent
breast cancer and colorectal cancer and,
secondarily, coronary heart disease hormone
replacement therapy, hypothesized to reduce the
risk of coronary heart disease and other
cardiovascular diseases and, secondarily, to
reduce the risk of hip and other fractures, with
increased breast cancer risk as a possible
adverse outcome and calcium and vitamin D
supplementation, hypothesized to prevent hip
fractures and, secondarily, other fractures and
colorectal cancer. Overall benefit-versus-risk
assessment is a central focus in each of the
three CT components. Women are screened for
participation in one or both of the
componentsdietary modification (DM) or hormone
replacement therapy (HRT)of the CT, which will
randomize 48,000 and 27,500 women, respectively.
Women who prove to be ineligible for, or who are
unwilling to enroll in, these CT components are
invited to enroll in the OS. At their 1-year
anniversary of randomization, CT women are
invited to be further randomized into the calcium
and vitamin D (CaD) trial component, which is
projected to include 45,000 women. The average
follow-up for women in either CT or OS is
approximately 9 years. Concerted efforts are made
to enroll women of racial and ethnic minority
groups, with a target of 20 of overall
enrollment in both the CT and OS.
5
Statistical Issues Arising in the Womens Health
Initiative Ross L. Prentice, Mary Pettinger, and
Garnet L. Anderson Biometrics 61, 899941
December 2005 Summary. A brief overview of the
design of the Womens Health Initiative (WHI)
clinical trial and observational study is
provided along with a summary of results from the
postmenopausal hormone therapy clinical trial
components. Since its inception in 1992, the WHI
has encountered a number of statistical issues
where further methodology developments are
needed. These include measurement error modeling
and analysis procedures for dietary and physical
activity assessment clinical trial monitoring
methods when treatments may affect multiple
clinical outcomes, either beneficially or
adversely study design and analysis procedures
for high-dimensional genomic and proteomic data
and failure time data analysis procedures when
treatment group hazard ratios are time dependent.
This final topic seems important in resolving the
discrepancy between WHI clinical trial and
observational study results on postmenopausal
hormone therapy and cardiovascular disease.
6
WHAT INSPIRED THE WHI?
PRE CLINICAL STUDIES SHOW CARDIOPROTECTION IN
LABORATORY AND ANIMAL STUDIES OBSERVATIONAL
TRIALS SHOW CLEAR CARDIOPROTECTION IN WOMEN USING
MENOPAUSAL HORMONE THERAPY
7
OBSERVATIONAL TRIALS ON HRT AND CVD
From Stampfer and Colditz, PREV MED, 1991
8
WHAT ARE THE DIFFERENCES BETWEEN THE WHI AND THE
STUDIES THAT INSPIRED IT?
CHRONOLOGIC AGE OF SUBJECTS MENOPAUSAL AGE (YEARS
SINCE LAST MENSTRUAL PERIOD) PHYSICAL CONDITION
OF SUBJECTS
9
THE ILL-EFFECT OF STUDY DESIGN ON SUBJECT AGE IN
THE WHI WHI ACCEPTED TO STUDY A DIFFERENT
POPULATION THAN IN THE OBSERVATIONAL TRIALS IN
ORDER TO AVOID DROPOUTS AND TO HAVE SUFFICIENT
POWER
10
WHI AVOIDED SYMPTOMS (VASO-MOTOR EPISODES) THAT
WOULD BETRAY THE PLACEBO/INCREASE DROP-OUTSWHI
STUDIED DISEASE EVENTS RATHER THAN PROGRESS OF
DISEASE
11
OBSERVATIONAL STUDIES SUBJECTS SELF-SELECT BY
SYMPTOMS
MENOPAUSAL TRANSITION
HRT
NO HRT
12
OBSERVATIONAL STUDIES SUBJECTS SELF-SELECT BY
SYMPTOMS
(WHI) RCT TRIAL ASSIGNS TREATMENT IRRESPECTIVE
OF SYMPTOMS
AVERAGE 12 YEARS POST-MENOPAUSAL WOMEN
MENOPAUSAL TRANSITION
HT
NO HT
HRT
NO HRT
13
WHI ACCEPTED TO IGNORE BIOLOGICAL PLAUSABILITY
(AND PUBLISHED DATA) REGARDING AGE AND HEART
DISEASE
14
ATHEROSCLEROTIC PLAQUE BURDEN MEASURED BY
CORONARY CALCIUM IN ASYMPTOMATIC WOMEN UNDERGOING
ELECTRON BEAM TOMOGRAPHY.
80
Percentiles of
Coronary Calcium
60
90th
Percent of Group
40
75th
25th
20
0
45-49
50-54
55-59
60-64
65-70
Age
Observational
WHI
Modified from Raggi, et al. Circulation
200010850-855
15
Early CVD Events in HERS
Incidence ()
Follow-up (years)
Modified from Herrington DM, et al. Circulation.
20021052962-7.
16

AGE-SPECIFIC OCCURRENCE OF VENOUS THROMBOSIS IN
THE EP ARM OF THE WOMENS HEALTH INITIATIVE
                                                
                                                  
                        
Cushman M, Kuller LH, Prentice R, et al. JAMA.
2004292(13)1573-1580
17
WHI BASELINE CHARACTERISTICS
Values are means (SD) Overall incidence of
prior cardiovascular disease 7.7 P .04 vs
HRT.
Writing Group for the Womens Health Initiative
Investigators. JAMA. 2002288321-333.
18
WHI EVENTS VTESummary by Year
Year HRTn () Placebon () Hazard Ratio
1 2 3 4 5 6 49 (0.58) 26 (0.31) 21 (0.25) 27 (0.34) 16 (0.27) 12 (0.23) 13 (0.16) 11 (0.14) 12 (0.15) 14 (0.19) 6 (0.11) 11 (0.26) 3.60 2.26 1.67 1.84 2.49 0.90
n number of patients () annualized
calculated from average exposure over ?60
months. z score for trend across all years
2.45 test for trend based on Cox proportional
hazard model with time-dependent treatment
effects. VTE includes deep vein thrombosis (DVT)
and PE.
Writing Group for the Womens Health Initiative
Investigators. JAMA. 2002288321-333.
19
WHI EVENTS CHDSUMMARY BY YEAR
Year HRTn () Placebon () Hazard Ratio
1 2 3 4 5 6 43 (0.51) 36 (0.43) 20 (0.24) 25 (0.32) 23 (0.39) 17 (0.33) 23 (0.29) 30 (0.38) 18 (0.23) 24 (0.32) 9 (0.16) 18 (0.42) 1.78 1.15 1.06 0.99 2.38 0.78
n number of patients () annualized
calculated from average exposure over ?60
months. z score for trend across all years
1.19 test for trend based on Cox proportional
hazard model with time-dependent treatment
effects.
Writing Group for the Womens Health Initiative
Investigators. JAMA. 2002288321-333.
20
BECAUSE IT DEVIATED FROM THE STUDIES THAT
INSPIRED THE WHI THE WHI RCT IS 10-FOLD
UNDERPOWERED TO TEST THE CARDIOPROTECTIVE
EFFECTS OF HRT IN WOMEN IN THE MENOPAUSAL
TRANSITION (AGE 49-55)
21
SOURCE OF THE INFORMATION ON THE 50-54 Y.O.
MODERATE-SEVERELY SYMPTOMATIC SUBJECTS IN THE EP
AND PLACEBO GROUPS OF THE WHI
22
CHARACTERISTICS OF THE SUBJECTS MAKING UP THE
50-59 Y.O. WHI ESTROGENPROGESTIN AND PLACEBO
GROUPS
EP Placebo Age 50-59 ( of total
group) 2839 (33.4) 2868 (33.1) 50-79 MENOPAUSAL
AGES lt5 1315 (17.1) 1224 (16.3) 5-
lt10 1467 (19.1) 1488 (19.8) 10- lt15 1611
(21.0) 1566 (20.9) /gt15 3286 (42.8) 3231
(43.0) ? 12.0 (Information from
Hays et al., NEJM, 3481839-54, 2003)
23
TOTAL (EP PLUS PLACEBO) MODERATE-SEVERELY
SYMPTOMATIC 50-54 YR SUBJECTS VS. TOTAL 50-59 YR
SUBJECTS
(2761/GROUP)
(287/GROUP)
(Information from Hays et al., NEJM,
3481839-54, 2003)
(Information from Hays et al., NEJM,
3481839-54, 2003)
24
POWER ANALYSIS FOR 50-54 Y.O. WHI SUBJECTS

• THE WHI 50-54 YO MODERATE-SEVERELY SYMPTOMATIC
  GROUP HAD 287 SUBJECTS PER GROUP.
• THE AGE-CORRECTED NUMBER OF EXPECTED EVENTS
  (NURSES HEALTH STUDY) IS 0.73 EVENTS PER 275
  WOMEN PER 5 YEARS
• IF THE ONE GROUP HAD 2 X 0.73 EVENTS DURING THE 5
  YEAR OBSERVATION PERIOD AND THE OTHER GROUP HAD 0
  EVENTS, IT WOULD REQUIRE gt 4000 WOMEN IN EACH ARM
  TO SHOW A STATISTICALLY SIGNIFICANT DIFFERENCE
  BETWEEN GROUPS.
• WITH A 42 DROPOUT RATE (WHI), THE NUMBER OF
  SUBJECTS NEEDED PER GROUP BECOMES 9000.

Naftolin F, Taylor H, Karas R, Fertil Steril.
81(6)1498-501. 2004
25
MISSING PLACEBO DATA IN YEAR FIVE RAISED THE
HAZARD RATIO AND TRIGGERED ACTION BY THE DRUG
SAFETY MONITORING BOARD
26
WHI EVENTS VTESummary by Year
Year HRTn () Placebon () Hazard Ratio
1 2 3 4 5 6 49 (0.58) 26 (0.31) 21 (0.25) 27 (0.34) 16 (0.27) 12 (0.23) 13 (0.16) 11 (0.14) 12 (0.15) 14 (0.19) 6 (0.11) 11 (0.26) 3.60 2.26 1.67 1.84 2.49 0.90
n number of patients () annualized
calculated from average exposure over ?60
months. z score for trend across all years
2.45 test for trend based on Cox proportional
hazard model with time-dependent treatment
effects. VTE includes deep vein thrombosis (DVT)
and PE.
Writing Group for the Womens Health Initiative
Investigators. JAMA. 2002288321-333.
27
WHI EVENTS CHDSUMMARY BY YEAR
Year HRTn () Placebon () Hazard Ratio
1 2 3 4 5 6 43 (0.51) 36 (0.43) 20 (0.24) 25 (0.32) 23 (0.39) 17 (0.33) 23 (0.29) 30 (0.38) 18 (0.23) 24 (0.32) 9 (0.16) 18 (0.42) 1.78 1.15 1.06 0.99 2.38 0.78
n number of patients () annualized
calculated from average exposure over ?60
months. z score for trend across all years
1.19 test for trend based on Cox proportional
hazard model with time-dependent treatment
effects.
Writing Group for the Womens Health Initiative
Investigators. JAMA. 2002288321-333.
28
WHI EVENTS INVASIVE BREAST CANCERSUMMARY BY YEAR
Year HRTn Placebon HazardRatio
1 2 3 4 5 6 11 (0.0013) 26 (0.0031) 28 (0.0034) 40 (0.0050) 34 (0.0057) 27 (0.0053) 17 (0.0021) 30 (0.0038) 23 (0.0029) 22 (0.0029) 12 (0.0022) 20 (0.0047) 0.62 0.83 1.16 1.73 2.64 1.12
n number of patients () annualized
calculated from average exposure over ?60
months. z score for trend across all years
2.56 test for trend based on Cox proportional
hazard model with time-dependent treatment
effects.
Writing Group for the Womens Health Initiative
Investigators. JAMA. 2002288321-333.
29
ASSESSING QUALITY OF LIFE EFFECTS IN GROUPS WITH
ONLY 11 SYMPTOMATIC SUBJECTS RESULTED IN A STUDY
THAT WAS VASTLY UNDERPOWERED TO ASSESS CHANGES IN
NUMBER OF SYMPTOMS PER DAY
30
MODERATE-SEVERELY SYMPTOMATIC 50-54 YR SUBJECTS
VS. TOTAL 50-59 YR SUBJECTS (EP PLUS PLACEBO)
(2761/GROUP)
(287/GROUP)
(Information from Hays et al., NEJM,
3481839-54, 2003)
31
STUDYING A SELECTED SUBSET GROUP FROM THE LARGER
RECRUITMENT NO INITIAL NEUROLOGICAL
EXAMINATION NO DIRECT DIAGNOSIS OF THE TYPE(S) OF
DEMENTIA LATER SHOWN TO HAVE A HIGH RISK OF
VENOUS THROMBOEMBOLISM THAT MAY LEAD TO STROKE
32
Estrogen Plus Progestin and the Incidence of
Dementia and Mild Cognitive Impairment in
Postmenopausal Women The Womens Health
Initiative Memory Study A Randomized Clinical
Trial Sally A. Shumaker, PhD, Claudine Legault,
PhD, Stephen R. Rapp, PhD, et al. JAMA.
20032892651-2662 Design, Setting, and
Participants The Womens Health Initiative Memory
Study (WHIMS), a randomized, double-blind,
placebo-controlled clinical trial, began
enrolling participants from the Womens Health
Initiative (WHI) estrogen plus progestin trial in
May 1996. Of the 4894 eligible participants of
the WHI study, 4532 (92.6) postmenopausal women
free of probable dementia, aged 65 years or
older, and recruited from 39 of 40 WHI clinical
centers were enrolled in the WHIMS. Intervention
Participants received either 1 daily tablet of
0.625 mg of conjugated equine estrogen plus 2.5
mg of medroxyprogesterone acetate (n2229), or a
matching placebo (n2303). Results The mean (SD)
time between the date of randomization into WHI
and the last Modified Mini-Mental State
Examination (3MSE) for all WHIMS participants
was 4.05 (1.19) years. Overall, 61 women were
diagnosed with probable dementia, 40 (66) in the
estrogen plus progestin group compared with 21
(34) in the placebo group. The hazard ratio (HR)
for probable dementia was 2.05 (95 confidence
interval CI, 1.21-3.48 45 vs. 22 per 10000
person-years P.01). This increased risk would
result in an additional 23 cases of dementia per
10000 women per year. Alzheimer disease was the
most common classification of dementia in both
study groups. Treatment effects on mild cognitive
impairment did not differ between groups
(HR, 1.07 95 CI, 0.74-1.55 63 vs 59 cases per
10000 person-years P.72).
33

• WHI ESTROGEN-ONLY ARM
• CONCORDANT WITH EP ARM EXCEPT NON-SIGNIFICANT
  FALL IN NEW BREAST CANCER DIAGNOSES AND
  CARDIOPROTECTIVE EFFECTS (EVENTS)
• INCREASED STROKES (EVENTS)
• INCREASED THROMBOEMBOLIC EVENTS
• PROTECTIVE EFFECTS ON FRACTURES AND COLON CANCER
  (EVENTS)

34
HAS THE WHI RESOLVED THE ISSUE OF
CARDIOPROTECTION OR NEUROPROTECTION BY E OR E P
IN WOMEN STARTING TREATMENT DURING THE MENOPAUSAL
TRANSITION? NO
35
CAN/SHOULD A RCT BE USED TO TEST THE
CARDIOPROTECTIVE OR NEUROPROTECTIVE EFFECTS OF
HRT?
YES (QUALIFIED) BUT, THE STUDY MUST START HRT
DURING THE MENOPAUSAL TRANSITION AND BE
ADEQUATELY POWERED TO ANSWER THE QUESTIONS IT
ASKS. IN THE CASE OF THE MENOPAUSAL TRANSITION
THERE ARE NOT ENOUGH EVENTS TO USE EVENTS AS AN
ENDPOINT.
36
SO, WHERE ARE WE NOW?
37

The WHI spent one billion dollars to prove that
starting an asymptomatic 63.3-year-old
postmenopausal woman on HRT will not decrease her
risk of having a heart attack

What about the rest of us?
Modified from the New Yorker by Erroll Norwitz,
2003
38
HRT WHERE ARE WE NOW?
IN THE NURSES HEALTH STUDY THE CARDIOPROTECTIVE
EFFECT DISAPPEARS AT THREE YEARS AFTER STOPPING
HRT
HERSH A, STEFANICK M, STAFFORD R JAMA 2004 291
47-53
39
KRONOS EARLY ESTROGEN PROTECTION STUDY (KEEPS)
A FIVE-YEAR 9 CENTER RCT OF WOMEN IN THE
MENOPAUSAL TRANSITION (STARTED 2004) WOMEN
HEALTHY 40-55 YEAR-OLDS WITHIN 12-36 MONTHS OF
LAST PERIOD THREE TREATMENT ARMS DAILY CEE PLUS
CYCLIC TRANS-VAGINAL PROGESTERONE 10 DAYS PER
MONTH DAILY ESTRADIOL BY SKIN PATCH PLUS CYCLIC
VAGINAL PROGESTERONE 10 DAYS PER MONTH
PLACEBO MAIN EVALUATIONS ANNUAL INTIMA-MEDIA
THICKNESS BY CAROTID ARTERY ULTRASOUND YEAR 0, 3,
5 CORONARY ARTERY CALCIUM BY TOMOGRAPHY PLUS
SAFETY STUDIES, ANCILLARY STUDIES
40
SUMMARY

• WHAT IS THE WHI?
• CRITICAL DESIGN FAULTS OF THE WHI
• ACCEPTING TO STUDY A DIFFERENT POPULATION THAN IN
  THE OBSERVATIONAL TRIALS IN ORDER TO AVOID
  DROPOUTS AND TO HAVE SUFFICIENT POWER
• ACCEPTING BIOLOGICAL IMPLAUSIBILITY
• STUDYING OUTCOMES RATHER THAN PROGRESS OF DISEASE
• APPARENTLY MISSING PLACEBO DATA IN YEAR FIVE
• ASSESSING QUALITY OF LIFE IN WRONG SUBJECTS
• SUBSET ANALYSIS OF WOMEN CHOSEN FROM THE LARGER
  GROUP OF RECRUITS BECAUSE OF A SINGLE FACTOR
  (AGE)
• STUDY OF DEMENTIA WITHOUT PRIOR NEUROLOGICAL
  EXAMINATION
• INTERPRETATIVE ERRORS BIOLOGICAL PLAUSIBILITY
• DID THE WHI MEET ITS OBJECTIVES NO
• WHAT HAS THIS COST?
• REDOING THE WHI KRONOS EARLY ESTROGEN
  PREVENTION STUDY (KEEPS)

41
SUPPORTFN HAS RECEIVED SUPPORT FROM NIH, PHARMA
AND OTHER COMMERCIAL INTERESTS IN THE PAST FIVE
YEARS
42
SUMMARY

• WHAT IS THE WHI?
• CRITICAL DESIGN FAULTS OF THE WHI
• ACCEPTING TO STUDY A DIFFERENT POPULATION THAN IN
  THE OBSERVATIONAL TRIALS IN ORDER TO AVOID
  DROPOUTS AND TO HAVE SUFFICIENT POWER
• ACCEPTING BIOLOGICAL IMPLAUSIBILITY
• STUDYING OUTCOMES RATHER THAN PROGRESS OF DISEASE
• APPARENTLY MISSING PLACEBO DATA IN YEAR FIVE
• ASSESSING QUALITY OF LIFE IN WRONG SUBJECTS
• SUBSET ANALYSIS OF WOMEN CHOSEN FROM THE LARGER
  GROUP OF RECRUITS BECAUSE OF A SINGLE FACTOR
  (AGE)
• STUDY OF DEMENTIA WITHOUT PRIOR NEUROLOGICAL
  EXAMINATION
• INTERPRETATIVE ERRORS BIOLOGICAL PLAUSIBILITY
• DID THE WHI MEET ITS OBJECTIVES NO
• WHAT HAS THIS COST?
• REDOING THE WHI KRONOS EARLY ESTROGEN
  PREVENTION STUDY (KEEPS)