Title: Andy Grieve, Executive Director
1Statistics and Its Place in the Evolution of
Medicine as an Evidence-Based Science
- Andy Grieve, Executive Director
- Statistical Research Consulting Centre
- Pfizer Global RD, Sandwich
2Outline
- Statistical Evidence is Everywhere
- What is meant by Evidence-Based
- Quantification in 18th Century British Medicine
- The Méthode Numérique of Louis (19th Century
France) - Radickes Method (19th Century Germany)
- Karl Pearson and Typhoid Fever (Early 20th
Century Britain) - Comparisons to Modern Evidence-based Medicine
3Statistics are Everywhere
- .. statistics .. is a mechanism for trying to
understand and explain what is going on in
society using measurement. It can be used in
every social aspect of society, every aspect of
the physical sciences, in any area of study. It
is the means of elucidating, rather than
obfuscating, what is really going on around us. - Charles Clarke, Secretary of State for Education,
on opening RSS 2004, Manchester, 7 September
4Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern.
5Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern. - Charities use statistics to relieve us of our
money.
6Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern. - Charities use statistics to relieve us of our
money. - Politicians use statistics to persuade us that
they understand society's ills and that they
deserve our support to solve them.
7Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern. - Charities use statistics to relieve us of our
money. - Politicians use statistics to persuade us that
they understand society's ills and that they
deserve our support to solve them. - Newspapers, television and radio use statistics
to spice up their reports to make them more
dramatic, more convincing, more compelling.
8Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern. - Charities use statistics to relieve us of our
money. - Politicians use statistics to persuade us that
thy understand society's ills and that they
deserve our support to solve them. - Newspapers, television and radio use statistics
to spice up their reports to make them more
dramatic, more convincing, more compelling.
- Companies use statistics to advertise their
products.
2 Out of 3 Dentists Surveyed Prefer
Crest Ivory Soap is 99.9 Pure Pall Mall
gives you More Puffs than the other Major Brands
(Based on Avg. Puffs per Cigarette)
9Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern. - Charities use statistics to relieve us of our
money. - Politicians use statistics to persuade us that
thy understand society's ills and that they
deserve our support to solve them. - Newspapers, television and radio use statistics
to spice up their reports to make them more
dramatic, more convincing, more compelling.
- Companies use statistics to advertise their
products. - Scientists use statistics to document their
findings and support their conclusions.
10Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern. - Charities use statistics to relieve us of our
money. - Politicians use statistics to persuade us that
thy understand society's ills and that they
deserve our support to solve them. - Newspapers, television and radio use statistics
to spice up their reports to make them more
dramatic, more convincing, more compelling.
- Companies use statistics to advertise their
products. - Scientists use statistics to document their
findings and support their conclusions. - Statistics are used in all forms of sport
11Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern. - Charities use statistics to relieve us of our
money. - Politicians use statistics to persuade us that
thy understand society's ills and that they
deserve our support to solve them. - Newspapers, television and radio use statistics
to spice up their reports to make them more
dramatic, more convincing, more compelling.
- Companies use statistics to advertise their
products. - Scientists use statistics to document their
findings and support their conclusions. - Statistics are used in all forms of sport
- Those with whom we agree use statistics to
reassure us that we're on the right side
12Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern. - Charities use statistics to relieve us of our
money. - Politicians use statistics to persuade us that
thy understand society's ills and that they
deserve our support to solve them. - Newspapers, television and radio use statistics
to spice up their reports to make them more
dramatic, more convincing, more compelling.
- Companies use statistics to advertise their
products. - Scientists use statistics to document their
findings and support their conclusions. - Statistics are used in all forms of sport
- Those with whom we agree use statistics to
reassure us that we're on the right side, - While our opponents use statistics to try and
convince us that we are wrong. -
13Statistics are Everywhere.
- Social reformers use them to convince us that
social problems are serious and deserve our
attention and concern. - Charities use statistics to relieve us of our
money. - Politicians use statistics to persuade us that
thy understand society's ills and that they
deserve our support to solve them. - Newspapers, television and radio use statistics
to spice up their reports to make them more
dramatic, more convincing, more compelling.
- Companies use statistics to advertise their
products. - Scientists use statistics to document their
findings and support their conclusions. - Statistics are used in all forms of sport
- Those with whom we agree use statistics to
reassure us that we're on the right side, - While our opponents use statistics to try and
convince us that we are wrong. - Statistics are one of the standard types of
evidence used by people in our society. -
14Evidence-Based Society
- Professor Adrian Smith Royal Statistical
Society Presidential Address 1996 - Mad Cows and
Ecstasy Chance and Choice in an Evidence-Based
Society - , in which informed quantitative reasoning is
the dominant modality in public debate, as well
as in the decision-making processes of
government, business and individuals - As well as "evidence based medicine ..
management", I sometimes enter discussions of
" evidence based" chemistry, metallurgy,
cosmology, geology, sociology ... and more
15Evidence-Based Medicine
- 1992 Journal of the American Medical Association
published a manifesto - A new paradigm for medical practice is
emerging. Evidence-based medicine de-emphasises
intuition, unsystematic clinical experience and
patho-physiological rationale as sufficient
grounds for clinical decision-making and stresses
the examination of the evidence from clinical
research
163 Classic Cultures to Determine Medical Evidence
- systemic-pathophysiological
- modern laboratory science, the Hippocratic,
galenic systems as well as more modern movements
such as homeopathy (Hahnemann) and anthroposphy
(Steiner)
Dominant since antiquity
- individual cases / medical judgement
- Collection of empirical evidence
- Hippocrates 1st aphorism experiment and
experience treacherous - A doctor did not test, he or she knew
- Perform all these duties calmly and adroitly,
concealing most things from the patientGive
necessary orders with cheerfulness and serenity,
turning his attention away from what is being
done to him sometimes reprove sharply and
emphatically.revealing nothing of the patient's
future of present condition
- The introduction of the empirical approach in
medicine was largely an 18th century British
initiative its most famous proponent James
Lind
17James Lind and the treatment of scurvy
18James Linds Experiment (1747)
- Scurvy killed more sailors than military action
- Experiment performed on navy ship
- On the 20th May 1747 Lind took 12 patients as
similar as I could have them. They all in general
had putrid gums, the spots and lassitude, with
weakness of their knees. - 2 each to one of 6 treatments all of which were
justified by dogma or ordinary experience - Standard diet
- water gruel sweetened with sugar in the morning
fresh mutton broth often times for dinner at
other times puddings, boiled biscuit with sugar
etc. and for supper barley, raisins, rice and
currants, sago and wine, or the like. - A diet almost guaranteed to cause scurvy !!!
19James Linds Experiment (1747)
- Two a quart of cyder a day
- Two otherstwenty five gutts of elixir vitriol
three times a day upon an empty stomach, using a
gargle strongly acidulated with it for their
mouths - Two others.two spoonfuls of vinegar three times
a day upon an empty stomach, having their gruels
and their other food well acidulated with it, as
also the gargle for the mouth - Two of the worst patients, with the tendons in
the ham rigid (a symptom none the rest had) were
put under a course of sea water. Of this they
drank half a pint every day and sometimes more or
less as it operated by way of gentle physic - Two others had each two oranges and one lemon
given them every day. These they eat with
greediness at different times upon an empty
stomach. They continued but six days under this
course, having consumed the quantity that could
be spared - The two remaining .bigness of a nutmeg three
times a day of an electuray recommended by an
hospital surgeon made of garlic, mustard seed,
rad. raphan., balsam of Peru and gum myrrh, using
for common drink barley water well acidulated
with tamarinds, by a decoction of which, with the
addition of cremor tartar, they were gently
purged three or four times during the course
20James Linds Experiment (1747)
- The consequence was that the most sudden and
visible good effects were perceived from the use
of the oranges and lemons one of those who had
taken them being at the end of six days fit four
duty. The spots were not indeed at that time
quite off his body, nor his gums sound but
without any other medicine than a gargarism or
elixir of vitriol he became quite healthy before
we came into Plymouth, which was on the 16th
June. The other was the best recovered of any in
his condition, and being now deemed pretty well
was appointed nurse to the rest of the sick - The next best treatment was cider as had been
shown previously in a large comparative trial. - The other treatments had no positive effect !
21James Linds Experiment (1747)
- Lind believed in empiricism because
- He understood the role of imagination and
suggestion in healing - An important lesson in physic is here to be
learnt, viz. The wonderful and powerful influence
of the passions of the mind upon the state and
disorders of the body. This is often overlooked
in the cure of diseases - Postulated that all observations from a series of
unselected cases rather than the habit of
publishing individual successful cases only
should yield the evidence for therapeutic
recommendations - He was identifying selection bias, observer
bias, publication bias
2218th Century Britain
- Lind was not alone
- He was one of a relatively large number of
medical graduates (particularly from Edinburgh
University) - They were characterised by
- being outside the medical establishment
- using quantitative measurement to challenge dogma
based on theory - non-conformists, Unitarians
- Formed medical societies which tended to be
regional - One influential, regional, group formed in
Warrington as an epicentre of Manchester,
Liverpool and Chester
23The Warrington Group
Conducted controlled experiments in the treatment
of rheumatism
Argued for recording of serial data successes
failures to avoid false conclusions if he
trusted to memory alone
Observations on Amputations (1779) operated in
25 cases, such as promiscuously occurred at the
Liverpool Infirmary, without the loss of a single
patient
Thomas Percival 1764
John Haygarth 1765
Corresponded with Richard Price about using bills
of mortality as a data bank useful for clinical
research.
Investigation of cold water bathing for fever. He
collected body temperatures to measure
effectiveness
Quantitative evaluation of his practice of
cleanliness and ventilation in Manchester as
prophylaxis against puerperal sepsis
2418th Century Britain
- The methods they used were termed medical
arithmetic unsophisticated - In therapeutics the arithmetic observationists
tests would involve mathematically the formation
of sums, the calculation of averages (I.e.
arithmetic means), and at highest, that of ratios
(e.g.. Success-to-failure ratios (Tröhler ,
1978)
25Monograph Published by Royal College of
Physicians of Edinburgh Based on his 1978 UCL PhD
26Basis of Argumentation in Medical Periodicals
(1733-1829)
0.7
0.6
0.5
0.4
Percent of Articles
0.3
Journal
0.2
0.1
0
One case
Two cases
gt Two cases
Other Articles
Type of Article
Source Ulrich Tröhler PhD Thesis, UCL, 1978
27Other examples in Tröhler of Medical Arithmetic
- The removal of bladder stones without killing
patients - Adopting a folk remedy for Dropsy
- The treatment of rheumatism
- Improving survival after amputation
- Treating Syphilis with mercury
- Treating fever with Peruvian bark (quinine)
28Bladder Stones Lithotomy
- Bladder stones much more common in 18th century
- Diet, urinary infections, elderly men hypertrophy
of the prostate - Celsus in the 1st century
- Stone pulled down per rectum into the perineum.
Once localised under the skin, in the previously
dilated part of the urethra a cut was made
directly to the stone. - The early lithotomists were itinerant, largely
self-trained - Pain extremely severe worth the risk of an
operation - High mortality
- Successful lithotomists used simple statistics
to maintain their reputations and stay in
business. - Friar Jacques, in Paris, William Cheselden in
London - Developed methods for cutting directly into the
bladder - Speed, convenience, greater success rate
29Bladder Stones Lithotomy
- William Cheselden Anatomy (1740)
- Keen to show the truthfulness of his data
- I cannot take the liberty to mention the names
of private patients, therefore I will give a
detail of those only which I cut in the hospital,
where the first 25 recovered, to the truth of
everyone of which I had above 20 witnesses, and I
do believe these patients are all living at this
time - Comparison (tabulation) of mortality rates by age
- Investigations of the influence of gender, weight
of stone - Investigations of time to death and recovery
30Digitalis and Dropsy
- William Withering Account of the Foxglove
(1785) - He was recommended the foxglove to treat dropsy
(oedema) by an old countrywoman - He undertook a prospective study lasting 15
years before he was prepared to publish 163
patients (156 from private practise) - It would have been an easy task to have given
select cases, whose successful treatment would
have spoken strongly in favour of the medicine,
and perhaps been flattering to my own reputation.
But Truth and Science would condemn the
procedure, I have therefore mentioned every case
. Proper or improper, successful or otherwise - Quality of data Withering only used his own
data because it might be misleading if the other
publications did not give all the relevant
details, failures as well as successes.
31The Treatment of Rheumatism
- John Haygarth The Imagination as a Cause and as
a Cure of Disorders of the Body (1800) - Trial of Perkins Metallic Tractors
- These were metallic rods supposed to cure
rheumatism by electrical/magnetic influence
- It was fashionable Institute of Perkinism in
London - Haygarths Experiment in two parts
- Wooden imitation tractors used (without their
knowledge) on 5 patients 1/5 were relieved. - The next day genuine tractors resulted in 1/5
patients being relieved - Single-blind, placebo controlled ??
32The Statistics of Amputation
- Edward Alanson Practical Observations on
Amputations (1779) - Alanson developed an improved amputation
technique since using his new method he had
operated on thirty-five cases, .without the
loss of a single patient - Prior to the this new method Alanson had been
present at 46 amputations of whom 10 patients had
died - Historic control data ?
- He had never refused to operate upon any case
that has presented, where a single person in
consultation has thought such an operation
advisable - Selection bias ?
33France Beginning of the 19th century
- 3 different visions of the physician
- The artist
- Professor Risueno dAmador promoted the vision
of the physician as an artist,relying on medical
tact and his intuition about the individual
patient - The statistician
- Pierre-Charles-Alexandre Louis had little
respect for tact and wanted to see the evidence - The determinist
- Claude Bernard a physiologist reject both the
artist and statistician views. Science was
certainty.
A great surgeon performs operations for kidney
stone by a single method later he makes a
statistical summary of deaths and recoveries, and
he concluded from these statistics that the
mortality law for this operation is two out of
five. Well, I say that this ratio means literally
nothing scientifically and gives us no certainty
in performing the next operation
34PCA Louis and the Méthode Numérique
35PCA Louis and the Méthode Numérique
- Medical doctor graduated 1813
- Most famous work (1823) on blood-letting and
pneumonia - Introduced the idea of the Méthode Numérique
for deciding whether treatments were effective - Méthode Numérique comparison of means
36Data from Louis (1835) - Effect of Blood letting
in surviving patients with pneumonia Duration
of disease (d days), number of bleeds (n),
Related to Time of First Bleed (days)
37 Data from Louis (1835) - Effect of Blood letting
in surviving patients with pneumonia Duration
of disease (d days), number of bleeds (n),
Related to Time of First Bleed (days)
50
40
Duration of Disease (Days)
30
20
10
0
1
2
3
4
5
6
7
8
9
Time of First Bleed (Days)
38Data from Louis (1835) - Effect of Blood letting
in surviving patients with pneumonia Duration
of disease (d days), number of bleeds (n),
Related to Time of First Bleed (days)
- Professor Peter Armitage Royal Statistical
Society Presidential Address 1983 Trials and
Errors The Emergence of Clinical Statistics - Analysed the data
-
- 1. Overall no difference between the 9 groups
- 2. Contrast 1-4 versus 5-9 not significant
(p0.038) - 3. Contrast 1-2 versus 3-9 significant
(p0.006) - Such modern analyses depend on understanding the
variability of the data
39Gustav Radicke and Understanding
Variation(Henrik Støvring, The Statistician ,
1999)
40Radickes Proposals (1858)
- Considered different types of means
- arithmetic, geometric, harmonic and quadratic
mean - Chose the arithmetic mean
- as the probable value of a variable quantity
determined under mean conditions - Relevance to nearly all medical applications
- Even under stable conditions the biological
responses of an individual vary - Measurement error, biological variation
- Effect of salt on the excretion of urine
41Radickes Proposals (1858)
- Remember Louis only looked at means
- Radicke pointed out the need to consider
variability - Three methods
- Suppose we have values xij (i1, , n j1, ..
, 2) - Absolute variation
- a Maximum xij - mj m1 - m2 gt 2a
- Fluctuation of Successive Means
- a Maximum difference between cumulative
means(mean of the first i observations) and mj
then follow 1. - Mean Error (Quadratic Mean)
- Standard deviation within each group sj m1 -
m2 gt 2(s1 s2 )
42Data from Kaupp (1855)Total Urinary Excretion -
Volume
- Kaupp experimented on himself for 92 days
- observed a rigorous diet (same menu measured to
the gram) - constant life-style (the life of an ordinary
student) - Seven different doses of salt for 12 days (15
versus 12) - Measured daily temperature
- Missing data on 5 days measurement error and on
one occasion he consumed 8 glasses of
non-dietary beer !!
43Data from Kaupp (1855)Total Urinary Excretion
Volume (cc)
33.6 g
28.7 g
19.0 g
14.2 g
9.3 g
1.5 g
23.9 g
Dose Of Salt
3000
2800
2600
2400
Volume
2200
2000
1800
1600
0
20
40
60
80
100
Time
Missing data simulated by Støvring from a Normal
density with m and s2 estimated from the
complete data
44Data from Kaupp (1855)Total Urinary Excretion
Urea (gm)
33.6 g
28.7 g
19.0 g
14.2 g
9.3 g
1.5 g
23.9 g
Dose Of Salt
Missing data simulated by Støvring from a Normal
density with m and s2 estimated from the
complete data
45Data from Kaupp (1855)Total Urinary Excretion
Volume ACF
46Data from Kaupp (1855)Total Urinary Excretion
Vol/Temperature
- Accounting for temperature
- Modern approach is to use analysis of covariance
- Kaupps proposals were attempts to estimate
linear relationship by grouping
47Data from Kaupp (1855)Total Urinary Excretion
Vol/Temperature
- Accounting for temperature
- Modern approach is to use analysis of covariance
- Kaupps proposals were attempts to estimate
linear relationship by grouping (linearity
questioned by Radicke)
- Adjusting urinary excretion data for different
temperatures before applying previous procedure - Conclusion no evidence of influence of salt
and/or temperature - Not supported by modern analysis
48Karl Pearson Almoth Wrightand typhoid
inoculation
49Almoth Wright Typhoid Inoculation
- 1892 made Professor of Pathology at the
Royal Army Medical College in Netley - Developed prophylactic immunization against
typhoid - Tested it on himself, friends and students
- 1904 published A Short Treatise on Anti-Typhoid
Inoculation - cogency of statistical evidence
- existence of a control group
- exact recording of all data
- Generally more advisable for the physician to
accept the general conclusions implied by the
statistics rather than to worry excessively about
the theoretical underpinnings
50Karl Pearson Typhoid Inoculation
- Pearson approached by War Office
- Could he establish whether Wrights
- results were significant ?
- Pearson suggested a prospective clinical trial
- Not taken up
- But he did analyse the available data
- Pearson published his work in the BMJ in Nov. 1904
Karl Pearson
51Inoculation Against Typhoid Karl Pearson (BMJ,
1904)
Indian Army 1901
Indian Army 1900
7th Hussars
City Imperial Volunteers
5th Battalion Manchester Regiment
Methuens Column
Garrison Ladysmith
Army Regiments / Hospitals
Staff 2nd Sec. Scot. Red Cross Hosp
Staff No. 9 General Hosp
Staff No. 8 General Hosp
Staff Imp. Yeo Hosp Pretoria (2)
Staff Imp. Yeo Hosp Pretoria (1)
Staff Imp. Yeo Hosp Deelfontein
Staff Portland Hosp.
0.001
0.01
0.1
1
10
100
Odds Ratios and 95 Confidence Interval (log
scale)
52Pearson BMJ, 1904
- Estimated correlation between immunity and
inoculation (tetrachoric correlation fitting a
bivariate normal density to a 22 table) - Theory published in Phil. Trans. Roy. Soc. 1900
- Bayesian Integrated over parameter space
- Approximate asymptotic s.e.
- He thought of the data that would convince him of
the efficacy of the vaccine. - Before working out the data, the impression
formed in my mind was that if the correlations
turned out to be fairly consistent and greater
than 0.4, we might consider the case for
inoculation proven if irregular and less than
0.2 unproven that is, it must be a very small
value, possibly more than balanced by concomitant
ills while, if lying between 0.2 and 0.4,
further cautious investigation by experimental
inoculation seemed desirable - He based numerical values on the known
effective smallpox vaccine. - Strong evidence 0.2 corresponds to an OR of
0.5 (2) and 0.4 to an OR of 0.2 (5)
53Pearsons Enteric Fever DataBy Grouping Log
Odds ratios Pearsons approach
?
?
?
Indian Army
?
?
?
Hospital Staffs
?
Arithmetic Mean
0.01
0.1
10
1
Odds Ratios and 95 Confidence Interval (log
scale)
54BMJ 1904
- He concluded the results although statistically
significant, they were irregular and the
lowness of the values suggested environmental
factors were influential. - Advocated suspension of the operation as a
routine method and further study
55Pearsons Enteric Fever DataBy Grouping Log
Odds ratios Modern approach
?
?
?
Indian Army
?
?
?
Single Regiments
?
?
?
Methuens Column
Ladysmith Garrison
?
?
?
?
?
?
Random Effect Model
Hospital Staffs
?
?
?
Fixed Effect Model
?
?
?
0.1
10
0.01
1
Odds Ratios and 95 Confidence Interval (log
scale)
56Wright versus Pearson
- Editors of BMJ agreed with Pearson vs. Wright
- They regretted that a newspaper of such
influence as the Times had come out in favour of
Wrights position in an attempt to - force on the War Office and on the army a
method which, on the testimony of its author,
requires the labour of years to perfect, and the
use of which involves possibilities of serious
mischief - Wright
- Pearsons mathematical principles were unerring
and beyond my intellectual ken - The advisory board were hiding behind Professor
Pearsons petticoats in their attempt to keep
anti-typhoid inoculation from being introduced. - Pearson
- There was a crying need for a more exact
treatment of statistics in medical science
57Modern Evidence-Based Medicine
58Evidence-Based Medicine
- Evidence-based medicine is the conscientious,
explicit, and judicious use of current best
evidence in making decisions about the care of
individual patients. The practice of
evidence-based means integrating individual
clinical experience with the best available
external clinical evidence from systematic
research . Without current best evidence,
practice risks becoming rapidly out of date, to
the detriment of patients - Sackett et al (1996), British Medical Journal
59Evidence-Based Medicine
- Requires
- Identification of Evidence
- Establishment of the Quality of the evidence
- Individual cases
- Non-controlled studies
- case-control studies (retrospective)
- randomised controlled studies
- double-blind, randomised controlled studies
- Synthesis of the Evidence
- Meta-Analysis
60Book Review Nature 392, 671 - 672 (16 April 1998)
- A new kind of alchemy is abroad in the world.
Arcane, esoteric and mesmerizing, it promises not
to turn base metal into gold, nor to provide an
elixir of youth, but to transmute statistical
sows' ears into scientific silk purses.
Meta-analysis, the systematic synthesis of
otherwise inconclusive research findings into
hard results, has, in the space of barely two
decades, come from a standing start to challenge
the blind clinical trial as the principal arbiter
of truth and the touchstone of knowledge across a
range of biomedical and social sciences
61Parallels Between 18th /19th Centuries
20th/Early 21st Centuries Tröhler (2001)
- Requires
- Intense discussions about how to assemble
reliable evidence about the effects of health
care (league tables ?) - An emphasis on going beyond the treatment of the
individual (clinical guidelines ?) - Commercialism (economic analysis)
- Allocation of scarce resources (cost of citrus
fruit) - Considerable medical innovation and a demand for
comparison to standard treatment - Empirical testing of medical interventions
62Conclusion
- Critical statistical evaluation of medical
interventions - medical arithmetic 18th century Britain
- Méthode Numérique 19th century France
- Evidence-based Medicine late 20th century
- Introduction depends on the appropriate climate
- Public understanding depends on increased
education