Getting the most out of a biostatistics consultation

1
Getting the most out of a biostatistics
consultation

• John Pearson
• University of Otago, Christchurch

2
Themes

• Who we are
• What we do
• How we work

3
Full time consultants
Assoc Prof Elisabeth Wells Department of Public
Health and General PracticeEmail
elisabeth.wells_at_otago.ac.nz Research
Interests Psychiatric epidemiology Survey
methods
Dr John Pearson Department of Pathology Email
john.pearson_at_otago.ac.nz Research
Interests Microarrays Genetics
Bioinformatics Survey Methods Statistical
Computing
4
How to find us

• http//www.chmeds.ac.nz/departments/pubhealth/bios
  tat.htm
• Stephen Sharp (admin, ground floor) can make
  bookings
• stephen.sharp_at_otago.ac.nz
• Tel  [phone]
• Supervisors should attend the first meeting

5
Two-way communication issues in consulting

• Client
• Be patient with a biostatistician who doesnt
  know your research area and is struggling to
  understand enough to advise well
• Biostatistician
• Be respectful and try to communicate with
  non-statisticians, taking account of the amount
  of statistics they know and trying to translate,
  if required

6
Biostatistical consulting

• UOC biostatistics webpage
• http//www.uoc.otago.ac.nz/departments/pubhealth/b
  iostat.htm
• says
• Biostatistical consultancy involves advice on
  study design, methodology, computer software,
  data analysis and preparation, and revision of
  articles and reports. The biostatisticians can
  also act as collaborators during research
  analysis or on long term studies, and provide
  training in statistical methods.

7
Biostatistics webpage (ctd)

• There is no charge for consultation services
  provided by biostatistical consultancy staff for
  students and members of staff of the School and
  the Canterbury District Health Board. However
  biostatistical work should be costed as part of
  research grant applications.
• WE ARE FREE BUT IN SHORT SUPPLY

8
Process

• Think first (preferably think lots)
• Read the literature
• Consult a biostatistician, well in advance of an
  ethics or grant application

In this sequence each step may result in your
going back to an earlier step and revising
9
Reasons for revising

• Examples
• Literature may show your project has been done or
  that you need to measure other variables
• The biostatisticians questions may require you
  to re-read articles or search again
• There may be ways of analysing the data which you
  had not thought of so you may revise your aims
  and objectives

10
Looping back and revising

• Looping back and revising is part of the usual
  process of setting up a research project
• The more you can think through things in advance
  the better
• Nonetheless revision is often required as you
  think further, and in more detail, and is nothing
  to be ashamed of

11
What to bring to a biostatistician

• Aims/objectives/hypotheses
• Suggested study design
• Expected size of effect based on the
  literature/clinical experience orthe
  desired precision of some estimate
• Any key papers
• Draft data collection instrument (if available)

12
Study design

• Observational studies- cross-sectional
  studies- cohort studies- case-control studies
• Intervention/experimental studies- for human
  trials usually only one or two
  interventions are used- for laboratory studies
  there will usually be a dose variable with
  several levels and often other experimental
  conditions as well

13
Study design

• Is a case-control study unmatched or matched
  (individually or frequency matched)
• In a treatment trial are there parallel groups or
  a cross-over design
• In follow-up of patients is it better to study
  more patients or fewer more often?

Within the main study designs there will be
options
14
What sample size do I need

• This is sometimes asked with no context
• That is like asking a travel agent What does it
  cost to travel?
• The answer depends on what you want to do.

15
Effect size or single estimate

• Estimating the size of an effect (eg the
  difference between Tx A and Tx B) and testing the
  hypothesis that there is an effect
• Estimating a single quantity (eg the proportion
  of the population living below the poverty line)
  no hypothesis here

When planning a study you need to decide if you
are
16
Effect sizes expected

• Use the literature as a guide to the size of
  effect you might find (similar studies, similar
  treatment for different problem, size of effect
  that would change clinical practice or policy)
• Consider differences in proportions, odds ratios,
  or differences in means (try to find out the
  standard deviations if possible)

17
Effect size and sample size

• For comparisons, expected effect sizes have to be
  thought through before a statistician can come up
  with a sample size
• Various scenarios can be looked at before
  deciding one is best
• If sample size is fixed, then it is possible to
  see what size of effect could be detected

18
Power and sample size

• Ethics applications mention the power of the
  study. This is the probability of obtaining a
  statistically significant result in your study if
  the world is as you suppose it (ie there is an
  effect of the size you propose).
• Power is often set at 80 (odds of 41) but
  sometimes at 90 or 95 for definitive studies.

19
Power (ctd)

• If the power is less than 80 then there is not
  much point in doing the study
• Biostatisticians dont like multiple small
  studies which are too small to detect the likely
  effects of different treatments

20
Hypothesis testing vs confidence intervals

• In the mid 1980s biostatisticians tried to move
  health research away from significance testing
  (results reported as significant or not) to
  confidence intervals around an estimate
• The estimate shows the most likely effect and the
  confidence interval shows the interval within
  which the true value is likely to fall

21
Precision of single estimates

• Sometimes there is no size of effect but just
  the precision of a single estimate.
• This is often how sample sizes for national
  surveys are set but it also can apply to small
  surveys
• You may want to know if only a small, moderate or
  high proportion of patients had a 12 month
  follow-up outpatient visit
• However if the survey result is 40 you want to
  know the precision (margin of error) 10-70
  is a different estimate from 35-45

22
Planning for precision
Planning for precision is based on the size of
the confidence interval expected

• If 4 out of 20 children have asthma then the
  prevalence (P) is 20, with a 95 confidence
  interval (CI) of (8, 42)
• For 20/100, P20, 95CI13, 29
• For 40/200, P20, 95CI15, 26

Agresti, A., and Coull, B. Approximate is better
than 'exact' for interval estimation of binomial
proportions. The American Statistician 52
119-126, 1998
23
Planning for precision
Planning for precision is based on the size of
the confidence interval expected
24
Planning for precision
Precision depends on P so it is necessary to look
at plausible values for P

• N100 P50 95CI40, 60
• N100 P5 95CI2, 11

What precision do you want? What precision can
you afford?
25
Diminishing returns

• Precision and power depend on the square root of
  N, not N
• To 1/2 a confidence interval you need to 4x the
  sample size, (not 2x it)
• Costs increase linearly with sample size
• There are diminishing returns from larger and
  larger samples

26
Data collection
Biostatisticians can advise on data collection

• This should be planned well in advance of
  starting collection
• A database or EXCEL spreadsheet should be set up
• Procedures for checking data are required

27
Questionnaires

• Most biostatisticians have experience in
  questionnaire design
• Biostatisticians will think about data entry
  issues, coding and analysis of the data, not just
  the questions themselves
• Questionnaires may look easy but there are many
  traps

28
Computing

• Almost no research is done without computers now
• Biostatisticians can advise on what software
  packages to use (but we use only some of them so
  if you choose another then the support will be
  limited.)

29
Analysis
Whether you carry out your own analyses or
whether a biostatistician does them depends on

• How much you know
• Whether or not you are a student (students have
  to learn to do their own analyses)
• How complicated the analyses are

30
Writing up the results

• Biostatisticians can help with this
• For papers or reports biostatisticians often
  write up the statistical methods and may write
  some or much of the results
• Students have to write their own results but will
  receive guidance from their supervisor(s) and may
  check with a biostatistician

31
Responding to referee criticism
The point of referees is to find weaknesses in
studies (as well as to praise good points).
Depending on the issues raised biostatisticians
may be involved in

• Rebutting criticism or making design changes
  suggested by grant reviewers
• Rebutting criticism or carrying out additional
  analyses from journal or report reviewers

32
Summary

• Biostatisticians can be involved with all stages
  of a project from design to publication
• Sometimes they are required only for technical
  advice at some part (but fixing problems which
  occurred because advice was not sought earlier is
  not liked)
• Sometimes they are part of the team

33
Take home messages

• Check statistics early
• Statisticians are mere mortals