Complex interventions, cluster trials, modelling and complexity

1
Complex interventions, cluster trials, modelling
and complexity

• Mike Campbell
• Professor of Medical Statistics
• Medical Statistics Unit
• ScHARR
• University of Sheffield
• UK
• m.j.campbell_at_sheffield.ac.uk

2
Outline of talk

• What are complex interventions?
• Uses of modelling in complex trials
• Complex systems and self organised criticality
• Agent based modelling
• Summary

3
What is a complex intervention?

• British Medical Research Council (2000)1 Complex
  interventions are built up from a number of
  components, which may act both independently and
  inter-dependently.
• Components include
• Behaviours
• Parameters of behaviours (eg frequency, timing)
• Methods of organising and delivering behaviours
• (eg GP/nurse, community/hospital)
• 1 MRC(2000) A framework for development and
  evaluation of RCTs for complex interventions to
  improve health

4
Quotes

• The greater the difficulty in defining precisely
  what, exactly, are the active ingredients of an
  intervention and how they relate to each other,
  the greater the likelihood that you are dealing
  with a complex intervention (MRC 2000)
• This definition is also consistent with a poorly
  thought through intervention (Hawe, 2004)1
• 1 Hawe P, Shiell A, Riley T (2004) Complex
  interventions how out of control can a
  randomised trial be? BMJ, 328 1561-1563

5
Complex intervention examples

• Patient education trials
• Patients have differing needs and levels of
  knowledge, the education needs to adapt to the
  requirements of the patients
• Multi-agency interventions
• Eg a stroke unit, where neurologists,
  gerontologists, physiotherapists and occupational
  therapists all work together
• Government initiatives
• Attempts to reduce waiting times

6
Cluster randomised trials

• Complex interventions nearly always have to be
  evaluated using cluster randomised trials
• Three main reasons
• Necessity have to administer intervention in
  clusters
• Contamination to prevent control subjects
  exposed to intervention
• Synergy of effect patients encourage each other

7
But

• Many cluster trials are negative eg Diabetes
  Care from Diagnosis, Hampshire Depression project
• Reasons
• 1) Dynamics of intervention poorly understood.
• 2) Under powered cluster trials need more
  patients than individually randomised trials
• 3) Over optimistic effect size
• 4) Trials often modifying human behaviour more
  difficult than say modifying a medical condition
• 5) Difficulty in recruiting

8
Drug Trials vs Trials of complex interventions
9
Modelling

• Increasingly, grant giving bodies in UK require
  early phase studies as well as qualitative
  studies in planning complex interventions.
• There is a big role for modelling in planning
  these studies.

10
Example of pre-trial modellingFalls trial1

• Falls are a public health problem 30 aged 65
  falling each year in UK
• Fear of falling is a major social problem
• Possible intervention falls risk assessment and
  falls clinic
• Is it worthwhile running a trial on this
  intervention?
• We constructed a model of intervention and put in
  reasonable parameters
• 1Eldridge et al J Health Serv Res Policy 2005

11
Referral pathway for falls

• Older people at home
• Assessed by social service or GP
• gtRefer to Falls Clinic and/or OT
• services
• Older people in residential homes
• Assessed by Support services or
• residential staff
• gt refer to Falls Clinic
• Older people in hospital
• Assessed by hospital consultant
• gt refer to Falls Clinic

12
Markov Model

• 11 falls related states
• Each state assigned a utility value which is
  assumed to have beta distribution with given
  parameters (obtained from literature)
• Utilities assigned to each (with 25th and 75th
  centile)
• No fear of falling 1.0 (0.90 to 0.99)
• Fear of falling 0.67 (0.35 to 0.99)
• Good fracture and aftercare in community
• 0.31 (0.01 to 0.60)
• Bad fracture and aftercare in nursing home
• 0.05 (0.01 to 0.07)

13
Transitions

• Annual clock with transitions mid-year.
• Death rates with/without fracture obtained from
  Office of National Statistics
• Fracture rates obtained from Public Health Common
  Data set
• Assume effect of treatment reduce risk of falling
  by 36
• (recent review gave range 20 to 66
• Probability of entry to nursing home 0.01
• Examples of transition probabilities
• No fear of fallinggtfear of falling 0.108
  (aged 65-74)
• No fear of fallinggt fracture 0.003 (aged
  65-74)
• Model starts with 1000 people and continues until
  estimates change little because majority dead.

14
Modelling falls trials
15
Net benefit measure

• Let ?C increase in costs from new intervention
• Let ?Eincrease in benefits from new intervention
  (usually measured in Quality adjusted life years
  QALYs)
• Suppose we are willing to spend R per QALY
• Then intervention is approved if
• R?E- ?Cgt0 .

16
Falls trial results

• Modelling showed that intervention would reduce
  falling by 2.8
• At 30,000 per quality adjusted life gained, only
  40 chance of intervention being cost effective
• Main reason for failure few people who actually
  fall are picked up by system
• Can test sensitivity of model to assumptions by
  changing them mainly sensitive to costs
• Decided not to proceed with main trial

17
Cochrane review

• Cochrane review showed effects of screening and
  treatment interventions of about 30
• Possible reason for difference our intervention
  is aimed at the community, not a specific target
  group

18
Modelling summary

• Pre-trial modelling very useful saved an
  expensive failure
• Modelling also used extensively by the British
  National Institute for Health and Clinical
  Excellence (NICE) to evaluate cost effectiveness-
  especially where trial data on costs non-existent

19
Feedback models

• Falls model complicated but not complex
• The simulation is linear one cannot go from
  outcome to intervention
• Complex model involves feedback

Intervention
Subjects
Outcome
20
Examples of feedback

• From patients
• General practitioner adjusts his/her consultation
  style depending on patient
• Educator adjusts mode of delivery depending on
  clients preferred mode of listening
• From outcomes
• Diabetic trainer will modify advice depending on
  HbA1c results

21
Complex adaptive systems

• Characterised by the following key properties1
• Consist of multiple component agents that are
  connected through local agent-agent interactions
• Interactions tend to be non-linear and feedback
  on one another
• The system can adapt to changes in internal and
  external environment
• Difficult to remove a part of system and replace
  it.
• System capable of learning
• 1 Complexity for Clinicians Holt T (ed)
  Radcliffe Publishing (2004)

22
Self-organised criticality

• Example sand box
• Build a pile of sand, grain by grain
• As pile grows, addition of further grain triggers
  a local motion whereby sand rearranges itself.
• As pile grows, large avalanches occur
• At this point, pile grows no further.
• Amount of sand added equals amount that slides off

23
Self organised criticality (2)

• Difficult to define. One attempt a statistical
  steady state that is produced by processes of
  infinite separation.. of time scales.. and
  exhibits scale invariance
• Used in geology to describe earthquake
  distributions, in evolution to describe
  punctuated equilibrium

24
Power law

• Let s be number of grains that topple in an
  avalanche
• Let N(s) be number of avalanches of size s over
  a long period of time
• Then can show that
• This is known as the power law and is a feature
  of self organised criticality
• Describes number of earthquakes of a particular
  intensity well.

25
General practitioner consultations as self
organised critical systems

• Must operate on edge of chaos
• If too rigid, such as sticking to guidelines, may
  miss important clues
• For example, during a consultation a patient
  casually mentions unexplained weight loss.
• Consultation changes direction dramatically
  (small input, big change)

26
General practice as a complex system1

• 142050 episodes of back pain with at least one GP
  consultation in New Zealand
• Plot log (frequency of episodes) against
  log(number of consultations per episode) i.e
  log(P) vs log(n)
• 1 Love T and Burton C (2005) General practice as
  a complex system a novel analysis of
  consultation data. Family Practice

27
Illustration of Power Law (data from Love and
Burton)
28
Conclusions by authors

• Family practice behaves as a complex adaptive
  system
• Suggests frequent attenders are not a discrete
  group
• The system comprising patients and their health
  care providers influences its own consultation
  rates
• Interventions should be aimed at whole systems
  and not targeted at individuals
• Since complex gtUnpredictable response to stimuli

29
Features of self organised criticality

• Non-linear responses
• Large inputs may result in little change
• Small inputs may result in large change

30
But

• Statisticians might suggest a truncated negative
  Binomial model
• Can develop probabilistic models and get a
  reasonable fit, which dont require any
  assumptions about criticality.

31
Is general practice a complex system?

• A power law suggests a complex system
• However, having a power law does not require that
  general practice is a complex system.
• Could find probability distribution to mimic
  observed frequencies

32
Critique of self-organised criticality (SOC)1

• SOC not a theory but a series of models connected
  by formal analogy
• SOC models too simple in real life situations
• SOC models useful as sketches capturing main
  features of a phenomenon
• SOCs useful as a new way of thinking about
  certain processes
• Useful for simulation
• 1 Frigg R(2003) Self-organised criticality what
  it is and what it isnt. Stud Hist Phil Sci 34,
  613-632

33
Agent-based modelling (ABM)

• ABM is based an a set of autonomous decision
  making entities called agents
• Each agent assesses situation and makes decision
• Agent based models can exhibit complex behaviour
  patterns
• ABM looks at behaviour of individuals
• ABM easy to program

34
Agent based modelling and simulation

• ABM allows non- linear feedback
• Agent based modelling can display complex
  behaviour and self-organised criticality
• ABM predicts emergent phenomenon
• Eg a traffic jam may move in opposite direction
  to the cars that cause it

35
Previous uses of agent based modelling

• Flows
• Traffic flows in a city
• How to regulate traffic lights
• Emergency evacuation of buildings
• Design of emergency exits
• Customer behaviour in theme parks
• Distribution of rides throughout park
• When to extend opening hours

36
Possible uses of agent based modelling in medicine

• Simulation of complex interventions
• Eg in patient education, people react differently
  to received materials. Some will seek more
  information from, say, the Internet. Others may
  react negatively.
• Simulation of organisations
• Eg a stroke unit

37
Agent based models in pilot studies

• I suggest that agent based models would be useful
  to simulate complex interventions.
• They could be used to explore how to design the
  intervention to achieve maximum effect.

38
Discussion

• Proponents of complexity often antagonistic to
  statistics (also to evidence based medicine and
  guidelines)
• They argue that statistics concentrates on the
  marginal values, ignoring interactions at
  individual level.
• Similarly, they argue that evidence based
  medicine tries to make each individual conform,
  ignoring local circumstances.
• I have yet to see good examples of agent based
  modelling in medicine.

39
Work to be done

• Use agent based models to model complex
  intervention
• Problem find simple rules.

40
Conclusion

• Advances in computing have thrown up many
  competitors to statistical analysis, eg neural
  networks, genetic algorithms.
• Complexity, self organised criticality and agent
  based modelling are new areas to be explored.
• May emerge as of peripheral interest to
  statisticians but an area to be aware of.