Research Design

1
Research Design Experiments Observations
Surveys Prof. Craig Jackson Head of Psychology
Division School of Social Sciences BCU health.bc
u.ac.uk/craigjackson
craig.jackson_at_bcu.ac.uk
2
!
Objectives Experimental studies within-subjects
studies between-subjects studies Observational
studies case-controls cohorts RCTs Bias Place
bo Control Groups
3
Introduction Types of research
Experimental vs. Observational
Longitudinal
vs. Cross-sectional
Prospective vs. Retrospective
4
Qualititative VS Quantitative Research False
opposition Observational methods equally
valid Complementary roles Qualitative equally
as hard to do
Quantitative
Qualitative
5
Experimental Studies Investigator makes
intervention A manipulation Then studies the
effects of that intervention Features Comparis
on e.g. before vs. after control vs.
treatment Always longitudinal
Always prospective Experimental
Clinical Trials RCTs
6
Rationale of Experimental Studies Evaluate
effectiveness of intervention / therapy Use
similar samples comparable groups Samples
reflect population Differences in outcomes due
to interventions (not differences between
groups) Independent Variable (IV) alters
Dependent Variable (DV) Best evidence of cause
and effect Sometimes inconclusive
7
Types of Experimental Studies Between Subjects
Studies Each group receives different
treatment Groups compared Within Subjects
Studies Each individual is measured before
after intervention Advantage that each
participant is own control Between subject
variability removed
8
Within Subjects Studies Cross-over-studies Each
patient receives treatment in sequence Washout
period between treatments Order of treatments
randomised Matched-pairs study Parallel
study Patient in arm 1 matched with patient in
arm 2 Matched based on prognostic factors Data is
linked Paired individuals
Gp A
Gp A
Treatment 2
Treatment 1
Gp B
Treatment 1
Treatment 2
Gp B
9
Avoiding Bias Validity of study depends on
avoiding bias Bias Systematic distortion of
results due to unforeseen factors gp1
pill gp2 no pill How will the no pillgroup
progress? Any effects of them knowing they have
no treatment? Handling differences may influence
complicate trial results Known as confounding
factors To minimize bias control
group randomisation blinding
10

• Placebo effect it really does work!
• Most effective medication known
• In approx. 30 of pop.
• Subjected to more clinical trials than any other
  medicament
• Nearly always does better than anticipated
• The range of susceptible conditions seems
  limitless
• Does not always occur
• Present in subjective and objective outcomes
• Negative outcomes can occur (Nocebo effect)
• Placebo
• Big pills better than smaller pills
• Red pills better than blue
• 4 pills better than 2
• 30 of pop.

Patients knowledge of their treatment causes
biase.g. Benedetti the Turin study
11

• Control Groups
• Allow comparison in Between Group studies
• Evaluations without comparison?
• Patient knowledge of their treatment causes bias
• e.g. Benedetti the Turin study
• Types of Control Groups
• no treatment group likely to be confounded by
  condition
• placebo group ethically dodgy?
• low dose group avoids ethical
  issues
• standard treatment group avoids ethical issues
• gold standard group avoids ethical issues
• historical controls unreliable due to many
  confounders

12
Control Groups Random Allocation Doesnt
guarantee groups will be homogonous Ensures
allocation independent of patient
features Avoids (sub)conscious allocation
bias e.g. severely sick people into treatment
groups Guarantees allocation to be
bias-free Non-homogenous groups may still occur
due to chance random errors
13
Comparison Groups Random Sampling Ensures
generalizability of findings to larger pop. e.g.
in-custody sample limitations Treatment effects
better detected if there is little between-group
variability Exclusion Criteria Inclusion
Criteria keep groups comparable Paradox greater
uniformity of sample less generalizable to
gen. pop
14
Blinding Importance of doing it Investigator
or Subject know treatment Bias Observations
and Judgements become less reliable Patient
responses change Positive outcomes in active
arm Negative outcomes in passive arm e.g. known
cancer diagnoses and deterioration Use max.
degree of blindness possible e.g. make subject
and investigator both blind if possible e.g.
A.A.Mason Congenital Ichthyosis and Hypnosis
1951
15
Blinding Methods of doing it Double-blind patien
t investigator blind
Treatment type
Patient interaction
Data manager
16
Un-blinding a problematic study Breaking code
anticipated in planning Criteria for breaking
code established and agreed Emergency access
to randomisation code Treatment stopped and
patient withdrawn Formal monitoring process
review and make recommendations
17
Blinding Methods Double-blind patient
investigator blind Single-blind patient
blind Triple-blind patient investigator data
monitor blind Double-dummy 2 treatments patien
ts get 2 pills (1 active, 1 dummy) Open
trials patient investigator aware of treatment
Randomisation in a double-blind trial Envelope
technique common Un-blinding ethical necessity
18
Subject Variables that confound research STABLE
FACTORS SITUATIONAL FACTORS Age Alcohol
(recent use Education Caffeine (recent
use) Sex Nicotine (recent use) Socioeconomics
Medication (recent use) Language Paints,
glues, pesticides Handedness Near visual
acuity Computer experience Restricted movement
(injury) Caffeine (habitual use) Cold /
flu Alcohol (habitual use) Stress Nicotine
(habitual use) Arousal / Fatigue Medication
(habitual use) Sleep Paints, glues, pesticides
Screen luminance Diabetes Time of
day Epilepsy Time of year Other CNS / PNS
disease Head injury (out gt1 hr) Alcohol / drug
addiction Physical activity
19
Randomized Controlled Trials in Practice 90
consultations take place in GP surgery RCT is
really 50 years old Potential problems 2 Key
areas Recruitment Bias Randomisation
Bias Over-focus on failings of RCTs
20
RCTs in Practice RCTs justified in situations of
genuine clinical uncertainty Samples large
enough to establish any worthwhile benefit
(effectiveness or cost, or both) Need for
larger numbers of participants More than are
available to single practices Requires club
together approach Practitioners no contractual
obligation (i) unwilling to take part if no
immediate benefit for clients (ii) while possibly
disrupting the delivery of service/care
21
RCTs in Practice Conflict of interest
between Role and Wish to benefit future
offenders Academic merit Long term nature of
practitioner and client relationship may
engender loyalties unfairly coerce clients to
give consent Patients' fears about confidential
ity risks of the intervention apparent
disadvantage of being allocated to a control
group may further inhibit recruitment Fail to
recruit consecutive clients may introduce
potential for selection bias
22
RCTs in Practice Provides rigorous, sound basis
for evaluating treatments May disrupt care Too
much disruption no reflection of real
practice Methodological problems reduce
scientific reliability of the results
(Recruitment Randomisation) Practice not a
laboratory People are not experimental
animals Case-control studies, retrospective,
prospective cohort studies, and descriptive
studies are all acceptable methods. Observation
is OK Should accept alternative methods when RCT
difficult or flawed
23
RCT Deficiencies Trials too small Trials too
short Poor quality Poorly presented Address wrong
question Methodological inadequacies Inadequate
measures of quality of life (changing) Cost-data
poorly presented Ethical neglect Participants
given limited understanding Poor trial
management Politics Marketeering Why still the
dominant model?
24
Observational Studies Investigator observes
existing situation Describes Analyses Interprets
No influence on events Longitudinal
observation studies case-control studies
retrospective cohort-studies prospective
Cross-sectional observation studies surveys
examining subjects at one point in time based on
random sample of interest population
25

• Observational Studies
• Look for associations
• Cause -gt Effect
• Exposure Illness
• Epidemiological
• Incidence
• Cause
• Prevention
• No control
• Cannot use classical experimentation
• No randomisation
• Bias

26
Case-Control Study Identify group with condition
/ offence (cases) Identify group without
condition / offence (controls) Both groups
compared for exposure to (hypothesized) risk
factors Greater exposure to risk factor in cases
than controls causal relation Lead time
bias Recruitment of cases at similar points in
time Newly labelled cases
27
Selection of Controls Cases have Theft
offences Controls could be other young people
patients or normals Matched Cases Controls
for age gender Option of 2 Controls per Case
Smoking years of cases and controls (matched
for age and sex) Cases Controls n456 n45
6 F P Smoking yrs 13.75 6.12 7.5 0.04 (
1.5) ( 2.1)
28
Case-Control Study Other Biases Recall
Bias Cases gt associations with exposures / risk
factors Unreliable Memories Retrospective
nature Over-reliance on recall Unreliable
Records Poor hospital records Repetitive,
incomplete, inaccurate, irretrievable,
interpretation Interview Bias Different
interviewers
29

• Cohort Study
• ID and examination of a group (cohort)
• Followed over time (20 years common!)
• Looking for condition development / other
  end-point
• Aetiology of condition (based on data collected)
• Data more reliable than case-control studies
• Requires large N
• Requires long follow up
• Inefficient
• Expensive (espec. rare outcomes)

30
Cohort Study Methods Volunteers in 2 groups
e.g. exposed vs non-exposed All complete
attitude survey every 12 months End point at 5
years groups compared for Health Status
Comparison of general health between users and
non-users of mobile phones ill healthy mobil
e phone user 292 108 400 non-phone
user 89 313 402 381 421 802
31
Cohort Study Other Biases Lost to follow
up Bias if reason related to exposure Validity
affected Group sizes change Membership
changes e.g ex-smokers Differential
mortality Change in circumstance e.g. job
change Exposures need calculation or
re-calculation Surveillance bias Investigator
aware of group membership Investigating exposed
members more
32
Cross Sectional Study Subjects contacted
surveyed just once Questionnaire (post, email,
phone) Random sample of defined pop. Limited
causality Not temporal relationships Little
insight into aetiology Source of descriptive
data Prevalence rates Volunteer bias Non
responses Self-selection Unrepresentative
sample
33
Further Reading Altman, D.G. Designing
Research. In Altman, D.G., (ed.) Practical
Statistics For Medical Research. London, Chapman
and Hall, 1991 74-106. Bland, M. The design of
experiments. In Bland, M., (ed.) An
introduction to medical statistics. Oxford,
Oxford Medical Publications, 1995 5-25. Daly,
L.E., Bourke, G.J. Epidemiological and clinical
research methods. In Daly L.E., Bourke, G.J.,
(eds.) Interpretation and uses of medical
statistics. Oxford, Blackwell Science Ltd, 2000
143-201. Jackson, C.A. Study Design Sample
Size and Power. In Gao Smith, F. and Smith, J.
(eds.) Key Topics in Clinical Research. Oxford,
BIOS scientific Publications, 2002.
34
Further Reading Jackson, C.A. Planning Health
Safety Research Projects in the Workplace.
Croner Health and Safety at Work Special Report
2002 62 1-16. Kumar, R. Research
Methodology a step by step guide for beginners.
Sage, London 1999. Abbott, P. and Sapsford.
Research methods for nurses and the caring
professions. Open University Press, Buckingham
1988. Bowling, A. Measuring Health. Open
University Press, Milton Keynes 1994 Polit, D.
Hungler, B. Nursing research Principles and
methods (7th ed.). Philadelphia Lippincott,
Williams Wilkins 2003.