User Involvement in Mental Health Research

1
User Involvement in Mental Health Research

• Diana Rose and Jenny Walke
• Service User Research Enterprise (SURE)
• Institute of Psychiatry
• Kings College London

2
Purpose

• To demonstrate how service user-focused research
  can add something to the methods and findings of
  conventional research
• To show why service user researchers are
  best-placed to do this
• Involving service users at all stages in the
  research process

3
Overview

• A little bit about one of us (DR)
• Some introductory slides
• Service user research
• User-focussed monitoring
• Systematic reviews
• Outcome measures
• Criticisms and answers
• The NIHR continuum

4
Having a double identity (DR)

• Service user all my adult life
• 1972-1986 academic career
• 1986 medically retired living in the
  community
• 1996 two identities come together to do user-led
  research

5
Colney Hatch Lunatic Asylum (est 1854)
6
Friern Hospital corridor 1976 I walked the line
7
Friern Hospital patient (1984)
8
Camden Mental Health Consortium Resettling Friern
Patients

• Camden central and north London
• Friern patients to be resettled to this area
• Camden Mental Health Consortium one of first
  local user groups
• Formed to make sure the resettlement was done
  well
• Not user-led at this point that came later

9
Service user research - history

• 1988 CMHC and GPMH project on new inpatient
  provisions in District General Hospital
• 1996 two user-led projects based in NGOs
• Strategies for Living (Mental Health Foundation)
• User-Focused Monitoring (Sainsbury Centre for
  Mental Health)
• Political workers members of the user movement
  and took research questions from the movement
• To make a contribution our evidence for the
  movement

10
User research moves on (or backwards)

• Early 2000s two groups in universities
• SURESearch (Birmingham)
• Service User Research Enterprise (SURE)
• Service user researchers have insider
  knowledge
• SURE has collaborative management structure
  one conventional university researcher and one
  user researcher
• SURESearch more complex but head is
  conventional university researcher

11
Activity

• Now test your knowledge - correct answers
  revealed on final slide.
• 1. What was the original name of Friern Hospital?
• Bedlam
• Colney Priory
• Colney Hatch Lunatic Asylum
• 2. How many patients did Friern accommodate by
  the 1950s?
• 500
• 2000
• 5000

12

• 3. What does CMHC stand for?
• Community Mental Health Clinic
• Centralised Mental Hospital Care
• Camden Mental Health Consortium
• 4. When were two reports from service user-led
  projects published?
• 2000
• 2002
• 2005

13
Further Reading

• Taylor B The Last Asylum. London Penguin 2014.
• Survivors History Group, Mental health and
  survivors' movements and context
  http//studymore.org.uk/mpu.htm

14
User-Focussed Monitoring
15
Method of Peer-Review of Mental Health Services

• Started in Central London in 1996
• Community services
• Hospital services
• Expanded across UK and across service types
• Now also used in Nordic countries
• Norway User ask user

16
UFM is User-Led

• Co-ordinator who is a user but also with research
  skills
• Visits local area to identify service users
  interested in project
• Get together over lunch to make final decision
  about whether they want to be involved

17
Some UFM projects questionnaire-based

• Group devises questionnaire on the basis of their
  experience of the service to be assessed
• Takes many meetings as co-ordinator synthesises
  ideas and then brings back to group for amendment

18
Training

• Some involved in UFM projects have never done an
  interview
• Some basic training
• But mostly role-play opportunity to experience
  answering as well as asking the questions

19
More qualitative projects

• Focus groups have been used
• Topic guide devised in the same way as for
  questionnaires but shorter as want to give group
  scope to expand on their views
• Some training for this too

20
Collecting the information

• Interviewers go to many different venues CMHTs,
  hospitals, participants homes, charities
• Focus groups not usually held in NHS venues
  more neutral
• De-briefing co-ordinator speak with interviewer
  immediately after each interview to check no
  problems

21
Analysing the data

• For questionnaires there is some statistical
  analysis
• Problematic from involvement perspective as few
  service users have these skills
• Easier for members of UFM team to be involved in
  analysing qualitative data
• Can bring their experience to bear

22
What happens to the report?

• UFM projects are locally commissioned e.g.
  Trusts, local authorities, charities
• Report goes to commissioners
• Up to them what they do with it
• Some take very seriously and results in
  measurable changes
• Some take not so seriously!
• Sometimes have feedback days for staff

23
Activity

• 5. When did peer review of mental health services
  start in Central London?
• 1983
• 1990
• 1996
•  
• 6. Why arent focus groups typically held in NHS
  venues?
• Lack of space
• Lack of neutrality
• Lack of insurance
•  

24

• 7. At what level are UFM projects commissioned?
• Local
• National
• International

25
Further Reading

• Rose D, Fleischmann P, Schofield P Perceptions
  of User Involvement a User-Led Study.
  International Journal of Social Psychiatry 2010,
  56(4)389-401.

26
New method patient-centred systematic reviews
27
Example - ECT

• Consumers perspectives on ECT
• Two main researchers had experienced ECT
  insider knowledge
• Adapted method of systematic review to make it
  user-focused
• Included peer-reviewed literature as normal in
  systematic reviews
• Included the grey literature
• Included qualitative data testimonies
• Assembled 26 clinical papers and 9 authored by
  service user groups
• Testimonies sourced from a video archive and the
  internet

28
Main themes

• Perceived benefit
• Retrograde memory loss
• Information
• Consent and perceived coercion

29
Perceived benefit

• Conventional research showed much higher levels
  of satisfaction with ECT than user research no
  overlap in the estimates of benefit between the
  two groups
• Critique of method in grey literature
• We answered with critique of method in clinical
  papers

30
ECT continued

• BOTH conventional and grey literature showed high
  levels of memory loss though conventional did not
  discuss this or even said it was not important
• BOTH showed people did not feel informed but
  again this absent from discussion in clinical
  papers
• Perceived coercion testimonies showed some
  people felt coerced into signing form
• Same results but different interpretations not
  like the finding on perceived benefit where there
  were different results

31
The Dispute

• Paper published in a high-profile medical journal
• Royal College of Psychiatrists issued press
  release disputing what we had said about memory
  loss
• Mendacious they mis-quoted themselves

32
Policy

• NICE new guidelines on ECT as our research work
  being done
• Consumer review influenced
• Especially around information and consent
  including about risk of memory loss
• User research can have an effect on national
  policy

33
Activity

• 8. Which two of the following are omitted from
  orthodox systematic reviews?
• Grey literature
• Randomised controlled trials (RCTs)
• Peer-reviewed literature
• Qualitative testimonies 
• 9. Issues of consent and memory loss are
  associated which of these treatments?
• Cognitive Behavioural Therapy (CBT)
• Occupational therapy
• Mindfulness
• Electro-convulsive therapy (ECT) 

34
Further Reading

• Rose D, Fleischmann P, Wykes T Consumers' views
  of electroconvulsive therapy A qualitative
  analysis. Journal of Mental Health 2004,
  13(3)285-293.

35
Patient Generated Patient Reported Outcome
Measures (PG-PROMs)
36
The claim of neutrality in Randomised Controlled
Trials (RCTs)

• RCTs considered the gold standard in medicine
• Neutrality depends on blinding (not knowing
  certain details)
• But is everything in an RCT neutral?
• Outcome measures devised by clinicians and
  academics
• May not be the outcomes that matter to service
  users
• Try to develop measures that are valued by
  service users (and others) in mental health

37
Patient Reported Outcome Measures (PROMs)

• Much talk of PROMs
• But only filled out by patients no say in which
  questions are asked
• Patient-Generated PROMs (PG-PROMs)
• Medication side-effects
• Experiences of inpatient care

38
Method

• Participatory research
• Attempts to reduce the power relations between
  researcher and researched
• In user-focused research, researchers have the
  same or similar experiences as the participants
• All are mental health service users
• A new development even within participatory
  research
• Potential influences on treatment and policy

39
Procedure

• Focus Groups which meet twice
• Recruited because they have experience of what
  the measure is attempting to tap
• Facilitators/researchers have experience of the
  treatment or service that is being evaluated
• On basis of focus group discussions, researchers
  draw up draft measure
• Taken to Expert Panels for amendment and
  refinement and that the language is their own
  language
• Feasibility study to make sure it is easy to
  complete refinements all the way
• Psychometric testing
• About 200 participants involved in all

40
Example in-patient care

• Much anecdotal evidence and evidence in the
  grey literature that profoundly disliked
• My experience too
• Wanted to do something more rigorous
• Finally became the main outcome measure in an RCT
  evaluating the introduction of psychological
  therapies on acute wards
• Collaborative but our part is user-led

41
User Measure VOICE Participants

• People who had been in-patients in the local
  Trust within the previous two years
• One group specifically made of participants who
  had been detained

42
Focus groups

• 6-8 people 4 groups
• One facilitator with experience of in-patient
  care and other also a service user
• Meet twice to make sure we have accurately
  captured their views
• Thematic analysis using Nvivo software after both
  1st and 2nd wave

43
Drafting the measure

• Done by the researchers on the basis of the
  qualitative analysis
• Quantitative and qualitative questions
• Researchers also use own experience

44
Expert Panels

• One drawn from focus group members and one
  independent
• Also been inpatients in previous two years
• Tasked with amending and refining the measure and
  making sure that language and layout are
  appropriate
• Usually quite a few changes made at this stage

45
Feasibility study

• Final stage of constructing measure
• 50 people complete the measure and we find out
  which parts are easy to complete and which not
• Iterative process
• In this project the participants were actually in
  hospital

46
Psychometrics

• Mixed methods research
• We do this because it is appropriate but also to
  show that user-led research can be rigorous
• One mainstream researcher said users could never
  produce measures because they would never
  understand factor analysis

47
Psychometrics Continued

• Test-retest reliability the same people fill in
  the measure twice with an interval of a week in
  between
• Are the scores the same the 2nd time? This would
  mean the measure is stable.
• VOICE very stable
• People in this exercise mostly a diagnosis of
  psychosis which might compromise stability.
• Good test-retest reliability because measure
  developed by service users?

48
Psychometrics cont

• Criterion validity assess measure against an
  existing one
• Expect some differences because of means of
  development close relationship but there were
  differences
• Acceptability how does it feel to fill it in
• Enjoyable?
• Distressing?
• Right length?

49
Activity

• 10. Who compiles most existing outcome measures?
• Patients families and carers
• Clinicians and academics
• Hospital administrators
•  
• 11. What is blinding?
• Conducting research without obtaining consent
• Assigning participants to different groups
• Temporarily withholding information to reduce
  bias

50

• 12. What name is given to techniques for finding
  out whether a measure is stable and valid?
• Feasibility study
• Psychometrics
• Expert panels
• Quantitative methods

51

• 13. What is the main purpose of a feasibility
  study?
• To ensure a measure is easy to complete
• To check cost-effectiveness of a measure
• To advertise your research

52
Further Reading

• Evans J, Rose D, Flach C, Csipke E, Glossop H,
  McCrone P, Craig T, Wykes T VOICE Developing a
  new measure of service users' perceptions of
  inpatient care, using a participatory
  methodology. Journal of Mental Health 2012,
  21(1)57-71.

53
Challenges to user-led research and user-produced
knowledge
54
Frank Scepticism

• Peter Tyrer, past editor of the British Journal
  of Psychiatry, writes
• The engine of user involvement, while welcome
  in principle,.may drive mental health research
  into the sand.

55
Power

• Most of the projects we have been involved with
  are collaborative
• Nearly always headed up by professor(s) of
  psychiatry or psychology
• Not just status or naked power although that
  exists more subtle
• Are you a researcher or are you a patient?
• I wonder what your diagnosis is, then.
• Undermining user-produced knowledge

56
Hierarchies of Evidence

• RCT is the gold standard
• But RCTs are not neutral outcome measures
  devised by clinicians
• Expert opinion counts as evidence but only as
  the weakest form
• Experts are psychiatrists
• Users as experts a different knowledge
  perspective
• The Cochrane Hierarchy needs revisiting

57
The charge of bias 1

• Said, mostly implicitly, that user-research is
  biased, anecdotal and carried out by people who
  are over-involved ENMESH conference
• We make no pretence of neutrality
• But all research comes from a certain standpoint
• Conventional researchers think what they do is
  obvious nothing is obvious
• User researchers more explicit about this than
  mainstream researchers
• In my opinion the word bias should be banished
  from research discourse and all researchers
  should clearly say where they are coming from.

58
Bias 2

• Although implicit some seem to think that
  irrational people (the mad) cannot engage in the
  supremely rational activity of science
• We epitomise unreason (Foucault)
• So more difficult to find legitimacy and
  credibility than consumer researchers in other
  medical disciplines
• Paradox user research in mental health ahead of
  that in other specialties

59
Some answers?
60
A new epistemology for user-led research
learning from feminism

• Timeless oppositions
• Reason / unreason
• Culture / nature
• Intellect / emotion
• First are male attributes and valorised
• So have women been excluded from science and
  science not attentive to the concerns of women

61
Timeless oppositions and madness

• The mad positioned as nature, emotion and
  crucially unreason
• The Enlightenment valorised reason and so
  positioned the mad as its antithesis
• Unable to reason
• Unable to labour
• The Great Confinement

62
Standpoint epistemology and strong objectivity

• Standpoint epistemology has been seen as
  essentialist
• Sandra Harding and strong objectivity
• We have access to two discourses
• Our own
• That of conventional science
• A more complete picture even a challenging one

63
Implications

• User-led research can add to the jig saw or it
  can challenge conventional research
• But needs to critique the methodologies and
  epistemologies of conventional research to do
  this successfully

64
Other ways service users may become involved in
research

• Everybody in SURE employee of IoP
• BRC Service User Advisory Group (SUAG)
• Reference groups for projects e.g. The Interface
  Study
• Sitting on Steering Groups
• Problem of tokenism although in SURE also
• Should be parity of esteem and payment
• Training resources for service users

65
INVOLVE (NIHR) Continuum

• Consultation Collaboration User-control
• Researcher-initiated
  
• Collaboration split Jointly initiated
• User-initiated

66
Activity

• 14. The Cochrane Collaboration is associated with
  which type of evidence?
• Systematic reviews
• Focus groups
• Ethnography
• Service user research
• 15. Which two of the following statements are
  true?
• RCTs are neutral
• RCTs are considered the gold standard of
  evidence
• Service user research is neutral
• User-led research can reduce the power imbalance
  in the research process.

67

• 16. Sandra Harding used which term to describe
  research by groups traditionally excluded from
  knowledge production?
• Enlightenment
• Strong objectivity
• Unreason
• Evidence-based medicine
• 17. Which is NOT a level of user engagement on
  the INVOLVE continuum?
• Consultation
• Collaboration
• Facilitation
• User-control

68
Further Reading

• Sweeney A, Beresford P, Faulkner A, Nettle M,
  Rose D (eds.) This is Survivor Research.
  Ross-on-Wye PCCS Books 2009.

69
Activity Answers
Q. ANSWER(S) Q. ANSWER(S)
1. c 10. b
2. b 11. c
3. c 12. b
4. a 13. a
5. c 14. a
6. b 15. b, d
7. a 16. b
8. a, d 17. c
9. d