National User Experience Surveys

1
National User Experience Surveys SSRG Seminar, 16
November 2009
2
Background

• NHS IC currently run the Social Care User
  Experience surveys on behalf of interested
  stakeholders which include
• Councils, DH and CQC
• Survey Programme
• 2002/03 Older People Receiving Home Care
• 2003/04 Physically Disabled and Sensory
  Impaired Users 18 to 64
• 2004/05 Children
• 2005/06 Older People Receiving Home Care
  First IC run survey
• 2006/07 Physically Disabled and Sensory
  Impaired Users 18 to 64 (Cancelled due to poor
  response rate in pilot)
• 2007/08 Equipment and Minor Adaptations
• 2008/09 Older People Receiving Home Care
• 2009/10 Equipment and Minor Adaptations
  Voluntary Carers Survey
• 2010/11 onwards Putting People First

3
Life of A Survey
DH
Survey Objective
Question Areas
Cognitive Testing
Guidance for Pilot
CQC
Ethical Review
Design Data Return
Questionnaire for Pilot
Councils
Volunteer Councils
Send out Qnaires
Chase non-responders
Draw Sampling Frame
Code up Qnaires
Validate and raise queries
Analyse
Return to IC
Revise Qnaire
Develop Guidance
Ethical Review
Design Data Return
Draw Sampling Frame
Chase non-responders
Code up Qnaires
Send out Qnaires
Return to IC
Validate and raise queries
Analyse
Indicators
Report
Data Available For Benchmarking
Data Queries
Return to the Start
4

• Sample of results from previous surveys
• Older People Receiving Home Care
• Equipment and Minor Adaptations

5
Q1 Comparison of the 2002-03, 2005-06 and
2008-09 percentages for Overall, how satisfied
are you with the help from Social Services that
you receive in your own home?
58 of respondents were extremely or very
satisfied with their home care in 2008-09 Down 1
pt on 2005-06 and same as 2002-03
Source UES 2002-03 (87,000 respondents),
2005-06 (82,000) provisional 2008-09 (77,000)
6
Q2 Do your care workers come at times that suit
you?¹
33 of respondents said their care workers always
come at times that suit them down on previous
years14 of respondents said their care workers
sometimes come at times that suit them up on
previous years
Source UES 2002-03 (87,000 respondents),
2005-06 (82,000) provisional 2008-09 (77,000)
7
Do the care workers come at times that suit the
client (Q2) by the degree of satisfaction (Q1)
Those that were extremely satisfied were more
likely to report that their care workers come at
times that suited them
Come at times that suited
(24,500)
(17,100)
(27,400)
(4,400)
(2,700)

• Less than 0.5 per cent
• 1. Includes quite dissatisfied, very dissatisfied
  and extremely dissatisfied.

Source UES 2008-09 provisional
8
95 of respondents said the equipment had made
their quality of life better
-
Q10 How has the equipment/minor adaptation
affected the quality of your life?
-
4
27
68
Made it much better
Made it a little better
Had no effect
Made it a little worse
Made it a lot worse
Source UES 2007-08
- Less than 0.5 per cent 1. Based on 67,000
respondents.
9
Those who had a choice in what they received were
more likely to report that their quality of life
had improved
Source UES 2007-08
10
Those who have stopped using their equipment were
less likely to have been shown how to use it
Source UES 2007-08
1. Users could choose more than one response for
why they have stopped using their equipment 2.
Excludes the answer No, but I did not need to be
shown.
11
Sample of work carried out by PSSRU on National
UES

• Concepts, item development testing
• Focus groups exploratory interviews
• Literature searches
• Cognitive testing
• Validity reliability
• Extension studies with volunteer councils
• Wider range of data
• In-depth analyses
• Enhancing value
• Analyses of extension studies data
• Conducting linking with related research

12
Studies to date

• Home care of older people
• National extension to 2003 UES
• Provider study 2004/05
• National extension to 2006 UES
• Equipment
• Developing concepts
• Developing testing items
• Extension to 2008 UES
• Younger adults
• Developing testing items
• Younger adults UES 2007
• Carers
• Developing concepts
• Developing testing items

13
Research questions

• Quality and outcomes
• How do we best reflect these?
• Explaining variations i.e. what factors affect
  quality and outcome?
• Performance Indicators
• Strengths weaknesses
• Do they reflect what we want them to?
• Are they reliable indicators of changes in
  performance?
• What is the impact of non-response?
• Alternative approaches/ solutions

14
Does satisfaction PI measure quality?
15
Sensitivity single-item v. composite item
16
Within LA variation between providers
17
How Cumbria use results from National UES (1)

• Example illustrates dissemination and sharing of
  results
• Analysis is done in full for Integrated
  Management Team (Senior Managers)
• Recommendations around specific issues are
  highlighted e.g. around Safeguarding question
  (added by us)

18
How Cumbria use results from National UES (2)

• Summary of analysis sent to all Adult Social Care
  staff electronically from the Directors Office
• Details appear in In the Know our fortnightly
  Knowledge Management bulletin
• Summary of analysis sent to all those who
  responded to the survey
• Press release
• Provider Forums (if timings appropriate)

19
How Cumbria use results from National UES (3)

• Summary produced for providers by our Contracts
  Team
• 2008/2009 results sent to all Home Care Providers
  we contract with
• Highlights specifics we want to them to note
• This includes high satisfaction areas as well as
  areas for improvements
• Offer to present / discuss in more detail

20
How Cumbria use results from National UES (4)

• Contract Team found that the results reaffirm
  what is found at individual audit level
• They are continuing to work with providers to
  improve SU satisfaction especially around changes
  to care provision and time spent at users' homes
• They will re-examine their audit process for next
  year in response to the results about knowledge
  of how to raise safeguarding issues

21
How Derbyshire use results from National UES

• In-depth analysis along the lines of NHS IC and
  PSSRU using frequency counts and
  cross-tabulations in SPSS
• Analysed by SU group, gender, age, ethnicity,
  locality / area
• For Disability Survey (2007) also analysed by
  whether in receipt of Direct Payments, with a
  caring responsibility (additional question)
• Comparisons made across these groups to look for
  any differences
• Comparisons made to previous results to identify
  increase or decrease in satisfaction /
  performance over time
• Comments analysed using thematic analysis

22
How Derbyshire use results from National UES

• Results presented in Perspectives report now
  well recognised throughout the Department
• Provides in-depth analysis
• Graphical representations
• Easy to read
• Includes some comments to provide fuller picture
• Concludes with Opportunities and Actions section

23
How Derbyshire use results from National
UESProcess

• Perspectives report presented to Senior
  Management Team meeting
• Report approved and possible actions discussed at
  senior level
• Report disseminated widely throughout the Dept
  cascaded via Area Managers published on DCC
  website and press release
• Service Improvement Plan developed in
  collaboration with relevant Assistant Director
• Assistant Director nominates a senior responsible
  person to oversee the SIP
• SIP monitored centrally along with all
  departmental action plans to ensure completion
• Outcome measures collected and collated for Self
  Assessment Survey and evidence for ISO 9001

24
How Derbyshire use results from National
UESOutcomes
25
How Derbyshire use results from National
UESOutcomes

• An example from Home Care Survey SIP
• What service users said they wanted to be
  better informed about day to day changes in their
  care
• What we did - We introduced Staff Plan, an
  electronic scheduling system, which sends out a
  weekly schedule to service user and home helps
  the week before scheduled care
• Outcome a 5 increase in satisfaction for
  service users being kept informed about day to
  day changes in their home care

26
How Derbyshire use results from National
UESOutcomes

• An example from Disabled People Survey SIP
• What service users said people wanted more
  opportunities for employment
• What we did - We launched the Disability
  Employment Project to help people with Learning
  Disabilities to find work including giving people
  training to travel independently and supporting
  them through job applications and interviews
• Outcome in the two years since its launch, the
  project has helped 350 people find work