Feedback from reporting systems

1
Feedback from reporting systems

• Insights from a scoping study of methods of
  providing feedback within an organisation in
  England and Wales- 2004-6
• Wallace LM, Koutantji, M,BennJ, Spurgeon P,
  Vincent C
• DH Patient Safety Research Programme

Applied Research Centre Health and Lifestyle
Interventions
2
Rationale for focus on feedback learning

• UK Dept of Health report An Organisation with a
  Memory (2000)
• NHS does not actively learn from failures
• Existing systems take a long time to feed back
  information and recommendations
• There is little or no systematic follow-up of
  recommendations
• Recommended establishing reporting and learning
  systems at local and national level.

Applied Research Centre Health and Lifestyle
Interventions
3
Rationale for focus on feedback learning

• National Audit Office survey1 Committee of
  Public Accounts2 A Safer Place for Patients
• emphasis on reporting and counting, but not on
  analysis and learning
• there is a need to improve sharing of solutions
  by all organisations
• lessons learnt on a local level are not widely
  disseminated either within or between trusts
• considerable complexity in reporting and feedback
  channels currently exists (multiple agencies
  responsible)

Applied Research Centre Health and Lifestyle
Interventions
4
PSRP Feedback Research Programme
Scoping review of feedback from safety
monitoring systems in health care and high risk
industries (Imperial College London)

• Requirements for effective safety feedback with
  rationale
• Model of feedback/control process for safety
  monitoring systems

2. Expert Panel Review
1. Literature Review
Empirical studies of feedback systems in 351
trusts and 3 in-depth case studies (Coventry
University)
Expert Review Workshop with health care
professionals and domain experts
OUTPUT Design for effective safety feedback
systems for health care
Applied Research Centre Health and Lifestyle
Interventions
5
Safety issue management process
1. Detect
1. Incident report monitoring
2. Capture
3. Classify
4. Analyse
Iterate in ongoing cycle
2. Safety issue analysis
5. Prioritise
6. Investigate
7. Formulate
3.Solutions development systems improvement
8. Implement
9. Monitor
6
Corrective action and safety information feedback
Improvements in the design of work systems
Identified system vulnerabilities
Safer Work Systems
SAFETY FEEDBACK Learning lessons from
operational experience
Information on operational risks
Increased awareness of front line staff
7
OPERATIONAL LEVEL
LOCAL ORGANISATIONAL LEVEL
SUPRA- ORG. LEVEL
SAFETY ISSUE PROCESS
Incident reports
High-level and external reporting requirements
All classified incidents
Incident repository
1. Incident report monitoring
Single incidents priority issues identified for
follow-up
Aggregated data from multiple incidents
Local clinical work systems
2. Safety issue analysis
Corrective action loop
Care providers patients
Root causes, contributory factors and key trends
Integrate support changes
3.Solutions development systems improvement
Local implementing agents leadership
Systems improvements
8
Reporting systems in expert panel
9
Example feedback mechanisms from review of 23
best case health care reporting systems

• Staff bulletin board postings with safety issues
  raised and actions taken (Holzmueller et al.,
  2005 Lubomski et al., 2004)
• Targeted staff training programmes (Takeda et
  al., 2003)
• Development of manuals on error prevention
  (Wilf-Miron et al., 2003)
• One-to-one telephone debriefings with reporters
  (Wilf-Miron et al., 2003)
• Departmental presentations and quality meetings
  (Parke, 2003)

Applied Research Centre Health and Lifestyle
Interventions
10
Modes of feedback from incident reporting
Safety Action and Information Feedback from
Incident Reporting (SAIFIR) framework
11
15 Requirements for effective feedback Examples
based on expert opinion
12
Framework for Safety Action and Information
Feedback from Incident Reporting (SAIFIR)
13
Survey of feedback learning in NHS trusts

• 351 trusts responded (out of all 607 English and
  Welsh trusts contacted) 58 response rate
• Administered between November 2005 and March 2006
• Respondent Local risk management leads
• Survey items
• Development of an open, no-blame culture
• Development and structure of local level
  reporting systems
• Analysis and use of information from incident
  reports
• Formulation and implementation of safety
  solutions
• Feedback mechanisms and methods of dissemination

Applied Research Centre Health and Lifestyle
Interventions
14
Comments on the SAIFIR model applied to UK NHS

• Not linear system-has to account for reporting by
  observers, not always those involved in
  incident, or can arise from other healthcare
  services in pathway- unclear to whom feedback is
  needed.
• Filtering, and feedback occurs prior to
  reporting- (Mode A)
• Non Healthcare systems are more mature- can short
  circuit some analysis by having standard
  responses to known incident types as causes known

Applied Research Centre Health and Lifestyle
Interventions
15
Mode A acknowledgement, clarification,
categorisation

• UK NHS
• Only a third trusts acknowledge and inform re
  incident handling.
• Needs dialogue with reporter and affected staff-
  e systems show promise

• Research-
• Pre-categorisation of incidents- does it deter
  reporting/ lack detail?

Applied Research Centre Health and Lifestyle
Interventions
16
Mode B rapid response recovery prevention
actions

• UK NHS
• Little evidence of this occurring- and fear in
  NHS of premature formulation and harmful action.
• Would require systems to identify known incident
  types and reliable actions.

• Research
• Experiments in rapid feedback are needed- using e
  mail and other targeted means- analogue and field
  experiments.

Applied Research Centre Health and Lifestyle
Interventions
17
Mode C Dissemination of learning

• UK NHS
• Limited effectiveness shown in survey reliance
  on one way communication known to be ineffective
  but most used.
• Audit of newsletters- most use for alerts,
  national guidance, not targeted feedback.

• Research
• Identifying common incident scenarios with known
  causes- testing most effective methods of
  communicating required actions, and uptake- eg by
  requiring an implementation plan and triggers for
  action.

Applied Research Centre Health and Lifestyle
Interventions
18
MODE D Outcome of incident investigation

• UK NHS
• 68 trusts communicate some information to
  reports/ affected staff and these have only
  modest levels of impact.
• Wider sharing of outcomes (esp. causes and
  solutions rather than incident types) required
  within services , across trusts, within regions
  and the national bodies (e.g. NPSA).

• Research
• Experiments on effective modes of feedback and
  impact on intentions and actions re future
  reporting and safety actions. Are vignettes more
  effective, tan data reports?

Applied Research Centre Health and Lifestyle
Interventions
19
MODE E Implementing actions to improve safety

• UK NHS
• One third of trusts do not produce effective
  guidelines as a result of incident reporting, and
  25/351 had no system for monitoring if this was
  implemented.
• Recommended changes should be targeted,
  actionable, monitored, in double loop learning.

• Research
• Using known methods of improving adherence to
  guidance- test impact on safety behaviour and
  reporting.

Applied Research Centre Health and Lifestyle
Interventions
20
High level conclusions

• Attention must be given to the use of information
  from incident reports to improve safety
• There is a lack of evaluative evidence concerning
  effective modes of feedback
• There is wide variation in trusts practice in
  terms of information and action feedback to front
  line work systems
• Little evidence of capacity for rapid action in
  Trust systems
• Little evaluation of impact of actions upon
  operational safety
• Feedback should be timely and targeted to
  specific practitioners
• Safety actions should be monitored and their
  effectiveness evaluated

Applied Research Centre Health and Lifestyle
Interventions