MONITORING THE QUALITY OF CARE USING CLINICAL INDICATORS

1
MONITORING THE QUALITY OF CARE USING CLINICAL
INDICATORS
Jan Mainz, MD, Ph.D. Project Manager, The
National Indicator Project, Denmark Associate
Professor, University of Aarhus,
Denmark WWW.NIP.DK
2
DEFINITIONS

• Indicators provide a quantitative basis for
  clinicians, providers, organisations and planners
  aiming to achieve improvement in care and the
  processes by which patient care is provided.
• (ISQua, Melbourne 1999)
• Indicators are quantitative measures that can be
  used to monitor and evaluate the quality of
  important governance, management, clinical, and
  support functions that affect patient outcomes.
• (Joint Commission, 1990)

3
DEFINITIONS

• Indicators should be explicit statements of
  desirable (or undesirable) structural, process or
  outcome dimensions.
• They should be supported by either research that
  establishes the efficacy or effectiveness of the
  indicators by a formal process of obtaining
  experts consensus.
• The tools for measurement should be tested and
  evaluated for reliability, validity and
  feasibility.
• Results should be repeated in a format that
  maximizes the likelihood that the information can
  be interpreted and used in appropriate decision
  contexts.
• (RAND, 1998)

4
DEFINITIONS

• Indicators are measures that assess a particular
  health care process or outcome.
• (European Medical Associations, 1992)
• Indicators are measures of the clinical
  management and outcome of care and are objective
  measures of either the process or outcome of
  patient care in quantitative terms.
• (ACHS, 1993)

5
USES OF INDICATORS

• To document the quality of care
• To make comparisons
• Over time
• Between places (e.g. hospitals)
• To make judgements and priorities
• e.g. choosing a hospital or surgery
• e.g. organising medical care
• To support accountability
• To support quality improvement
• Transparency for society

6
SAME MEASURE CAN SERVE MUTIPLE PURPOSES

• Physician
• - How am I doing?
• Patient
• - What are my chances?
• - Which is the best hospital?
• - Which is the best doctor?
• Society
• - What does it cost?

7
KEY CHALLENGE

• Everyone wants measurement
• No one wants to be measured

8
INDICATORS

• Must be precisely defined i.e.
• Be based on agreed definitions which can be
  uniformly implemented.
• Have specificity
• Be valid and reliable (validation and reliability
  testing)
• Have discrimination ability
• Be risk-adjusted to enable comparison
• Relate to clearly identifiable events
• Permit useful comparisons
• Evaluation and review would incorporate changes
  over time

9
INDICATORS

• Significance
• Ownership reflected in the development and use
• Widely accepted
• Ease of data extraction
• Be interpreted in the light of socio-economic and
  cultural issues.
• Responsive
• Will not violate patient confidentiality
• Be cost effective
• Be public available

10
INDICATORS ARE BASED ON

• Best evidence
• (cochrane, metaanalyses, RCT etc)
• Consensus among health professionals

11
INDICATORS

• Can be categorised by
• Type of care
• Preventive
• Acute
• Chronic
• Function
• Screening
• Diagnosis
• Treatment
• Follow up
• Modality
• History
• Physical examination
• Laboratory/radiology study
• Medication
• Other interventions
• Generic or disease specific
• Rate-based or sentinel

12
DEFINITIONS E.G NOSOCOMIAL INFECTIONS

• Dirty Operations in which a perforated viscus or
  pus is found.
• Contaminated Operations breaching the
  gastrointestinal, respiratory and genitourinary
  tracts, or in which a break in aseptic technique
  occurs and in traumatic wounds.
• Clean All other operations where the criteria
  set out in dirty and contaminated do not
  apply.
• Wound infection Any surgical wound from which
  purulent material drains or is obtained.
• Hospital-acquired bacteraemia A positive blood
  culture for inpatients who were afebrile on
  admission (i.e. temperatures less than 37,4C) on
  blood collected 48h after admission.
• Reference ACHS. J. Qual. Clin Practice 1997

13
DATA FORMAT E.G NOSOCOMIAL INFECTIONS

• Clean and contaminated wound infection
• A) Numerator The number of patients who develop
  wound infection from the fifth post-operative day
  after (i) clean surgery, (ii) contaminated
  surgery.
• B) Denominator The total number of patients
  undergoing (i) clean and (ii) contaminated
  surgery within the time period under study who
  have a post-operative length of stay of 5 or more
  days
• Hospital-acquired bacteraemia
• A) Numerator Total number of patients who
  acquire bacteraemia as defined above.
• B) Denominator Total number of patients in
  hospital during the study period.
• Reference ACHS. J. Qual. Clin Practice 1997

14
CONCEPTUAL FRAMEWORK

• Structural quality
• assesses health system characteristics that
  affect the systems ability to meet the health
  care needs of individual patients or a community
  (e.g. the nurse-to-bed ratio in a hospital)
• Process quality
• assesses what the provider did for the patient
  and how well he or she did it (e.g. proper
  diagnostic approach to symptoms)
• Outcome quality
• assesses the influence of the health care
  delivery process on the individuals health (e.g.
  morbidity and mortality)

15
EXAMPLES OF STRUCTURE INDICATORS

• Numbers of specialists compared to other doctors
• Access to specific technologies
• Availability of specific units (e.g.. stroke
  units)
• Clinical guidelines revised every 2nd year
• Physiotherapists associated to specific units

16
EXAMPLES OF PROCESS INDICATORS

• Patients treated according to clinical guidelines
• Patients with MI, who received thrombolyses
• Door to needletime for MI patients
• Breast cancer patients lt 75 years, who got
  axillary resections
• Waiting time for doctor contact for patients
  admitted acute

17
OUTCOMES OF DISEASES (THE FIVE Ds)

• Death A bad outcome if untimely
• Disease A set of symptoms, physical signs and
  laboratory abnormalities
• Discomfort Symptoms such as pain, nausea,
  dyspnoea etc.
• Disability Impaired ability connected to usual
  activities at home, work or in recreation
• Dissatisfaction Emotional reactions to disease
  and its care, such as sadness or anger

18
EXAMPLES OF INTERMEDIATE OUTCOME INDICATORS

• HbA1C for diabetics
• Lipid profile for patients with hyperlipidemia
• Numbers of lymph nodes removed at breast cancer
  surgery
• Blood pressure for hypertensive patients

19
OUTCOME INDICATORS

• Mortality
• Morbidity
• Functional status
• Health measurement status
• Work status
• Complications
• Quality of life
• Patient satisfaction

20
THE OUTCOME OF CARE

• The Patient
• Demographic factors (age, sex, height)
• Lifestyle factors (smoking, alcohol, weight,
  diet, physical exercise)
• Psychosocial factors (social status, education)
• Compliance
• The Illness
• Severity, prognosis
• Comorbidity
• The Treatment (Prevention, diagnostics, care,
  rehabilitation, therapy and control)
• Competence
• Technical equipment
• Evidence based clinical practise
• Efficacy, accuracy
• The Organisation
• Use of clinical guidelines

21
PROBLEMS WITH INDICATOR USE

• Inappropriate definitions
• Ranking instability
• Discriminative power
• Viewed as absolute measure of quality
• Lack of timely access
• Data incompleteness
• Lack of interest
• Lack of trust
• Lack of ownership

22
DIMENSIONS OF THE QUALITY OF CARE

• Quality of the technical care in terms of
  prevention, diagnostics, treatment and
  rehabilitation
• Quality of the interpersonal relationship in
  terms of communication and information
• Quality of the organisation of care in terms of
  continuity and coordination

23
WHAT DO WE KNOW ABOUT THE QUALITY OF THE
TECHNICAL CARE?

• Lack of documentation about how major illnesses
  are treated in the health care system
• Few goals regarding the technical quality
• Lack of outcome assessment
• Lack of resource evaluation
• Persisting variations
• No formal monitoring systems
• The principal quality problems and their
• prevalence and incidence are unknown

24
THE NATIONAL INDICATOR PROJECT-a concerted
action between

• The Ministry of Health
• The National Board of Health
• The County Counsellors Association
• The Scientific Societies
• The Danish Medical Association
• The Danish Nursing Association
• The Danish Physiotherapist Association

25
THE NATIONAL INDICATOR PROJECT AIMS

• Improving prevention, diagnostics, treatment and
  rehabilitation
• Documentation for making priorities
• Information for patients and consumers

26
THE NATIONAL INDICATOR PROJECT

• All major diseases are evaluated
• Evidence based process and outcome indicators are
  derived by health professionals on national level
• Health professionals and clinical epidemiologists
  are responsible for data-collection, analyses,
  evaluation and interpretation of results
• Hospitals are compared at county and national and
  international levels
• Audit activities are organised at county and
  national level
• Improvements are initiated if necessary

27
BASIC PRINCIPLES

• Health professionals develop evidence based
  standards and indicators for all major diseases
• Health professionals assess and interpret results
  before public release of data

28
THE NATIONAL INDICATOR PROJECT

• Established 2000
• Developed 6 sets of indicators covering 96
  individual clinical indicators
• Mandatory participation by all hospitals and
  relevant clinical departments in Denmark.

29
DISEASES

• Stroke
• Hip fracture
• Schizophrenia
• Acute surgery
• Heart failure
• Lung cancer

30
INDICATORS

• Stroke
• Stroke patients treated at stroke units
• Medical secondary prophylactic treatment
• CT/MR scan
• Patients assessed by physiotherapist
• Patients assessed by occupational therapist
• Assessment of nutritional status
• Mortality at 30 days, 3,6 and 12 months
• Discharge destination

31
CLINICAL INDICATORS STROKE I
32
CLINICAL INDICATORS STROKE II
33
CLINICAL INDICATORS STROKE III
34
CLINICAL INDICATORS STROKE IV
35
INDICATORS

• Schizophrenics
• Assessment of Side-effects of The Psychotic drugs
• Family Intervention
• Psycho - education
• Pharmacological Treatment
• Continuity of Contacts to The Health Care System
• Psychosis relapse
• Functional Status

36
CLINICAL INDICATORS SCHIZOPHRENIA I
37
CLINICAL INDICATORS SCHIZOPHRENIA II
38
CLINICAL INDICATORS SCHIZOPHRENIA III
39
CLINICAL INDICATORS SCHIZOPHRENIA IV
40
INDICATORS

• Lung Cancer
• Survival
• Delays in Diagnosis and Treatment
• Staging (Classification)
• Treatment and Resection (Appropriateness)
• Admission Time the last 3 months before Death
• Supporting Ambulatory Psychological Contact
  (Palliative Care)

41
CLINICAL INDICATORS LUNG CANCER I
42
CLINICAL INDICATORS LUNG CANCER II
43
CLINICAL INDICATORS LUNG CANCER IV
44
INDICATORS

• Hip Fracture
• Risk Assessment of Nutritional Status
• Pain Intensity at Mobilisation at 5th Post
  Operative Day
• Functional Status at 5th Post Operative Day
• ADL Functional Recovery
• Discharge Destination
• Reoperative by Different Types of Fractures
• Mortality at 30 Days, 4, 6, 12 months

45
CLINICAL INDICATORS HIP FRACTURE I
46
CLINICAL INDICATORS HIP FRACTURE II
47
CLINICAL INDICATORS HIP FRACTURE III
48
PHASE 1 COMMENCEMENT OF THE TASK

• Selection of group participants
• Classification of concepts, definitions and
  limitations
• Organising the work in the group. Creation of
  smaller working subgroups and delegation of
  projects to individual group participants

49
PHASE 2 OVERVIEW OF EXCISTING KNOWLEDGE AND
PRACTICE

• Presentation of knowledge and documentation from
  literature and meta analyses
• Consensus about existing knowledge, practice and
  conclusions
• Determination of the clinical epidemiological
  task

50
PHASE 3 DETERMINATION OF PROFESSIONAL INDICATORS

• Determination of professional indicators based on
  existing professional and clinical
  epidemiological knowledge
• Decisions about data collection. Inclusion
  criteria. Exclusion criteria.
• Determination of interpretation (how to read
  data, limination and acceptable tolerances)

51
PHASE 4 DATA COLLECTION

• The data collection should be supported by
  clinical epidemiologists
• Data from medical records, questionnaires,
  clinical databases, registers should be used

52
CLINICAL DATABASES

• A register, that contains specific clinical
  indicators, which can describe the quality of
  care for a specific patient group.

53
PHASE 5 DATA ANALYSES

• Analyses, evaluation, interpretation
• Professional discussions of processed data
  results

54
INTERPRETATIONS OF FINDINGS

• If differences are assessed in a clinical course
• there are different levels of explanation
• Bias due to selection of patients or bias in data
  collection
• Confounding (different prognostic factors between
  groups)
• Random variations
• There is a difference

55
PHASE 6 REFLECTIONS AND DESCRIPTIONS

• Reflections about cause-and effects. Connections
  between these and conclusions based on data and
  cause analysis
• Creations of reports to hospital quality
  committees
• Feed back to professionals
• Implementing quality improvement

56
PUBLIC ACCESS TO DATA

• Data are released to the public at
• - National level
• - County level
• - Hospital level
• - Clinical unit level
• Data on individual doctor level will not be
  released

57
COMMUNICABILITY

• Data must be transformed into meaningful and
  useful information, reportable in a relevant
  manner to all stakeholders
• Feedback required on a regular basis for
  behavioural change

58
PERSPECTIVES

• Research
• CME
• Quality development
• - Clinical guidelines
• - Patients pathways
• - Audit

59
INDICATORS

• Are strategic markers monitoring aspects of the
  quality of care
• Measure the extent to which set targets are
  achieved
• The surveillance of health care quality is
  impossible without the use of relevant indicators
• Should be valid (measure exactly what we want to
  measure)
• Should be sensitive (reflect correctly changes
  occurring given the situation)
• Should be specific (to avoid the measurement of
  changes arising from external factors not related
  to the objectives and targets)
• Should be evidence based
• The use of indicators should be followed by
  professional assessment, evaluation and
  interpretation

60
PERSPECTIVES

• THE EUROPEAN INDICATOR PROJECT
• under the flag of
• The European Union
• and
• ESQH

61
The European Indicator ProjectPRINCIPLES

• Health professionals develop evidence based
  standards and indicators for all major diseases
• One or two indicators for specific diseases shoul
  be developed
• Health professionals assess and interpret results
  before public release of data

62
The European Indicator ProjectAIM

• Improving prevention, diagnostics, treatment and
  rehabilitation
• Documentation for making priorities
• Information for patients and consumers
• International benchmarking

63
The European Indicator ProjectORGANISATION

• All interested European countries would be able
  to participate
• Representatives from each country establish a
  steering committee
• International and national fundings should
  finance the project
• Important diseases are identified which would be
  relevant for international comparisons
• The project should be conducted within a limited
  time period (e.g. 3 years)
• The project should be evaluated in order to
  decide whether it would be feasible to continue
  the project

64

• I am called eccentric for saying in public that
  hospitals, if they wish to be sure of
  improvement, must find out what their results
  are. Must analyze their results to find their
  strong and weak points. Must compare their
  results with those of other hospitals
• Such opinions will not be eccentric a few years
  hence.
• E.A. Codman, MD, 1917.

65
In God we trust everybody else has to bring
data