Development of an Automated Nephrotoxicity Pharmacosurveillance System

1
Development of an Automated Nephrotoxicity
Pharmacosurveillance System

• Michael E. Matheny, MD MS MPH
• Division of General Internal Medicine
• Vanderbilt University, Nashville, TN
• Geriatrics Research, Education Clinical Care
• Veterans Administration, Nashville, TN

2
Research Plan

• Background Significance
• Objectives
• Work Accomplished
• Work Proposed
• Training Activities Objectives

3
BackgroundSurveillance Rationale

• Phase 3 Trials insufficient to ensure adequate
  safety of medications and devices
• Low frequency events are not detected
• Protected populations (pregnant women, children)
  and more ill populations not represented
• Complications delayed by a number of years cannot
  be detected

4
BackgroundCurrent FDA Post-Marketing Surveillance

• Combination of mandatory and voluntary adverse
  event reporting
• Mandatory reporting by manufacturers and health
  facilities
• Voluntary MedWatch / MAUDE reports by providers
  and patients

2004 Drug-Related Adverse Event Reports 2004 Drug-Related Adverse Event Reports
Total 422,889
Manufacturer Facility Reports 401,396
MedWatch 21,493
5
BackgroundCurrent Post-Marketing Surveillance

• Phase 4 Trials
• Poor Compliance
• As of March 2006 report, 797 of 1231 (65)
  agreed-upon trials had yet to be started
• Barriers
• Lack of manufacturer incentives
• Expensive
• Drug already on the market
• Lack of regulatory enforcement

6
BackgroundFDA Response

• Legislation
• Incorporate Phase 4 Trial costs into Approval
  Process
• Increase quality of adverse event reporting
• Complementary Data Sources
• Commissioned IOM report The Future of Drug
  Safety
• Public Hearing in 2006
• Aggregate Existing Clinical Registry Data
• Promote Use of Routine Medical Data

7
BackgroundTransition

• Transition slide from surveillance to domain

8
BackgroundAcute Kidney Injury Statistics

• 1-7 of all hospital admissions
• 5-20 during an ICU stay
• Crude Mortality Estimates
• 15 isolated AKI
• 25 general inpatient
• 40 in ICU
• National trend of increasing incidence

9
BackgroundCauses of Acute Kidney Injury

• Acute Medical Conditions
• Rhabdomyolysis 15-20
• Sepsis 40-50
• Major Surgery 10-20
• Congestive Heart Failure 20
• Myocardial Infarction
• Any condition that generates a real or apparent
  decreased intravascular volume

10
BackgroundCauses of Acute Kidney Injury

• Medications
• Nephrotoxic medications cause 15-25 of AKI
• Aminoglycosides 8-26
• Single Daily Dose ? risk
• Amphotericin 50-80
• Lipid Formulation ? risk
• NSAID
• ACE Inhibitor
• Immunosuppressive Agents
• Radiocontrast Dye
• Separate category, cause 10-15 of AKI

11
Objectives

• Validation of an Automated Data Collection System
  for Acute Kidney Injury Monitoring
• Retrospective Evaluation of Acute Kidney Injury
  among Hospitalized Veterans
• Pilot an Acute Kidney Injury Automated
  Surveillance System among Hospitalized Veterans

12
Prior Work
13
Methodology Application

• Adaptation of Statistical Process Control
  Monitoring Methodologies
• Design and Implementation of a Web-Based
  Monitoring Application (DELTA)

Matheny ME, Ohno-Machado L, Resnic FS.
Monitoring Device Safety in Interventional
Cardiology. J Am Med Inform Assoc.
200613(2)180-7.
14
Methodology ValidationTIMI Randomized Trials
Method SPC BUS LR-SPC
TP 8 0 8
FP 4 0 2
FN 0 8 0
TN 23 27 25
Sensitivity 100 0 100
Specificity 85 100 93
Matheny ME, Morrow DA, Ohno-Machado L, Cannon CP,
Sabatine MS, Resnic FS. Med Decis Making. 2007.
Submitted
15
SPC vs BUS Sensitivity AnalysisSimulated PCI Data
Matheny ME. Deelopment of Statistical
Methodologies and Risk Models to Perform
Real-Time Safety Monitoring in Interventional
Cardiology. Master of Science Thesis
Cambridge Massachusetts Institute of
Technology 2006.
16
Clinical Event DetectionVascular Closure Device
Retroperitoneal Hemorrhage

• After manual review, access above inguinal
  ligament was significant and VCD no longer.
• Implemented Fellow Access Education Intervention

Matheny ME. Arora N, Ohno-Machado L, Resnic FS.
Rare Adverse Event Monitoring of Medical Devices
with the Use of an Automated Surveillance Tool.
AMIA Annu Symp Proc 2007. Accepted
17
Process of Care Evaluation (SPRT)Post-Procedural
Mortality Among 18 BWH Operators

• Found 1 clear Outlier, after case review, most
  were compassionate care
• Institution was less than 1.5 OR of national
  expectations

Matheny ME, Ohno-Machado L, Resnic FS.
Risk-Adjusted SPRT Control Chart Methods for
Monitoring Operator and Institutional Mortality
Rates in Interventional Cardiology. Am Heart J.
Accepted
18
Other Relevant Prior WorkRisk Prediction Modeling

• LR External Validation and Model Development (PCI
  Mortality)
• Matheny ME, Ohno-Machado L, Resnic FS. J Biomed
  Inform 200538367-375
• Evaluation of Support Vector Machine Modeling for
  use in PCI Clinical Registry Data
• Matheny ME, Resnic FS, Arora N, Ohno-Machado L.
  J. Biomed Inform. Accepted
• Literature Review - Development and Evaluation of
  Critical Care Mortality Models
• Ohno-Machado L, Resnic FS, Matheny ME. Prognosis
  in Clinical Care. Annu Rev Biomed Eng.
  20068567-99
• Book Chapter - Statistical and Machine Learning
  Risk Predication Modeling
• Matheny ME, Ohno-Machado L. Clinical Decision
  Support The Road Ahead (Ed Greenes RA) Elsevier
  2006.

19
Proposed Work
20
Conceptual Model
21
Validation of an Automated Data Collection
System for Acute Kidney Injury

• Hypotheses
• Clinical data that does not require free text
  NLP, such as medication orders and laboratory
  tests, will achieve at least 85 specificity and
  sensitivity.
• Clinical data that requires free text NLP, such
  as clinical diagnoses, will achieve at least 75
  specificity and sensitivity.

22
Development of ADCSData Sources

• All records in 2005-2006 for patients
  hospitalized in Nashville TVHS during 2006

23
Data Element Identification

• Literature Review of AKI Risk Factors
• univariate and multivaraite risk factors
• General Patient Demographics
• Manual chart review from randomly selected
  Nashville TVHS Patients hospitalized during 2006
  to identify locations and data quality for each
  identified data element

24
Data Element Selection

• Complete capture of all relevant variables may be
  infeasible from routine clinical data.
• Review of data element list with location and
  quality data with mentorship team
• Elements will be selected based on a combination
  of
• strength of association with AKI
• data collection quality
• algorithm complexity required for data extraction

25
Clinical Data Rule Development

• Criteria must be developed for each clinical
  diagnosis
• Example status post nephrectomy
• Identification Direct documentation
• Possible locations operative note, problem list,
  outpatient progress note, inpatient progress
  note, inpatient discharge summary, or CT scan of
  the abdomen.
• Requires free text processing of multiple
  sources, could use Perl word matching instead of
  concept-based indexing
• Example systemic inflammatory response syndrome
• Identification Direct documentation or by
  clinical criteria
• Clinical Criteria abnormal HR, Temp, RR or
  spO2, or WBC.
• Locations of direct mention problem list,
  inpatient progress note, inpatient discharge
  summary
• Requires use of inpatient lab and vital sign
  data AND/OR processed free text notes

26
Primary Outcome

• Acute Kidney Injury by RIFLE Criteria
• Risk
• Creatinine x 1.5 above baseline
• Or urine output lt 0.5 ml/kg/hr x 6 hours
• Injury
• Creatinine x 2 above baseline
• Or urine output lt 0.5 ml/kg/hour x 12 hours
• Failure
• Creatinine x 3 above baseline
• Or anuria or lt 0.3 ml/kg/hour x 12 hours
• UOP not well captured, that criteria frequently
  ignored in the literature
• Elevation needs to last at least 24 hours

27
Rule Validation

• An annotated VA corpus with AKI-related data
  elements does not exist, and must be created
• Power Calculations
• Assumption incididence of clinical data range
  from 0.2 to 0.8
• Alphabeta0.05
• One sided test of single proportion for
  diagnostic test sample (Flahault)
• Requires manual review of 400 charts for
• 85 sensitivity/specificity with lower CI of 70
• 75 sensitivity/specificity with lower CI of 58
• 60 training sample, 40 testing sample
• Random sampling from Nashville TVHS 06-07

28
Initial Estimate of Data Source Needs

• Demographics
• Labs
• Inpatient and outpatient Orders
• Vital sign data (inpatient/outpatient)
• Medications
• Problem List
• Physician authored Free Text Notes outpatient
  and inpatient
• Radio-contrast Dye If not in orders, will
  need radiology ordering, if not there, will need
  radiology free text

29
Retrospective Evaluation of Acute KidneyInjury
among Hospitalized VISN9 Veterans

• Hypotheses
• Concurrent use of diuretics, NSAIDs, ace
  inhibitors and/or calcium-channel blockers, which
  are known to cause pre-renal acute kidney injury,
  will result in significant synergistic renal
  injury.
• Use of loop diuretics among hospitalized patients
  with acute kidney injury will be associated with
  higher rates of incomplete recovery of renal
  function.
• Variation in the risk-adjusted incidence of acute
  kidney injury due to the process of clinical care
  can be detected on both an institutional and
  medical specialty level between VISN9 hospitals

30
Retrospective EvaluationData Sources

• Retrospective Patient population All VISN9
  patients hospitalized during 2006 2008
• Will need 2005 (maybe 2004) patient data for past
  medical history and baseline creatinines in order
  to analyze hospitalizations starting in 2006

31
Synergistic Pre-Renal Acute Kidney Injury

• Outcomes of Interest Pair-wise interaction
  between medications known to cause pre-renal AKI
• Populations 2 separate populations
• Admitted to hospital with AKI
• Develop AKI during hospital stay (gt48 hours after
  admission)
• SAS genmod - evaluate AKI (Risk, Injury, Failure)
• For the interaction terms of
• loop diuretics CCB
• loop diuretics ACE
• loop diuretics NSAIDS
• CCB ACE
• CCB NSAID
• ACE NSAID
• after full risk adjustment of these factors
• Age, gender, race, insurance status, medical
  co-morbidities, acute clinical conditions,
  co-existing medications

32
Loop Diuretic Use Among AKI Patients

• Outcomes of Interest gt3 month baseline
  creatinine after inpatient AKI
• Evaluate the association of failure to return to
  pre-hospitalization baseline and use of loop
  diuretics after the diagnosis of AKI in the
  hospital
• after full risk adjustment of these factors
• Age, gender, race, insurance status, medical
  co-morbidities, acute clinical conditions,
  co-existing medications
• Additional Data Requirements Additional time
  required to allow follow-up creatinine monitoring
  post-discharge

33
Evaluation of Variations in Clinical Care

• Develop multivariable risk prediction model for
  development of AKI among VISN 9 patients
  (separate manuscript)
• Evaluate risk-adjusted incidence of AKI by
  institution and clinical specialty to determine
  if outliers exist using VISN9 (overall) data as
  the baseline (uses above risk prediction model).

34
Retrospective Evaluation of Acute KidneyInjury
among Hospitalized VISN9 Veterans

• Knowledge Discovery (not hypothesis driven)
• Characterize the risk for acute kidney injury
  with the use of the following medications
• NSAIDs
• ACE Inhibitors
• ARBs
• Radiocontrast Dye
• Aminoglycosides
• Amphotericin B
• after full risk adjustment of these factors
• Age, gender, race, insurance status, medical
  co-morbidities, acute clinical conditions,
  co-existing medications

35
Pilot Evaluation of an Inpatient Acute Kidney
Injury Surveillance System

• Hypotheses
• The methodology and application will detect a
  simulated AKI event rate elevation among
  retrospective routinely collected clinical data.
• A prospective automated outcomes surveillance
  system evaluating acute kidney injury among
  hospitalized veterans will allow discovery of
  medication-related nephrotoxicity when pharmacy
  formularies are changed or new medications are
  introduced.

36
Surveillance System Simulated AKI event rate
elevation

• Use the retrospective 06-08 VISN9 data, and
  simulate an AKI event rate elevation in a
  medication not known to be nephrotoxic with a
  simulated outbreak algorithm frequently used in
  biosurveillance
• Allows a sensitivity analysis to detect how
  significant the elevation must be (how many cases
  in what period of time) before the system will
  alert.
• Good prelim data for a merit application

37
Prospective Pilot of an Inpatient Acute Kidney
Injury Surveillance System

• Having some trouble with this hypothesis.
• What I really want to do is to setup the
  prospective system after doing the simulated
  detection, but theres no way to know what, if
  anything, it will detect as time goes no.
• How to frame that?

38
Training PlansYear 1

• Formal Classes
• Database Development
• Perl course / regular expression parsing
• Seminar
• updates in acute kidney injury
• Ongoing Funded Grants
• POEM (Nashville, TN) - Attend grant meetings
  dealing with clinical concept tagging project
• DELTA-MASSDAQ (Boston, MA) Attend grant
  meetings for statistical and application
  development
• Conferences
• AMIA
• NKF
• VA HSRD
• Journal Club

39
Training PlansYear 2

• Formal Classes
• Statistical Process Control
• Methods for Confounding Adjustment in Prospective
  Cohorts
• Ongoing Funded Grants
• POEM (Nashville, TN) - Attend grant meetings
  dealing with clinical concept tagging project
• DELTA-MASSDAQ (Boston, MA) Attend grant
  meetings for statistical and application
  development
• Conferences
• AMIA
• NKF
• VA HSRD
• Journal Club

40
Training PlansYear 3

• Seminar
• Grant Writing
• Conferences
• AMIA
• NKF
• VA HSRD
• Journal Club

41
Training PlansYear 4

• Conferences
• AMIA
• NKF
• VA HSRD
• Journal Club
• Submit a HSRD IIR Merit Grant Application

42
Michael Matheny, MD MS MPH michael.matheny_at_vande
rbilt.edu Tennessee Valley Medical Center -
NashvilleGeriatric Research, Education and
Clinical CareRoom 4-B1101310 24th Ave.
S.Nashville, TN 37212
The End