Can Australian GPs trust their software

1
Can Australian GPs trust their software?
Dr Douglas IR Boyle1, Prof Siaw-Teng Liaw1, Prof
Iain Morrison2, Dr Maddalena Cross1
1 University of Melbourne School of Rural Health
2 University of Melbourne
Department of Information Systems
2
Background - Conduit RHAN

• CONDUIT A Collaborative Network and Data using
  Information technology
• A partnership between 3 local practices, the
  Goulburn Valley Division of General Practice and
  Goulburn Valley Health
• A technical pilot for a Rural Health Academic
  Network (RHAN) across the Goulburn Murray Valley
• RHAN CONDUIT Sharing clinical information for
  audit, research and clinical care

3
Data Quality - Addressing the Issues

• The need for good quality data (and data quality)
  is recognized
• BUT
• Are we doing enough?
• What about errors due to organizational, software
  or site configuration issues?
• Databases all exhibit an error rate
• What about poor system design?
• What about unplanned shut-downs, power faults?
• These issues are well known but rarely reported

4
Observational Study

• The data content at three practices from one
  Australian computer system was analyzed.
• Results are deliberately obfuscated to protect
  the vendor software supplier.
• An ODBC interface was written for the extraction
  of anonymized data (written informed consent).
• Tables, linkages key fields were identified.
• Data extracted covered Demographics, medical
  history, laboratory results medication
• The primary table linkage was via a practice
  identifier

5
Observational Study

• An XML Schema was generated covering the database
  tables and fields extracted.
• This was translated into a Document Object Model
  (DOM) using ADO.NET.
• This association allowed us to track errors in
  expected table and field associations.
• Multiple records per individual patient were
  identified by comparing standard record linkage
  fields.
• Data field values out-with normal bounds were
  also examined e.g. Blood Pressure.

6
Results

• In one practice, data corruption prevented
  operation of the interface. Subsequent analysis
  indicates that this may have been due to an
  un-planned shut-down during a database write
  operation. Other causes are possible.
• Rows of data in some tables had no corresponding
  master patient identifier (orphaned data)
• Prescriptions were recorded as issued, but it was
  impossible to tell who whom.
• This error was not limited to one table

7
Results

• Some patients had more than one patient
  identifier
• Problem was identified mainly in patients with
  complex histories
• Errors were present in 0.1 of patients
• This represents 10 patients / 1,000
• There were also clear data entry faults where
  laboratory results were recorded that were not
  compatible with life. These did not contribute to
  the above error rate and were not further
  examined.

8
Discussion and Conclusions

• This small study showed an error rate of 10/1,000
  patients even though data entry errors such as
  range checking were excluded.
• This error rate may be reasonable if the data is
  used for large-scale epidemiological research
• Is this error rate too high if data is to be used
  for clinical care purposes?
• Most practices use combinations of electronic and
  paper records. Some use all electronic (32.81).
  Is it safe to use combinations?
• Is a reluctance to embrace fully electronic
  solutions justified by the error rates reported?

1 Henderson J., Britt H., Miller G. Extent and
utilisation of computerisation in Australian
general practice MJA 2006 185 84-87
9
Discussion and Conclusions

• There is a clear trend towards the replacement of
  paper records with fully electronic records.
• This trend raises new issues
• If electronic patient information is pooled
  (integrated) one clinical misclassification /
  wrong diagnosis can spread to many systems.
• There is a limit to how effectively non-coded
  clinical data can be shared.
• Errors are not necessarily easy to spot.
• Modern DBMSs impose rules and procedures that
  can help minimise risk, but
• Older e.g. non-relational (pre-relational)
  databases should be avoided.
• There is no excuse for not validating user input
  e.g. BP 1140 / 80

10
Discussion and Conclusions

• Proposition - we need
• A ready assessment of the extent of corruption in
  existing systems
• To mandate fail-safe strategies in software and
  user interface design
• To employ strategies to monitor, assess and
  improve data quality as part of clinical
  governance

The Australian Commission for Safety Quality in
Health Care, Quality Improvement, Scotland,
to name (but not pick on!) two organizations
work hard to improve clinical outcomes, but are
there standards procedures for the management
governance of data quality in clinical
databases?...
11
Discussion and Conclusions

• Strategies need to address (amongst others)
• Training
• Data entry How do I record data accurately?
• Philosophical Why is it important?
• Guidelines for system selection highlighting
  capabilities of user interfaces but also system
  robustness.
• Is data validated?
• Is the data coded and to which standard?
• System Security
• Can procedures be circumvented?...

12
Conclusion

• With increasing reliance on electronic computer
  records, physical errors in clinical databases
  are an increasing problem.
• As database linkages increase, the clinical
  implications of a single database error can
  multiply.
• Quality assurance programmes need to recognize
  the importance of data in clinical care and
  assess data quality as a routine part of clinical
  governance.