Selection of an LIS: Pitfalls and Recommendations

1
Selection of an LIS Pitfalls and Recommendations

• Association for Pathology Informatics (API)
  Course
• September 9, 2007
• Walter H. Henricks, M.D.
• Director, Center for Pathology Informatics
• Cleveland Clinic

2
Learning Objectives

• Develop a structured approach to system
  evaluation
• List key elements of an RFP or similar needs
  assessment for system selection
• Describe important elements of contracts
• Anticipate pitfalls in the process of evaluating
  LISs and vendors

3
Structured Approach to System Evaluation

• RFP development and evaluation
• Vendor/system demonstrations
• Client reference checks
• Contract negotiation

4
Preparatory Steps Project Team Designation

• Composition depends on scope and objectives of
  the new system, but key point is that all
  affected groups (stakeholders) are represented
  and provide input
• Laboratory
• Clerical
• Billing
• Sales and Marketing
• Pathologists and lab directors
• Information technology

5
Development of Specifications and Requirements

• Understand project scope and type of system
  contemplated
• Define and map out elements of workflow and
  operations
• Translate workflow elements into system
  requirements and expectations
• Convert functional requirements into
  specification statements or yes/no questions

6
LIS Selection ProcessDue Diligence

• RFI and RFP
• Vendor presentation
• Site references
• Site visits
• Vendor financial condition
• Litigation search

• Vendor personnel
• Concurrent vendor projects
• Vendor use of third parties
• User group
• Trial use
• Contract

7
Preliminary Vendor Evaluation

• Initial criteria
• No./size/complexity of installations
• Prior experience with vendor (could be a or -)
• Hot button issues for your site based on
  reasons for project
• CAP Today annual system review series on
  different system types based on vendor survey
• Request for Information (RFI) questionnaire
  based on initial criteria to screen vendors

8
Structured Approach to System Evaluation

• RFP development and evaluation
• Vendor/system demonstrations
• Client reference checks
• Contract negotiation

9
Request for Proposal (RFP)

• A document that
• specifies an organizations requirements for
  particular products and/or services
• invites vendors to submit bids to provide such
  products and/or services to the organization
• outlines the conditions for the bidding process
• Purposes (Elevitch and Aller)
• Describe your lab and institution to the vendor
• Obtain vendor responses to your requirements
• Obtain complete information about a vendors
  capabilities
• Set the rules for bidding, acceptance, and
  negotiation

10
Request for Proposal (RFP)

• RFP asks vendor to spell out
• if specified system capabilities are present
• other attributes of system, e.g. hardware,
  maintenance, interfaces
• corporate information
• pricing
• Major component of RFP is detailed query about
  required functionality and features
• RFP responses become part of final contract

11
RFPs Other Points

• Process of RFP development can be valuable
  exercise in practice, some consider RFPs
  misleading, impractical, or constrictive
• Main value of RFP is adding structure and
  documentation to the process and ensuring that
  all information is gathered
• RFP can be particularly useful for discerning
  red flags that may eliminate a vendor.

12
RFP Content Outline (example)

• Management Summary
• Functional scope of future system
• Administrative information and instructions for
  responding
• Core functionality questions/checklists
• Technical architecture description

• Interface support
• Hardware and software support
• Implementation services
• Training services
• Pricing
• Vendor information

13
RFP I. Management Summary

• Your goals and objectives for the system
• Brief overview of desired functionality
• Background about your site and the context in
  which the new system will be deployed
• Scope of laboratory services
• Multiple labs on single LIS
• Statistics related to test volumes, patient
  volumes, beds, etc.

14
RFP II. Functional Scope of Future System
(optional can spell out in the specific
sections)

• Narrative overview of elements desired or scope
  and intended uses of new system, e.g.
• AP, CP, molecular, microbiology, BB, outreach
• Interfaced systems (e.g. electronic medical
  record)
• Specific constraints of your technical
  environment
• Emphasis of most important features needed
• Highlights of potential, specific challenges or
  considerations in your environment

15
RFP III. Administrative Information and
Instructions for Response

• Vendor commitment and conditions for bidding
• Confidentiality
• Representation of fact
• Inclusion of response in forthcoming contract
• Selection process
• Due dates and selection timeline
• Basis for evaluating vendor (criteria here need
  not be all-inclusive)
• Format for RFP response packet
• Electronic AND printed with tabbed sections

16
RFP IV. Core Functionality Checklist

• Choices for each item Yes/No, beta, future,
  custom
• Opportunity for vendor to add comments
• Potential sections and sub-sections
• Global functions
• Pre-analytic functions
• Analytic functions
• Post-analytic functions

17
LIS Functionality Checklist Global Functions

• Test definitions
• Worksheets/worklists
• Specimen logs (AP)
• Interfaces EMR, ADT, instrument, billing
• Specimen tracking
• Management reporting and related tools
• QC handling
• Data security and audit trails
• Billing
• Demographic data management

18
LIS Functionality Checklist Global Functions
(contd.)

• Areas with special requirements
• Blood bank / donor center
• Microbiology
• Molecular
• Extended features SLAMs
• Outreach program support (e.g. client-oriented
  view of data, billing, customer service)
• Web portal
• Digital imaging and/or document imaging
• Voice recognition

19
LIS Functionality Checklist Pre-analytic

• Patient identification
• Specimen identification/accession numbers
• Label generation and specifications
• Phlebotomy lists
• Test order management
• Specimen receipt and tracking

20
LIS Functionality Checklist Analytic

• Reflex testing
• Rules/calculations
• Autoverification
• Histology protocols
• Result entry and verification
• Electronic signature
• Templates and coded text/reports/comments
• Special procedure handling (e.g. molecular, flow)
• Cytology requirements (e.g. Pap smears)

21
LIS Functionality Checklist Post-analytic

• Report/result distribution modalities
• Printing batch, on-demand
• Electronic and web-based
• Fax
• Report formatting
• Amendments, corrected reports, addenda
• Results viewing and inquiry

22
RFP V. Technical Architecture

• Vendor recommendations for your site based on
  information that you supply
• Ask for diagram(s) that shows system components
  and connectivity
• Configurations and third parties for hardware,
  software, printing
• Capacity limits
• Security features
• Database structure, maintenance
• Test environment(s)

23
RFP VI.-VII. Vendor Support

• Application Interfaces (Foreign Systems)
• Existing interfaces types, systems, and
  versions
• Time requirements for implementation of new
  interface
• Hardware and software
• Installation process
• Ongoing support problem resolution, escalation,
  definition of service levels and vendor
  availability or response times
• Processes for bugs, enhancements, and customs
• Documentation
• Vendor requirements and expectations for your site

24
RFP VIII.-IX. Implementation and Training
Services

• Implementation
• Project staffing
• Project management methods
• Time estimates for implementation phases
• Client requirements staff, time, expertise
• Training
• Client resources required
• Users and system managers
• Documentation

25
RFP X. Itemized Components of Pricing

• Hardware
• Software
• Application
• OS, database, 3rd party
• Interfaces
• Implementation services
• Training

• Recurring maintenance fees
• 24 hour support
• Warranty
• Options lease, purchase, transaction-based,
  subscription-based

26
RFP XI. Vendor Information

• Corporate profile
• Company staffing sales vs. technical support
  personnel
• Research and development commitment
• User groups / user input
• Client references and contacts

27
Evaluating RFP Responses

• Traditional method is numerical
• Assign weighted value to each item based on
  importance
• Assign score to each answer (e.g. yes3, beta2)
• Multiply weight x score weighted score
• Total the weighted scores for all answers
• Choose the system/vendor with highest score
• Numerical method may be constraining and
  misleading if used as sole or primary criterion.

28
Evaluating RFP Responses

• Alternative method is less strictly numerical
• Tabulate all non-yes responses by
  no/beta/future/custom for each vendor
• Decide if any of the absent functionality is a
  deal-breaker requirement
• Determine which of the absent features are the
  biggest negatives for your environment
• Emphasize these in future evaluation of vendor
• Decide which vendors negative profile is best
  suited to your site

29
Structured Approach to System Evaluation

• RFP development and evaluation
• Vendor/system demonstrations
• Client reference checks
• Contract negotiation

30
Vendor/System Demonstrations

• Philosophy demonstration is not just a
  30-minute overview but rather a crucial
  opportunity for the vendor to show in-depth how
  features of the system will meet your business
  needs.
• The most effective demonstrations result from
  considerable preparation on the part of the
  prospective client and the vendor.

31
Vendor Demonstrations Script

• Develop detailed script for vendor based on
• Your highest priority functional requirements
• Scenarios that are challenges in your current
  environment (requires knowledge of current
  operations)
• Items from the RFP that require clarification and
  visualization
• Permit unstructured time for special features and
  new development
• Ask vendor to show examples of printed materials
  related to scripted scenarios
• Reports
• Requisitions
• Labels
• Packing lists (outreach)

32
Vendor Demonstrations Preparation

• Emphasize that only currently available
  functionality should be shown
• If beta or future features are shown, they must
  be clearly noted as such
• In advance, provide to vendor examples of
  (anonymized) documents you currently use such as
  different report types, forms, requisitions, and
  others that relate to the demo
• Prepare Internet access and have backup
  capabilities for vendor demo equipment (e.g. PC,
  projector, printer, etc.)

33
Vendor Demonstrations Evaluation

• Convert script into functionality checklist and
  comment form for those attending demo
• Assess subjective aspects evaluate and rate
  (e.g. excellent, acceptable, poor)
• Ease of use
• Special features not easily covered by RFP (e.g.
  navigability)
• User interface

34
Structured Approach to System Evaluation

• RFP development and evaluation
• Vendor/system demonstrations
• Client reference checks
• Contract negotiation

35
Client Reference Calls/Site Visits

• Best site is one similar to you in case volume
  and test mix to you and whose implementation is
  similar to your vision
• LIS(s) used, including versions and modules
• AP, CP, Blood Bank /- Donor Center, Micro,
  outreach
• Interfaces orders, results, EMR, other LISs
  confirm version numbers of interfaced systems
• Special features imaging, web, voice, etc.

36
Client Reference Calls/Site Visits Things to
Assess

• Satisfaction with vendor
• Satisfaction with system
• Interface implementation
• Unexpected costs
• Installation and implementation times
• User training experience
• Surprises unpleasant and pleasant
• Lessons learned and things to do differently

37
Other Factors to Evaluate for Lab System Vendors

• Vendor as partner
• Commitment to product
• Experience and expertise in area(s) of interest
• Other product lines (potential competition for
  vendor resources)
• Expertise and experience with interfaces
• Number of staff, technical vs. marketing
• Financial stability
• Technology platform

38
Structured Approach to System Evaluation

• RFP development and evaluation
• Vendor/system demonstrations
• Client reference checks
• Contract negotiation

39
Contracts for Information SystemsComponents

• Definitions, deliverables, and agreements
• Responsibilities of vendor and client
• Cost itemization and payment schedule (payment
  not all up front!)
• Options for recourse and termination
• Escrow right to obtain software code if the
  vendor becomes insolvent

40
Contract Basics

• Definitions, deliverables, timelines, and
  agreements
• Responsibilities of vendor and client
• Payment milestones
• Options for recourse and termination
• Escrow right to obtain source code if vendor
  becomes insolvent

41
Possible Payment Milestones in LIS Contracts

• Contract signing
• Hardware delivery
• Software installation
• Successful testing and validation of system
  modules and/or interfaces
• Go-live
• Post go-live 60-90 days of successful function

42
Contract Sections (example)

• Deliverables and definition of terms
• Software license terms
• Documentation
• Pricing and payment terms
• Price protection
• Ownership
• Confidentiality
• Site preparation

• Delivery and Installation
• Acceptance terms
• Training
• Warranty and maintenance
• Intellectual property rights indemnity
• Remedies
• Change in scope
• Termination
• Other general provisions

43
System Contracts Price Components

• Hardware
• Software
• Licenses
• Interfaces
• Historical data conversion
• Standard
• Custom

• Implementation services
• Training
• Annual support

44
System Contracts Hardware Items for Pricing

• Database server
• Print server
• Fax server
• Interface server
• Web server
• Image server
• Thin client server

• PCs
• Networked printers
• Barcode label printers
• Cassette labelers
• Digital cameras
• Bar code scanners
• Any necessary network infrastructure upgrade

45
System Contracts Software Licenses for Pricing

• Application
• per seat vs. concurrent users vs. other measure
• Database management system
• Thin client
• Network/server
• Fax software

• Web software
• Voice recognition software
• Imaging software
• Other add-ons and third party licenses
• Annual Maintenance fees

46
System Contracts Interfaces for Pricing

• Order entry
• Results reporting
• Patient management (ADT)
• Web portal system
• Other LISs (Ref lab)

• Automation system
• Analytical instruments
• Cassette labeler
• Voice recognition
• Billing system(s)
• Other systems

47
Application Service Provider (ASP) Model

• Contractual arrangement in which a provider
  delivers a software application or suite of
  services to a client over a network, typically
  the Internet outsourcing
• Hardware, software, security, upgrades entirely
  managed by vendor
• Fixed subscription fees, either per transaction
  or flat rate

48
Service Level Agreement (SLA)

• Contract that defines the technical and business
  parameters of the relationship between IT vendor
  and client (esp. ASP model)
• Key terms
• Responsibility extent of services provided
• Performance system uptime and response time
• Remediation how problems are addressed
• Escalation process and vendor response times for
  system problems
• Quantifiable metrics system uptime

49
Potential Pitfalls in System Contracts

• Definition of dates as they relate to payment
  milestones and contract term When does the
  clock start?
• Delivery Date, Acceptance Date, Live Date,
  Effective Date
• Service Level Agreement for ASP hosting may
  require a separate agreement from your software
  agreement data center support may be a different
  organization from that providing software

50
Potential Pitfalls in System Contracts

• License-based pricing
• Server licenses
• End-user licenses
• Number of users per client site
• Additional charges related to ASP hosting fees
• Transaction-based pricing
• Minimum charge
• Definition of transaction some order/result
  types may count as more than one transaction
• Additional fees to establish a new client site

51
Pitfalls in LIS System/Vendor Selection

• Forgetting that an RFP also describes and
  characterizes your lab to potential vendors
• Over-reliance on any one aspect of the RFP
  (unless a drop dead issue in your lab)
• Failure to incorporate RFP responses into
  contract
• Inadequate preparation for vendor demonstrations
  and client reference checks

52
Pitfalls in LIS System/Vendor Selection(cont.)

• Making allowances (or requiring) substantial
  system customizations as a way to meet your needs
• Overlooking potentially obscure elements of
  pricing, e.g. database licenses, third party
  software
• Falling in love with bells and whistles at the
  expense of meat and potatoes
• Failure to assess the vendor firm as a
  prospective partner for your organization

53
System Selection and Implementation Summary

• RFP preparation and evaluation of vendor
  responses require commitment, effort, and time.
• Prepare well to get the most from vendor
  demonstrations and client reference calls.
• Recognize that many potential pitfalls lurk in
  the process, and it may be up to you to discover
  and to prevent them.