Conducting HIV Treatment Research in Resource Poor Settings

1
Conducting HIV Treatment Research in Resource
Poor Settings

• October 20-21, 2003
• The Pier 5 Hotel
• Baltimore, MD

2
Workshop 8Data Management (DM)

• Malathi Ram, Ph.D.
• Department of International Health
• Johns Hopkins Bloomberg School of Public Health
• Baltimore, MD

3
Practical Considerations in Resource Poor Settings

• Local IT Infrastructure
• Use / availability of equipment
• Networking facilities
• Internet connectivity
• Hardware / software usage
• Other Infrastructure
• Power Lines
• Phone Lines
• Power Outages

4
Practical Considerations (Continued)

• Local Culture
• Political situation
• Corruption
• Power structure
• Work Schedule and Ethics
• Days/week worked
• Leave structure
• Holiday structure
• Personnel and Training
• Hiring local personnel
• Amount and type IT experience
• Experience in research methodology
• Training in Ethics and confidentiality

5
General Precautions and Safeguards for Database
Operations

• Study Leadership that is sensitive to need for
  data security
• Staff experienced in the operation of a database
  and in protecting it against loss or misuse
• Signed assurance from each employee that he/she
  understands the safeguards and precautions to be
  followed and the consequences of a willful
  disregard of them
• Periodic staff meetings to remind database
  personnel of required operating procedures and
  safeguards
• Periodic review of required operating procedures
  and established safeguards by study leaders
• Monitoring of adherence to precautions and
  safeguards via periodic on-site checks

Source Meinert, C.L. 1986
6
Patient Confidentiality Safeguards

• Data flow procedures from clinic to data center
  that exclude transmission of patient identifying
  information
• Electronic storage of patient identifying
  information in encrypted form or separately
  stored from other files
• Physical separation of pages containing patient
  identifying or confidential information from
  other pages of data forms
• Proscription against distribution of data
  listings that contain patient identifying
  information
• Proscription against use of patient identifying
  information in any published data listing
• Secure procedures for disposing of computer
  output from aborted runs that contain patient
  identifying information
• Denial of access to any patient record stored in
  the data center to persons outside the center
  without the written consent of the patient

Source Meinert, C.L. 1986
7
Safeguards against misuse

• Limit the number of persons in the data center
  who have access to the original study forms or
  any related data file, especially those
  containing patient identifying information
• Restrict access to the analysis computer files
  containing study results through use of passwords
  or other means
• Proscribe release of any data listing or files
  without approval of the study leadership
  committee
• File completed study forms, data storage devices
  such as disks or CDs in an attended, locked area

Source Meinert, C.L. 1986
8
Loss Safeguards

• Maintain a duplicate file of the original study
  records (eg. by requiring clinics to maintain a
  copy of forms and records sent to the data
  center)
• Microfilm original data forms, computer listings,
  study manuals, meeting minutes, consent forms,
  etc, for storage in a secure off-site location
• Establish and maintain a series of backup disks
  for the analysis database for restoration in the
  event of system malfunction

Source Meinert, C.L. 1986
9
Loss Safeguards (Continued)

• Store copies of backup disks of the main database
  in an off-site fireproof location
• Establish strict guidelines for access to backup
  disks to avoid unauthorized use in restoration
  efforts
• Maintain backup disks of all essential programs
  used for editing, inventory, storage, retrieval
  and analysis of study data, as well as programs
  used for the operating system

10
Choice of Data Capture Method

• Initial costs New hardware and software,
  programming, DM personnel, training,
  documentation (SOPs), validation of all new
  interfaces, and security
• Ongoing costs Training, maintenance,
  communications, backups, and ongoing validation
• Accuracy Similarity of data to source data, ease
  of error identification
• Speed Set-up time, lag times, rapidity of entry,
  accessibility of sponsor/PI to data, potential
  for integrated validation

Source Rondel, R.K. et al. 2000
11
Choice of Data Capture Method

• Security Patient anonymity, confidentiality, and
  encryption of data
• Flexibility Simplicity, adaptability to changing
  requirements/environments, compatibility to
  existing systems, reliability
• Regulatory GCP/ICH requirements to be
  maintained, SOPs, Audit trails, documentation

12
Elements of CRFs I

• Content Do you need to collect it?
• Study objectives and endpoints (Ideal to prepare
  analysis plan prior to designing CRFs)
• Use of standard definitions terminology
• Objective Vs Subjective measures
• Precision of data item (Eg Age)
• Separating clinical care data from study data
• No ambiguity
• Use tested questions from other studies where
  available

Source Rondel, R.K., et al. 2000
13
Elements of CRFs - II

• Presentation Are you asking the question
  correctly?
• Type of Response expected
• Open (text, numeric, alpha numeric)
• Closed (binary and multiple choices)
• Combination (extends the range of closed format
  by the addition of an open format, eg. Others,
  specify)
• Analogue scales (alternative rating response, eg.
  Perceptions and feelings)
• Use of format familiar to the form filler to
  reduce errors
• - dd/mm/yy in Europe, South Asia, and parts of
  Canada
• mm/dd/yy in USA and parts of Canada
• Yy/mm/dd in Scandinavia and South Africa
• Use simple and unambiguous wording (avoid passive
  voice, double negatives, eliminate unnecessary
  phrases, avoid leading questions, avoid compound
  questions)

14
Elements of CRFs - III

• Methodology how to minimize/avoid problems that
  users have?
• Format use of type face, type size, case,
  spacing that enhances readability
• Identifiers on each page
• Pagination (Eg. 1 of 5, 2 of 5.)
• Version number of CRF in header / footer
• Units next to measurement or open response
• Flow of questions and skip patterns
• Clear and concise instructions
• Avoid asking same information on multiple CRFs

15
CRF Development Process - I

• Safeguards against errors of Omission
• Allow adequate time for developing and testing
  forms before starting data collection
• Solicit content advice and inputs from persons
  not directly involved in the development process
• Review data forms used in similar trials
• Ask persons not directly involved in the
  development process to review proposed data forms
  for deficiencies
• Test data forms under actual study conditions
  before use in the study

Source Meinert, C.L. 1986
16
CRF Development Process - II

• Safeguards against errors of Commission
• Distinguish between data needed for patient care
  and those needed to address the objectives of the
  trial
• Make certain every data item scheduled for
  collection is of direct relevance to achieving a
  stated aim or objective of the trial
• Establish an appropriate set of review and
  approval procedures in order for new items to be
  added to existing data forms

17
Data Quality

• The following two expressions are relevant to
  data Quality
• Garbage in, garbage out
• If inaccurate data is entered, only inaccurate
  information will result.
• Pay now or pay more later
• Prevention is usually more economical and less
  painful than cure.

Siragusa, T.J. 2001
18
Data Quality Characteristics

• The right data The data I need
• With Completeness All the data I need
• In the right context Whose meaning I know
• With the right accuracy I can trust and rely on
• Without redundancy I have a single version
• In the right format I can use it easily
• At the right time When I need it
• At the right place Where I need it
• For the right purpose So I can accomplish my
  objectives

Source English, L. 2000
19
Quality Assurance

• Definition
• Any procedure, technique, or method carried out
  during the trial, that maintains or enhances the
  reliability, reproducibility, or accuracy of the
  data from the trial.
• Examples
• Repeat readings of ECGs
• Data checks for missing, inconsistent, or outlier
  values
• Independent reprogramming of an analysis
  procedure
• Double data entry
• Duplicate lab determinations
• Special committee to code cause of death

Source Meinert, C.L. 1998
20
Schemes for Quality Control

• Fixed time
• Repeat measurements by the same or different
  person during an examination
• Aliquot determination in the same or different
  runs
• Replicate readings by the same individual within
  a short period of time or by two different
  individuals at the same time
• Over time
• Periodic submission of masked laboratory samples
  containing a known or fixed concentration of a
  substance
• Resubmission of previously read records to the
  same individual or reading center for rereading

21
Quality Control Credos and Requirements

• Credos
• To err is human
• No one purposely sets out to collect poor quality
  data
• Data that are collected without ongoing quality
  checks are best left uncollected
• The only way to have any assurance regarding data
  quality is to check, check, and check
• Perfection is impossible
• Quality control is everyones responsibility
• Requirements
• Quality conscious staff
• Timely data flow from clinic to processing center
• Expeditious data processing
• Computer hardware and software
• Organizational structure for implementing
  correction procedures

22
Quality Control Aids

• Aids
• Reference handbooks and manuals
• Standardized equipment and procedures
• Tested data forms
• Trained and certified data collectors
• Numbered policy and procedure memos
• Experienced clinic coordinators
• Site visits
• Conference phone calls, meetings and video
  conferences
• On-site data entry

23
Quality Control for Paper-based Data Collection
and Entry

• Forms should be thoroughly checked for
  deficiencies at the time of completion and prior
  to data entry
• Data forms should take the shortest time route to
  the entry site
• All data should be entered as they appear on the
  forms
• Entries should be made directly from the form
• All items on a form should be keyed at the same
  time
• Data entries should be checked for accuracy

24
General Edit Rules

• Computer checks are preferable to hand checks
• Edit queries should be directed to the persons
  responsible for data collection
• Changes made to a data file as a result of edit
  queries should be documented
• Entries in the electronic file with outstanding
  edit queries should be flagged

25
Types of Edit Checks

• Improper record linkage
• Unanswered items
• Impossible answers
• Inconsistent information (within or across forms)
• Abnormal or outlier values
• Suspicious changes from one exam to the next
• Inadmissible codes
• Uncertified technician
• Improper treatment or protocol violation

26
Performance Monitoring

• Definition
• Any method of summarizing data during the course
  of the trial, that is designed to detect
  deficiencies in the performance of specific
  activities in the trial
• Examples
• Comparison of actual recruitment vs recruitment
  goals
• Count of missed exams over time by clinic
• Counts of dropouts over time by clinic
• Counts of treatment protocol departures over time
  by clinic
• Data entry error rates over time by operator

27
Performance Monitoring

• Purpose
• Provides descriptive data on clinic performance
• Provides measures of relative standing of clinics
• Facilitates the identification of practices that
  may need correction
• Provides the database to support corrective
  actions taken

28
Management

• Functions
• Leadership
• Direction
• Decision making
• Delineation of functions
• Delegation of responsibilities
• Communication
• Mistakes
• Failure to designate who is in charge
• Delegation of responsibility without authority
• Overlapping responsibilities
• Overlooking areas of responsibility
• Ill defined communication channels
• Undefined decision making structure

Source Meinert, C.L. 2000
29
Result of Faulty Management

• Poor quality data
• Staff dissatisfaction or indifference
• High staff turnover
• Conflicting decisions, false starts, wasted
  efforts
• Inefficiency

30
Elements of Misconduct in Science as Defined by
the National Academy of Sciences

• Questionable Research Practices
• Failing to retain study data for a reasonable
  period of time
• Inadequate research records
• Refusing to allow reasonable access to data by
  others
• Using inappropriate statistical methods to
  enhance significance of findings
• Exploiting or inadequately supervising research
  subordinates
• Naming authors without regard to significant
  contribution to the research reported

Source National Science Foundation, 1991.
31
Elements of Misconduct in Science as Defined by
the NAS (Continued.)

• Misconduct in Science
• Data fabrication (making up data)
• Data falsification (changing data values)
• Plagiarism
• Other Misconduct
• Sexual and other harassment
• Misuse of funds
• Gross negligence
• Vandalism
• Violation of research regulations
• Conflict of interest

Source National Science Foundation, 1991.
32
Causes of Misconduct

• Academic pressure to publish
• Financial gain
• Professional vanity
• Lack of understanding of the research process
• Lack of sufficient technical and ethical training
  to all study personnel
• Undue pressure to meet performance guidelines
• Inadequate supervision or lack of supervision
• Overworked personnel
• Bottomline
• Documentation, organization, and quality control
  procedures for data are vital in both review of
  allegations and prevention of misconduct.

Source Piantadosi, S. 1997
33
Data Management is a Horizontal Function

• In the course of performing data management
  activities, the DM staff interacts with
• - Clinical staff
• - Laboratory staff
• - Pharmacy staff
• - Other study staff, such as nurses or
  counselors
• Some important qualities that the DM staff
  should have are
• - Detail oriented
• - Technical experience
• - Sense of ethics
• - Sense of team work

34
References

• Meinert, C.L. 1986. Clinical Trials Design,
  Conduct, and Analysis, Oxford University Press,
  NY
• Rondel, R.K. et al. 2000 Clinical Data
  Management, 2nd edition, John Wiley Sons, Ltd,
  p. 84
• National Science Foundation, 1991. Misconduct in
  Science and Engineering Research final rule.
  Fed. Reg. 56 (May 14) 22286-22290
• Sitagusa, T.J. 2001. Collaborate for Better Data
  Quality. DM Review, Published in DM Direct
• English, L. 2000. Plain English on Data Quality
  Information Quality Management The Next
  Frontier. DM Review, Published in DM Direct
• Piantadosi, S. 1997. Chapter 18 in Clinical
  Trials, John Wiley Sons, Ltd, pp 432-469