Case Completeness and Data Accuracy in the National Program of Cancer Registries

1
Case Completeness and Data Accuracy in the
National Program of Cancer Registries

• KK Thoburn, CDC/NPCR Contractor
• RR German, M Lewis, P Nichols, F Ahmed, CDC
• J Jackson-Thompson, University of
  Missouri-Columbia

2007 Annual NAACCR Conference Detroit,
Michigan Tuesday, June 5, 2007
2
NPCR Cancer Surveillance System (NPCR-CSS)

• Established in 2000
• In collaboration with SEER, collects cancer
  incidence data for the entire US population
• NPCR-CSS data submissions are evaluated for case
  completeness (CC), percentage of
  death-certificate-only (DCO) cases, percentage of
  unknown/missing values for selected required data
  elements, and percentage of cases passing
  critical edits for data quality
• Measures incidence rates by various
  characteristics
• Data estimates CCR progress in meeting NPCR
  Program Standards

3
NPCR Technical Assistance and Audit Program
(NPCR-TAA)

• ORC Macro conducts NPCR-TAA
• Performs case completeness (CC) and data accuracy
  (DA) audits at the level of the hospital
  reporting to the CCR
• Uses sample of cases from selected facilities for
  reabstraction
• Lung and bronchus, colon and rectum, prostate,
  and female breast
• These primary sites represent over 50 of cases
  reported to the NPCR-CSS
• Provides technical assistance to CCRs

4
NPCR Annual Program Evaluation Instrument
(NPCR-APEI)

• Web-based instrument completed annually by all
  NPCR-funded CCRs
• Assessment of NPCR program objectives, registry
  operations, and data use

5
Present Study

• Usefulness of NPCR-CSS data depends on
  completeness of case reporting and accuracy of
  submitted information
• NPCR-CSS data are frequently used to measure
  incidence and survival
• Race, sex, age, subsite, and stage at diagnosis
• Present national-level study
• Summary analysis of NPCR-TAA information by
  primary site
• Covariate analysis of NPCR-TAA results with
  information from the NPCR-APEI, and CCR
  achievement of USCS publication and NAACCR
  certification standards

6
Methods

• NPCR-TAA audit results for 34 of 45
  NPCR-supported CCRs audited at the time of
  analysis were included
• 1998 through 2001 audit diagnosis years were
  included
• Overall average case completeness and data
  accuracy rates, and site-specific data accuracy
  rates were calculated
• DA rates calculated by site for each of the 13
  data audited data
• Average CC and site-specific DA rates were
  examined across the covariates
• Small-sample Students t tests (a 0.05)

7
Methods

• CC rates ()
• 100 - ( of missed cases / total of cases
  identified) x 100
• DA rates, overall and site-specific ()
• ( of data elements with no discrepancies /
  total of data elements reabstracted) x 100
• DA rates, audited data elements ()
• ( of reabstracted cases with no
  discrepancies on data element / total of cases
  reabstracted) x 100

8
Methods Sources of Variables

• NPCR-TAA
• CC and DA rates
• Primary site
• Audited data elements
• Source of missed cases
• USCS publication
• Met publication standards?
• NAACCR Web site
• Certified by NAACCR?
• Certification level

• NPCR-APEI
• Type of current funding?
• Ratio of FTE positions to central registry
  caseload?
• Ratio of CTRs to central registry caseload?
• Supplementary reporting sources reporting cases?
• Case-finding audits at reporting facilities?
• Reabstracting audits at reporting facilities?
• Annual report issued?

9
Results Case Completeness

• 34 CCRs, diagnosis years 1998 to 2001
• 41,521 lung and bronchus, colon and rectum,
  prostate, and female breast were identified
• 1,503 cases were missed
• Overall case completeness rate of 96.4

10
Distribution of Missed Cases
(N 1, 503)
Lung and bronchus
19.9 (n 299 )
Female breast
32.7 (n 491)
Colon and rectum
18.8 (n 283)
Prostate
28.6 (n 430)
11
Distribution of Missed Cases by Type of
Case-finding Source
Lung and bronchus
Colon and rectum
Female breast
Prostate
n 324
n 315
n 544
n 458
0.3
0
.
7

0.6
3
.
7

2.2
0.6
0.3
0.2
1.3
0.2
4
.6

3.1
29.0
45.8
40.6
43.2
52.0
53.9
55.6
62.0
Autopsy record
Radiation therapy log
Medical records disease index
Pathology report
Cytology report
Other
Counts include missed cases from more than one
case-finding source.
12
Site-Specific Data Accuracy
99.7
99.8
Gender
100.0
99.9
Lung and bronchus (n 2,448)
99.8
99.9
State of residence
99.7
Colon and rectum (n 2,063)
99.5
99.5
Prostate (n 2,289)
99.2
Date of birth
99.4
99.3
Female breast (n 3,210)
99.8
98.7
Behavior
99.9
98.9
99.4
97.4
Primary site
100.0
99.9
95.4
99.4
Laterality
99.4
99.1
98.4
97.8
Race
97.5
98.0
96.9
97.1
Sequence number
98.2
97.5
95.5
98.5
Diagnosis date
95.3
96.2
89.1
88.3
Histology
97.6
89.7
89.1
93.6
Subsite
100.0
81.2
89.7
Grade
93.9
89.9
85.9
83.5
84.3
SEER Summary Stage
92.0
93.5
0
10
20
30
40
50
60
70
80
90
100
Percentage of records containing no discrepancies
13
Covariate Analysis

• Ratio of FTEs and CTRs to CCR Caseload
• Higher CC and DA rates were found for the more
  well-staffed registries (not significant P gt
  .05).
• Higher DA rates were found for registries staffed
  with a greater number of CTRs (significant for
  colon and rectum, prostate)
• Higher CC rate was found for registries staffed
  with a greater number of CTRs (not significant P
  gt.05).

14
Covariate Analysis

• Supplementary Reporting Sources
• Higher CC rates were found for registries with
  pathology laboratories and/or radiation therapy
  centers facilities reporting (P lt .01).
• Higher site-specific DA rates were found for
  registries with pathology laboratories and/or
  radiation therapy centers facilities reporting
  (not significant P gt.05).

15
Covariate Analysis

• Met USCS Publication Standards/Achieved
• NAACCR Certification
• Higher CC and average site-specific DA rates were
  found for CCRs that achieved these 2 milestones
  (only colon and rectum significant P .02,
  USCS P .04, NAACCR certification)
• NAACCR-certified CCRs generally had higher data
  element-specific DA rates than non-certified CCRs

16
ConclusionsSummary Analysis of NPCR-TAA Data

• Underscores importance and effectiveness of
  conducting CC and DA audits at reporting
  hospitals
• Enables identification of general and
  site-specific case-finding and abstracting issues
• Demonstrates overall high accuracy and
  completeness of NPCR-CSS incidence data on
  cancers of the lung and bronchus, colon and
  rectum, prostate, and female breast
• Overall CC (96) higher than NPCR-CSS 24-month
  standard (95)
• Overall DA rate (95), and high DA rates found by
  demographic data elements and primary site lend
  confidence in incidence rates
• Provides guidance to users of the data

17
ConclusionsCovariate Analysis

• Shows that combining data from the NPCR-TAA and
  the NPCR-APEI provides additional valuable
  information that neither program can provide
  individually
• Offers perspective on how CCR operations effect
  case completeness and data quality
• Underscores importance of CCRs having
  well-trained staff having supplemental reporting
  sources and attaining compliance with national
  data standards

18
Conclusions

• NPCR funding and technical assistance help CCRs
  in developing and enhancing effective registry
  operationsespecially in areas such as staffing,
  training, and monitoring and in improving the
  completeness and quality of registry
• Present study demonstrates
• Positive outcome of enhanced registry operations
• Benefit of NPCR support for a high-quality,
  statewide, population-based CCR

19
Special Appreciation

• Arizona
• Arkansas
• Colorado
• District of Columbia
• Florida
• Georgia
• Idaho
• Illinois
• Kansas
• Maine
• Maryland
• Massachusetts
• Michigan
• Minnesota
• Missouri
• Montana
• Nebraska

• Nevada
• New Hampshire
• New York
• North Carolina
• North Dakota
• Ohio
• Oklahoma
• Oregon
• Pennsylvania
• Rhode Island
• Tennessee
• Texas
• Vermont
• Virginia
• Washington
• West Virginia
• Wisconsin