Providing Context for Understanding

1
Providing Context for Understanding

• The Data Life Cycle and the National Population
  Health Survey

E. Hamilton IASSIST 2005
2
With thanks to . . .
Chuck Humphrey Gary Catlin Mario Bedard France
Bilocq Vicki Crompton Michael Sivyer The NPHS and
CCHS project teams
and special thanks to those valiant survey
respondents!
3
Research Project

• Research Objective
• to discover evidence of use of the National
  Population Health Survey

• Methodology
• Using framework of data life cycle, searched
  established peer-reviewed databases as well as
  grey literature sources for evidence of data use
  and knowledge transfer relating to the NPHS

4
What Lies Inside the Black Box?
Revise Recycle Retire Re-purpose Respect
Genesis
Maturity
?
5
Inside the Black Box
Research Development
Questionnaire Development
User Consultation/ Content Determination
Data Collection
Data Gap Analysis
Data Processing
Data Quality Measurement
Evaluation Revision
Dissemination
User guide and data file release
6
The Data Life Cycle

• A. Data Gap Analysis
• Documenting perceived needs and survey
  expectations
• B. Administration and Planning
• Documenting the process
• C. NPHS Use
• gt60 theses gt400 articles in gt150 journals
• D. Revision and Gap Analysis
• Documenting changes

7
NPHS Background

• A national longitudinal health survey launched in
  1993/4 to fill a significant data gap in health
  information
• Intended to provide more comprehensive
  information on population health statusa key
  component of Canadas health information
  infrastructure
• Dissemination of data and information were deemed
  a critical part of the survey plan at the outset

8
NPHS Administration

• Household survey general and health components
• Complex survey design using strata and based on
  LFS frame Respondents to be followed for 20
  years
• Institutional survey
• Northern survey

Cycle 1 Household survey in the field in June 1994
9
NPHS Data Products
10
NPHS User Guide
Background Objectives Survey Content Sample
Design Data Collection Data Processing Data
Quality Guidelines for Tabulation, Analysis
Release Approximate Sampling Variability
Tables Weighting File Usage
Plus Questionnaire Record layout (General and
Health) Data Dictionary Derived and Grouped
variables CV Tables list
11
Life Cycle Approach Revealed More
Interviewer Training manuals Self-study guides
for interviewers Expert Group reports Questionnair
e Focus group reports Treasury Board
documents Comparisons with other surveys Jacknife
vs bootstrap Coding concordances
And so much more!
12
Training Guide Example
DEP_Q26EE1A
During the last 12 months, did EXPERIENCE C
happen to you? R. Well, yes, there was one
night, we were drinking and I just
Q. At this point the respondent stops talking and
you see a tear come to her eyes. What do you do?
13
Training Guide Example

• Exercise-Minimize Non-Response Tactics Sad/Upset
  Respondent
• Stop for a minute. Be responsive to the
  respondent in a supportive way give the
  respondent a chance to collect themselves, and
  help them get on track with the interview.
• That must have been very upsetting..offer to
  take a break from the interview.
• Ask if the respondent is able to continue with
  the interview.
• If not, offer to continue the interview at a
  later date.

14
Researcher Needs

• Why did wording of question change over time?
• Why this particular question? Why not another
  wording?
• Why this content? Why not broader/ more specific?
• What scales were adopted or modified for use in
  this survey?

15
Researcher Needs

• What training was given to interviewers?
• How were difficult issues resolved? What was
  consideredand what was rejected?
• What is the relationship between this survey and
  other surveys of similar subject matter?
• How has the sample changed (over surveys and over
  time)?
• What should I know to analyze this survey
  appropriately?

16
Observations

• There is a rich array of documentation throughout
  the survey life cycle
• This documentation should be identified and
  bundled intellectually with the data
• Surveys with a repetitive component require
  special preservation efforts for this rich
  documentation

17
It is important!
Senator Lavoie-Roux My questions have already
been asked more or less by my colleagues because
I was wondering how the data were collected
To me it was really an important issue to know
to what extent your statistics could be trustedI
dont say that in a negative waysince the
methods used for data collection are fairly weak.
With the methods that you use, do you
believe that we can trust the data? Are you sure
that it is accurate?
18
There is more to come

• Thank you for your attention!