Data and Evaluation Update

1
Data and Evaluation Update

• Nikki D. Bellamy, Ph.D.
• SAMHSA/CSAP
• February 14, 2006

2
Copies of the Presentation

• For the sake of trees
• Requests for an electronic copy of this
  presentation can be sent via email
  
• nikki.bellamy_at_samhsa.hhs.gov

3
New Contractor

• CSAP consolidated its data and evaluation
  contracts to form a new Data Coordination and
  Consolidation Center (DC3 or DCCC)
  
• Purpose to provide information technology and
  data management services to CSAP and its grantees
  
  

4
DC3 Contacts

• Where to send Data Submission?
• Laurie Brannon
• Constella Group, LLC 10401 Fernwood RoadSuite
  200Bethesda, MD 20817-1110ATTN SAMHSA
  Methamphetamine/Inhalant Data Submission
  
•  
• Laurie can be reached by phone [phone] or
• via email lbrannon_at_constellagroup.com

5

• Question regarding data integrity
• Virginia Mulkern, Ph.D.
• Phone [phone]
• ext. 2315
• Fax [phone]
• vmulkern_at_hsri.org

6
Reminder Data Submissions to DC3

• Baseline (Pre-test) forms by
• October 13, 2006
• Exit (Post-test) forms by
• January 15, 2007
• Follow-up forms by
• February 16, 2007
• We will send out reminders via email

7
Process Data

• During stoppage with contractor evaluation
  activities capacity building and other process
  measures were not completed
  
• Starting in May and June 2006 the DC3 will be
  contacting you to gather final information to
  complete an overall process measure that captures
  the three-year grant period

8
Data Snapshot for Preliminary Analysis

• Cohort II Methamphetamine/Inhalant Projects
• (Funding 2003-2006)
• 11 of 15 have provided data 10 included
• Those not providing data- 2 infrastructure, 1 on
  wrong form, 1produced no outcome data, 1 required
  detailed editing
  
• Data received in raw and electronic form
• As of 12/15/05 received 2331 instruments from
  youth and 57 adult in English
  

9
Key Issues In Data Processing

• Grantees went over 18 months with no data
  support
  
• Data were submitted on incorrect forms
• Participant identification numbers wrong or
  missing
  
• Grant year not consistently reported correctly

10
Initial Data Cleaning Protocol Key Features

• Recode out of range and missing values as
  missing
  
• Skip patterns assessed for consistency
• Clarified demographic values
• Recoded some variables for simpler preliminary
  analyses
  

11
Data Included in Preliminary Analyses

• Adult
• 57 baseline- intervention
• 21 with pre and post- test data
• 0 comparison group respondents

• Youth
• 961 baseline intervention
• 645 pre- and post-test
• 150 comparison baseline
• 92 pre- and post-test comparison

12
Percent of Youth Who Reported being Very
Truthful
13
Youth Retention Rates by Intervention and
Comparison Groups
14
Baseline comparison of Intervention and
Comparison Groups on AgeYouth
15
Percent of Hispanics Among Intervention and
Comparison GroupsYouth
16
Percent of Youth Intervention and Comparison
Group Who Report Smoking
17
Percent of Youth Intervention and Comparison
Groups Who Report Drinking Alcohol in the Past 30
Days
18
Percent of Youth Intervention and Comparison
Group Who Report Using Marijuana
19
Percent of Youth Intervention and Comparison
Group Who Report Moderate or Great Risk From
Smoking 1 Packs per day
20
Percent of Youth Intervention and Comparison
Group Who Report Moderate or Great Risk from
Smoking Marijuana Once or Twice
21
Percent of Youth Intervention and Comparison
Group Who Report Moderate or Great Risk from
Using Marijuana Regularly
22
Percent of Youth Intervention and Comparison
Group Who Report Moderate or Great Risk from
Use of Crystal Meth
23
Percent of Adults Who Report Being Very Truthful
24
Percent of Adult Participants Who Have Completed
At Least High School by Completion Status
25
Percent of Females Among Adult Participants by
Completion Status
26
Distribution of Adults by Hispanic Ethnicity
27
Drug Use Among Adults At Baseline
28
Changes In Adult Drug Use

• Only 2 adult smokers matched. Neither quit
  smoking on post test.
  
• Only 5 matched responses on alcohol consumption.
  None quit drinking at post-test
  
• No matched data on users of marijuana, cocaine,
  or the amphetamine or methamphetamine
  

29
Final Comments

• The data gathering process and quality of data
  collected have suffered from disruption in
  support across sites
  
• Nonetheless, there are some promising results
• Effect size cannot be determined
• Retention appears to be a general problem
• Data quality issues need to be addressed

30
Contacts

• Nikki Bellamy
• nikki.bellamy_at_samhsa.hhs.gov
• CSAP Project Officers
• Wilma Pinnock
• Patricia Sabry

31
Questions