Governor

1
Governors Prevention Initiative for Youth
Evaluation

• Jane A. Ungemack, Dr.P.H.
• Evaluator
• University of Connecticut Health Center

2
Evaluation

• Systematic efforts to collect and use
  information
• Document program implementation
• Describe target populations/participants
• Inform and improve program performance
• Access program effectiveness
• Increase accountability
• Increase understanding

3
Governors Prevention Initiative for Youth

• Long-term Goal
• Reduce substance use among adolescents
• Intermediate Goal
• Reduce risk factors and increase protective
  factors for substance use in the individual,
  peer, family, school and community domains
• Target Group
• 12-17 year old youth

4
Evaluation Framework Overview
5
Evaluation Approach

• Process evaluation
• documents program implementation and activities
• Outcome evaluation
• assesses program effects or impacts

6
Capacity-Building for Evaluation

• Science-based approach
• Evaluation and assessment as integral parts of
  the program design
• Community-level
• Program-level

7
Evaluation Team Relationship with Grantees

• Training and technical assistance
• Instruments and administration protocols
• Consultation and collaboration
• Statewide coordination

8
Grantee Responsibilities

• Develop a program plan based on the logic model
• Specify measurable program objectives
• Cooperate and collaborate with UConn Evaluation
  Team
• Coordinate community survey
• Collect and submit process data
• Collect and submit outcome data
• Commit time and effort to evaluation activities

9
Assessing Community-Level Outcomes CSAP
Requirements

• School survey
• Use of core substance use, risk and protective
  factor measures

10
Community-Level Assessment School Survey

• Mandated
• 7th-10th grade students
• Representative sample (minimum n500 125/grade
  level)
• Year 1 and Year 3
• First administration February-April, 2000

11
School Survey

• Self-administered during a classroom period
• Anonymous and confidential
• Parental consents
• Sampling, instrument and administration protocols
  provided by UConn Evaluation Team

12
School Survey

• Measures
• Demographic characteristics
• Lifetime and current use of ATOD
• Risk and protective factors
• Limited community-specific items

13
School Survey

• Grantee responsibilities
• Planning/coordination with UConn Evaluation Team
• Planning/coordination with school personnel
• Instrument duplication
• Data cleaning
• Data entry
• Analysis

14
Assessing Program-Level Outcomes CSAP
Requirements

• Select a minimum of three programs (for each of
  three domains)
• Measure program outcomes using core measures
• Include a sufficient sample size for analysis
• Collect pre- and post-test data

15
Program-Level Evaluation

• Process evaluation
• Document program implementation and activities
• Outcome evaluation
• Assess program outcomes

16
Program-Level Evaluation Process Evaluation

• Each program will be responsible for reporting
• Prevention strategies
• Types of activities
• Dosage
• Number served
• Participant characteristics (age, gender,
  race/ethnicity, etc.).

17
Process Evaluation

• Minimum Data Set (MDS)
• Instruments, protocols, and training provided by
  the UConn Evaluation Team

18
Program-Level EvaluationOutcome Evaluation

• Based on the logic model, identify measurable
  objectives that you will address in your program
• Program objectives should be selected from one or
  more of the risk/protective factors included in
  the RFP list of Connecticut Intermediate Outcomes

19
Outcome Evaluation

• UConn Evaluation Team staff will work with each
  grantee to finalize program-specific objectives
  and measures
• All grantees will be asked to participate in a
  pre/post-test assessment of age-eligible
  participants as appropriate

20
Outcome Evaluation

• Pre- and Post-Test Assessments
• Youth participation will be voluntary
• Confidential
• Informed consents
• Standardized instrument plus optional
  program-specific items
• Minimum sample size 50

21
Considerations for Estimating Evaluation Costs

• Personnel (.25 FTE minimum recommended)
• Computer equipment
• Photocopying
• Office supplies
• Data collection and cleaning
• Data entry

22
Evaluation Themes for the Governors Prevention
Initiative

• Evaluation at all levels
• Science-based
• Capacity-building
• Collaboration
• Coordination
• Sustainability