MultiSite Evaluations: Methodological Challenges and Opportunities

1
Multi-Site Evaluations Methodological Challenges
and Opportunities

• Margarida Southard, Linda M. Dean,
• Gary Liebert, Dogan Tozoglu
• of Leon County Schools, Florida
• Presented at Southeast Evaluation Association,
  February 2006

2
Outline

• Highlights from multi-site evaluation literature
  Margarida Southard
• Reflections multi-site evaluation of Florida
  Reading Initiative (FRI) Gary Liebert
• Reflections multi-site evaluation of Read 180
  Dogan Tozoglu
• Questions and discussion

3
What is a Multisite (MSE) Evaluation?

• Turpin and Sinacore (1991) define MSEs as
  common-evaluations of large scale social programs
  such as evaluations in health services.
  Distinguished features are the involvement of
  multiples sites and the conduct of a cross-site
  evaluation activity using original data.

4
(No Transcript)
5
Why Conduct MSE Evaluations?

• Generalizability
• Increase sample size
• Examine the impact of an intervention across a
  range of sites

6
Issues in Conducting MSE

• Choice of number of sites
• Understanding the programs /sites
• Staffing Training in Evaluation
• Stakeholders Buy-in
• Variability of implementation across sites
• Standardization of data collection
• Outcome measures
• Site-by-site analysis vs. data pooled

7
Florida Reading Initiative (FRI)

• Gary Liebert

8
What isFlorida Reading Initiative (FRI)?

• North Florida Educational Consortium (NEFEC)
  originated and coordinates
• Goal achieve 100 literacy
• Provides professional development for teachers
• Provides support for schools
• Schools use research-based reading strategies of
  their choice must be school-wide

9
FRI in Leon County Schools

• LCS participated since 2002-2003
• Teachers trained in reading strategies by NEFEC
• Last year, 3 schools involved school-wide
• Reading coaches provide assistance
• Logic model developed for each school with
  training student achievement obj.
• Each school used multiple strategies

10
Multi-site challenges in FRI

• Choice of Sites
• Variability across sites
• Implementation Plan
• Participatory Approach
• Outcomes
• Data collection
• Data analysis
• Unit of analysis

11
READ 180

• Dogan Tozoglu

12
What isREAD 180 ?

• Research-based reading intervention for
  struggling readers grades 4-12
• Structured 90 minutes reading instruction daily
• Developed at Vanderbilt University released
  nationally 1999
• Goal assist struggling readers
• Provides computer software, books, structured
  assignments and small group work

13
READ 180 in Leon County Schools

• One pilot school started in 2002-2003
• Teachers trained by company
• Last year, 5 schools involved, 7 teachers
• Each license for one student per school year 60
  licenses per school

14
Multi-site challenges in READ 180

• Choice of Sites
• Variability across sites
• Implementation Plan
• Participatory Approach
• Outcomes
• Data collection
• Data analysis
• Unit of analysis

15
Questions andDiscussion