Access Center 3rd Annual Information Sharing Community Meeting

1
Access Center3rd Annual Information Sharing
Community Meeting

• October 4, 2005

Works in Progress A Report on Middle and High
School Improvement Programs Steve Fleischman,
CSRQ Center Director
2
Presentation Overview
Presentation Overview

• Why We Need Better Evidence
• How to Find and Judge Evidence
• Evidence on Middle and High School Improvement
  Programs
• Working together to improve Middle School
  programs and practice

3
Need for Better Evidence

• Claims, Claims, Claims
• Need and demand for better evidence to guide
  school improvement
• Importance of matching research methods to
  questions asked
• Importance of implementation
• Judging the quality of research (Who does it? Who
  do you trust?)

4
Three BIG Questions

• What works?
• How do you know?
• So what?

5
Sources of Evidence

• Sources of Evidence for Decision Making in
  Education
• Empirical Evidence
• Professional Wisdom
• Why Are Both Needed?
• Without professional wisdom education cannot
• adapt to local circumstances
• operate intelligently in the many areas in which
  research evidence is absent or incomplete.
• Without empirical evidence education cannot
• resolve competing approaches
• generate cumulative knowledge
• avoid fad, fancy, and personal bias
• Adapted from a presentation by Grover (Russ)
  Whitehurst, IES Director, US Department of
  Education (http//www.ed.gov/offices/OERI/presenta
  tions/evidencebase.html)

6
How will we know if its a strong study ?

• What to look for in research studies on program
  or practice effectiveness
• Detailed description of the study sample (Does
  the program serve kids like mine?)
• Indication that the program is the likely source
  of change in students outcomes (RCTs are
  strongest source of evidence)
• Pre and post test
• Comparison group
• Indication that the findings reported are based
  on appropriate methods of statistical analysis
• Sufficient detail on the findings and
  implementation of the intervention to allow its
  replication

7
Resources for Judging Research

• Slavin, R.E. (2003). A readers guide to
  scientifically based research. Educational
  Leadership, 60, 12-16. http//www.ascd.org/publica
  tions/ed_lead/200302/slavin.html
• Fashola, O.S. (2004). Being an informed consumer
  of quantitative educational research. Phi Delta
  Kappan, 85, 532-538.
• Stringfield, S. (1998). Choosing Success.
  American Educator. http//www.aft.org/pubs-report
  s/american_educator/fall98/ChoosingSuccess.pdf
• Lauer, P. A. (2004). A policymakers primer on
  education research How to understand, evaluate
  and use it. http//www.ecs.org/html/educationIssue
  s/Research/primer/foreword.asp
• Institute of Education Sciences, U.S. Department
  of Education (2003). Identifying and implementing
  educational practices supported by rigorous
  evidence A user friendly guide. (Prepared by the
  Coalition for Evidence Based Policy,
  )http//www.ed.gov/rschstat/research/pubs/rigorou
  sevid/rigorousevid.pdf

8
What do we know about effective programs?

• Comprehensive School Reform
• Educators Guide to Schoolwide Reform (AIR)
  http//www.aasa.org/Reform/
• CSR Meta-Analysis (Borman et al)
  http//www.csos.jhu.edu/CRESPAR/techReports/Repor
  t59.pdf
• Catalog of School Reform Models
  (NWREL)http//www.nwrel.org/scpd/catalog/index.sh
  tml
• Reading
• Florida Center for Reading Research
  http//www.fcrr.org/FCRRReports/reportslist.htm
• Math (and, soon, other topics)
• What Works Clearinghousehttp//www.whatworks.ed.g
  ov
• Substance Abuse
• SAMHSA, U.S Department of Health and Human
  Serviceshttp//modelprograms.samhsa.gov/
• Safe Supportive Schools
• U Colorado, Center for the Study and Prevention
  of Violencehttp//www.colorado.edu/cspv/blueprint
  s/

9
AIR Resources for EvidenceBased School
Improvement

• Center for Effective Collaboration and
  Practicehttp//cecp.air.org
• Center for Implementing Technology in
  Educationhttp//www.citeducation.org
• Comprehensive School Reform Quality (CSRQ)
  Centerhttp//www.csrq.org
• K8 Access Centerhttp//www.k8accesscenter.org/
• National Center for Mental Health Promotion and
  Youth Violencehttp//www.promoteprevent.org
• National Center for Technology Innovationhttp//w
  ww.nationaltechcenter.org
• National Center on Education, Disability and
  Juvenile Justicehttp//www.edjj.org

• National Center on Student Progress
  Monitoringhttp//www.studentprogress.org
• National Coordinator Training and Technical
  Assistance Centerhttp//www.k12coordinator.org
• National Evaluation and Technical Assistance
  Center for the Education of Children Who Are
  Neglected, Delinquent, or At Riskhttp//www.negle
  cted-delinquent.org
• National Reporting System for Adult Education
  (NRS)http//www.nrsweb.org
• Supplemental Educational Services Quality (SESQ)
  Centerhttp//tutorsforkids.org
• Technical Assistance Partnership for Child and
  Familyhttp//www.air.org/tapartnership
• What Works Clearinghousehttp//www.whatworks.ed.g
  ov

10
More AIR Resources

• Safe, Supportive and Successful Schools Step by
  Step (Osher, et al) (www.sopriswest.org)
• ASCD Educational Leadership column on Research
  Matters (www.ascd.org)
• CSRQ Center (www.csrq.org)
• AIRs School District Consulting Services
  (www.air.org)

11
CSRQ Center
CSRQ Center What we do

• Produce consumer-friendly CSRQ Center Reports.
• Develop partnerships to promote knowledge and use
  of CSRQ Center reports and tools.
• Provide technical assistance in partnership with
  selected states, districts, and schools.

12
CSRQ Center Reports

• Works in Progress A Report on Middle and High
  School Improvement Programs (January 2005)
• CSRQ Center Reports on Elementary School CSR
  Programs (Fall 2005)
• CSRQ Center Reports on Education Service
  Providers (Fall 2005)
• CSRQ Center Reports on Middle and High School CSR
  Programs (Fall 2006)
• CSRQ Center Reports on Elementary School CSR
  Programs (revised and expanded, Fall 2006)

13
CSRQ Center
CSRQ Center Reports Framework

• CSRQ Center Reports are produced using Quality
  Review Tools (QRT). Reports features
• Basic Program Information
• Dimensions of Quality. Strength of evidence of
• Positive Effects on Student Achievement
• Positive Effects on Additional Outcomes
• Family and Community Involvement
• Strong Link between Research and Program Design
• Program Providers Support for Implementation
• Program Providers Financial Viability
• Evidence of Program Providers Capacity to
  Deliver High-Quality Services to All Schools

14
Works in Progress Report

• Contents, Suggested Uses and Limitations
• Report Overview (see handout)
• The Structure of Works in Progress
• Key challenges
• Responses to key challenges including research
  findings
• Considerations
• Resources and references
• CSR Section

15
Works in Progress Report (2)

• Guidance on Middle School Key Issues
• Transition to Middle School
• Literacy and Reading
• English Language Learners
• Violence and Bullying
• Alcohol, Tobacco, and Other Drugs
• Parental Involvement
• Transition from Middle to High School
• CSR Models and Key Issues

16
Works in Progress Report (3)

• Guidance on High School Key Issues
• Transition to High School
• Literacy and Reading
• English Language Learners
• High School Dropouts
• Violence
• Alcohol, Tobacco, and Other Drugs
• Transition from High School to Postsecondary
  Settings

17
Works in Progress Report (4)

• Guidance on High School Key Issues
• Transition to High School
• Literacy and Reading
• English Language Learners
• High School Dropouts
• Violence
• Alcohol, Tobacco, and Other Drugs
• Transition from High School to Postsecondary
  Settings

18
Works in Progress Report
Application of WIP

• Each group is a school improvement team
  comprised of teachers and administrators. You are
  seeking to address the two primary issues in
  middle or high school you have identified.
  Discuss the following questions
• How might these resources be useful to you?
• How could you use these resources to move to
  move to next steps on school improvement?

19
Works in Progress Report
WIP Limitations

• Survey of Issues-Not Exhaustive (starting point,
  not ending one)
• Need for Better Evidence
• Suggestive Evidence Not Definitive (not a what
  works report, but a desk reference)
• Need for Changes in Multiple Areas (need
  comprehensive approaches)
• Effective Implementation is Key
• Need for Alignment with Local Efforts

20
Working Together to Promote Evidence- Based
Improvement

• Take the lead on insisting that policies,
  programs, and approaches are evidence-based.
  (Show me the evidence.)
• Once made, stick to policies that are based on
  sound evidence and give them time to work. (No
  quick fixes.)
• Work in partnership with the CSRQ Center (Better
  Evidence. Better Choices. Better Schools.)

21
CSRQ Center Contact Us

• American Institutes for Research
• 1000 Thomas Jefferson St, NW
• Washington, DC 20007-3835
• www.csrq.org
• Steve Fleischman, Director
• sfleischman_at_air.org 202/403-5989