Benchmarking for Performance

1
Benchmarking for Performance

• Implementing an Effective Benchmark Testing
  Strategy in the School District of Philadelpia

• Dr. Gregory Thornton, Chief Academic Officer
• John DiLello, Director, Chief Academic Office

2
Benchmarking for Performance
Benchmark testing and reporting through the
Instructional Management System has allowed our
entire district to have immediate feedback on
students progress. Our teachers are able to
differentiate their instruction, thus enriching
and remediating our students learning
experiences on an individual basis.Cecilia P.
CannonAssociate Superintendent Office of
Curriculum and Instruction The School District
of Philadelphia
3
What do we want benchmarks to do?

• Accurately predict the performance of students on
  high stakes state tests, OR
• Provide detailed information about student
  performance, in order to
• Continuously improve the effectiveness of
  instruction
• Accelerate student learning
• Are these mutually exclusive?
• Can both be accomplished?

4
Benchmark Assessments

• What kind?
• Curriculum-Reflective
• Assessing what was taught over the six-week
  period (aligned to standards)
• The kind many teachers seem to want
• State-Assessment-Predictive
• Assessing the full range of standards taught on
  the State assessment every six to eight weeks
• The kind many administrators seem to want
• Is it possible to get the best of both?

5
Purposes and Beliefs
6
Some Beliefs Regarding Benchmark Tests

• Benchmarks are most effective when tightly
  aligned to the curriculum
• Benchmarks are most effective when they are
  turned around quickly, so teachers can use the
  data to adjust instruction
• Benchmarks are most effective when associated
  with built in time to respond with intervention
  or enrichment
• Benchmarks should support a broad and rich
  curriculum, and not narrow or impoverish the
  curriculum by targeting it too closely to a few
  selected and tested standards
• Benchmark tests should be part of an integrated
  improvement strategy that includes a rigorous and
  well- aligned curriculum

7
Users, Intended Purposes, Unintended Effects

• Teachers
• Principals
• Regional and Central Administrators
• Support Staff
• Parents
• Students

8
Philadelphia's Benchmark Tests,Intended
Purposes Teachers

• Improve the Quality of Instruction Raise the
  Bar, Close the Gap
• Assess students mastery of the taught
  curriculum Identify patterns, strengths and
  weaknesses in student learning
• Help identify patterns, strengths and weaknesses
  in instruction
• Direct students to effective and indicated
  instructional interventions or enrichments
• Promote an evidence-based coherent improvement
  model
• Promote professional learning and sharing by
  creating communities of common practice around
  the core curriculum

9
Philadelphia's Benchmark Tests, Intended
Purposes Principals

• Improve the Quality of Instruction Raise the
  Bar, Close the Gap
• By grade, classroom and/or school get a sense of
  how students are performing related to specific
  standards addressed in the curriculum
• Get a sense of teacher effectiveness related to
  specific standards and areas of the curriculum
• Get a sense of how their school is performing in
  relationship to other schools in the region or
  district
• Help promote improvement strategies related to
  the use of data
• Help in the development of professional learning
  communities
• Inform types of support and professional
  development

10
Philadelphia's Benchmark Tests, Intended
Purposes Central Administration

• Improve the Quality of Instruction Raise the
  Bar, Close the Gap
• Inform professional development strategies
• Inform coaching and other support strategies
• Assess the effectiveness of core curriculum,
  support materials and intervention strategies
  (360)
• Assess performance across schools and regions to
  identify patterns, needs, strengths
• Provide quality leading data for continuous
  improvement

11
Philadelphia's Benchmark Tests,Intended
PurposesSupport Staff, Students and Parents

• Support Staff
• Individualize support
• Identify strengths and weaknesses
• Students
• Identify and understand performance targets
• Identify and work on remediation or enrichment
• Get a sense of ownership re learning
• Parents
• Get a sense of student mastery
• Get a sense of the written and taught curriculum
• Enhance connection to school

12
Processes
13
Where We Started, 2000

• Dramatic variations from school to school and
  classroom to classroom in practice around
  teaching to standards
• No explicit alignment of what is taught and what
  is tested
• Robust support documents for teachers to embrace
  standards-based instruction, but no explicit
  curriculum or performance targets

14
Context for Reform

• State takeover
• New CEO committed to creating a more coherent
  district and replicating effective practices from
  other urban districts
• Emerging research about urban reform
• Understandings regarding the need for coherence
  and alignment of what is taught and what is
  tested
• Rigorous requirements of NCLB
• Renewed interest in the use of data to structure
  improvement efforts. . .Leading to. . .

15
Concurrent Development and Implementation of the
Three Interrelated Reform Strategies

• Core Curriculum
• Benchmark Assessments
• Instructional Management System

16
Core Curriculum

• Radical shift in approach
• Need for equity and coherence
• Highly transient student population (Up to 35)
• Hard to staff schools
• Significant influx on new teachers yearly
• Variations in school leadership quality
• Need for communities of common practice
• Locally developed K-8
• Developed by Kaplan 10-12

17
Core Curriculum

• Associated with district-wide textbook and
  program adoption, but not program or
  textbook-based
• Detailed, prescriptive what, how, when
• Clear performance targets defined in six week
  chunks

18
Using the Core Curriculum and Benchmark
Assessments

• Instruction designed in six week chunks
• Five weeks of paced instruction
• PSSA checkpoints in week 2 and 4
• Benchmark assessment given in week 5
• Test results back by the end of week 5
• Benchmark data analyzed by teachers using
  protocol
• Based on review of data, one week for
  remediation, review, enrichment

19
How Students Take Benchmarks

• Computer
• Online, lab or in-classroom
• Paper and Pencil
• Scan Sheet
• 48 hour turn around
• Results available online
• Combination
• Students input on scan sheet teacher or aide
  inputs to computer for immediate results

20
Leveraging Technology The IMS

• Think about the potential for accumulating
  evidence in a student friendly way, in a teacher
  friendly way, in an administrator friendly way. .
  Think about all of the different assessment users
  and their different information needs, (then)
  build information systems around the diversity of
  those needs
• --Rick Stiggins

21
School District of Philadelphia IMS Overview
Administrators
Teachers
Parents
Students
District Curriculum
Assessment Data
Student Data
State Standards
Instructional Materials
Instructional Management System
PSSA
Student Information
Report Card Data
TerraNova/ Supera
DIBELS
Benchmarks
Instructional Resources
Curriculum
22
Benchmark Class Report
23
Benchmark Test Report Student Item Analysis
24
Results
25
Number of SDP Schools Meeting AYP Criteria
)
26
School District of Philadelphia AYP Timeline
02-03 Core Curriculum/ IMS introduced
District-wide Math and Literacy (Grades K-9)
02-03 Benchmark testing/ IMS introduced
District-wide Math and Literacy (Grades 2-9)
03-04 Core Curriculum introduced grades 10-12
Benchmark Pilot In 22 low-performing schools
03-04 Benchmark testing introduced Grades 10-11
27
Deep Change and Closing the Gap

• New Materials
• Core Curriculum
• Course Modules
• Adopted Materials
• Benchmark Tests

-
-

• New Behaviors and Practices
• Appropriate implementation of curriculum
• Differentiated strategies for
• underperforming student groups
• Planning and teaching for diversity
• Requisite skills for teaching to proficiency

Effort
Time

• New Beliefs
• Deep beliefs regarding
• Student capacity
• Student needs
• Teacher efficacy
• High and clear expectations for students and self

28
Mass Production Mass Customization
19/20th Century Paradigm
21st Century Paradigm
Sort Against Norm/Exclude
Individualize/Include
29
Management by Fact and Assessment for
Performance

• Unintended effects
• Longitudinal analysis
• Grades
• Assumptions regarding predictive validity
• Reactive and careless use for accountability
• Use for accountability with extreme caution
• PROCESS improvements related to achievement of
  teacher and student-set goals
• Create the scaffolding for improvement