Culturally Responsive Educational Assessment Practices

1
Culturally Responsive Educational Assessment
Practices

• Assessment to inform systems planning for
  raising the achievement of culturally and
  linguistically diverse students

Contributors Seena M. Skelton, Ph.D. Melanie
Horvath, Ph.D., Constance Reyes-Rau, M.Ed. SWO
SERRC Mireika Kobayashi, M.Ed. Cincinnati Public
Schools, Monica Mitter, M.Ed. SWO Regional School
Improvement and Barbara Murphy Ohio Department
of Education Office for Exceptional Children
2
Agenda

• Ohios call to action Raising the achievement of
  all students
• Learning from successes Beyond Islands of
  Excellence A Leadership Brief
• The role of assessment in systems action planning
• Data organization tools
• Promoting evidence-based interventions system-wide

3
Today we will

• Discuss the role assessment plays in systems
  action planning to raise the achievement of all
  students and close achievement gaps.
• Examine systems variables related to the
  achievement of CLD students
• Discuss key questions for guiding district-level
  assessment practices.
• View examples of tools to assist with collection,
  organization, and interpretation of relevant
  data
• Practice posing appropriate assessment questions
  within collaborative strategic planning for
  addressing system-level barriers to culturally
  responsive practices
• View examples of system level interventions from
  schools successful in closing achievement gaps
  and raising the achievement of diverse students.

4
Ohios Call to Action

• NCLB - No Child Left Behind Act of 2001
• High expectations for ALL students
• Districts and buildings held accountable for the
  performance of ALL students
• Disaggregation of student data by
• Race/Ethnicity
• Economic Disadvantagement
• Limited English Proficient
• Students with Disabilities

5
Ohios Call to Action

• AYP - Adequate Yearly Progress
• Measures gaps in achievement among different
  subgroups of students
• Trends over time Increasing numbers of
  districts and schools not meeting AYP as the
  number of tested grade levels has increased

6
Ohios Call to Action

• IDEIA - Individuals with Disabilities Education
• Improvement Act of 2004
• Aligned with NCLB
• Includes specific provisions addressing
  disproportionality, identified in Ohio as
  overrepresentation of students from specific
  racial/ethnic groups in the following categories
• Identification as a student with a disability
• Placement in more restrictive educational
  settings once identified (separate schools or
  outside regular classroom more than 60 of school
  day)
• Disciplinary Actions (Suspensions and Expulsions)

7
Ohios Call to Action

• Growing Diversity of Student Population
• Focus on academic and behavior prevention and
  intervention efforts that raise achievement
  levels of all students, especially those students
  from racial/ethnic backgrounds overrepresented in
  the areas of disability identification,
  placements outside the regular classroom and
  disciplinary actions
• Growing need to build the capacity of districts
  to provide effective services to students with
  diverse cultural and linguistic backgrounds

8
What is assessment?

• Assessment is the process of gathering relevant
  information to (1) specify or verify a problem
  (2) make decisions about students.
• Adapted from Salvia Ysseldyke, 1991
• Assessment is more than data collection!!!

9
The Collaborative Strategic Planning Process
Problem Definition
Evaluate the Plan
Problem Analysis
Plan Development Implementation
Goal Setting
10
Purposes of assessment include

• Screening
• Instructional Planning
• Monitoring Progress
• Program Evaluation

11
Culturally Responsive Assessment in Education

• Relevance
• Rigor
• Responsive

12
Putting assessment into context

• The overall goal of culturally responsive
  assessment in education is to inform planning
  focused on increasing the academic performance of
  culturally and linguistically diverse students

13
Putting assessment into contextNo Excuses -
Video Clip

• Schools and districts can raise the achievement
  of All students!
• Assessment should impact instructional delivery
  and student engagement

14
Putting assessment into context

• We need to question any assessment practice that
  does not lead to actions resulting in improved
  student academic performance.

15
Learning from successes Beyond Islands of
Excellence Article Brief

• Everyone Read pages 1-3
• Count off at your table 1-7
• Read the finding corresponding to your number
• Identify key points in the text
• Teach those key points to others in your group
• Expand your discussion from your collective
  knowledge.

16
Quality assessment is about asking the right
questions!

• Asking the right questions
• Guides data collection
• Directs data analysis
• Informs action planning
• Leads to high powered solutions

17
(No Transcript)
18
The Big Questions ofSystems Level Assessment
Is our system working?
For whom is our system not working?
In what areas is our system exhibiting
performance gaps?
19
What questions do people ask? Table Talk

• With colleagues around you, generate a list of
  questions related to addressing achievement gaps
  and disproportionality you have heard from
  district or building leadership teams.

20

• How many students do we need to pass the OAT to
  meet AYP ?
• Which students are close to passing the OAT we
  need to target those students?
• Who can we find to provide PD on teaching math to
  3rd grade African American males?
• How can we prevent those students on IEPs from X
  school district from moving into our district?
  Theyre the reason were disproportionate
• How can we put more students on alternate
  assessment?
• others

21
Helping teams re-frame Looking at the whole
picture
22
Only looking for the short-term solution and
not what will bring about lasting change

• If a district has 100 students with IEPs
• If the pass rate for the subgroup is 65
• And out of the 100 students 60 students met
  standards
• The district team concludes that if 5 more
  students pass the OAT then the district will meet
  AYP for the upcoming year
• The proposed plan by district leadership is to
  focus intervention efforts on the 5 students
  closest to passing the OAT

23
We need to look at the whole picture what will
bring about long lasting change

• Although the district may have missed AYP by 5
  students the district still has 35 students who
  are not performing well.
• We must help districts look at the whole picture.
  Any school improvement effort should focus on
  raising the achievement of all students.
• Intervention supports that would improve the
  achievement for the 5 students the district
  identified to meet AYP should also benefit the 35
  students left not performing well.
• Systemic prevention and intervention strategies
  would benefit all students.

24
(No Transcript)
25
It takes a systems approach
It has become clear that over the past decade we
need large-scale, sustainable reform and
improvement. - Michael Fullan, 2003
26
School B 3-4
School A K-2
Since School A does not have an AYP tested
grade, school B is the feeder school for school
A. School A will receive the same AYP status as
school B.
From SWRSIT, 2006
27
Assessing the system

• Should lead to determining how systems variables
  can be altered to improve student outcomes

28
District Level Assessment

• How are policies, people and practices used to
  support core instruction, intervention
  supports/services, and school climate across the
  district leading to improved outcomes for all
  students?

29
(No Transcript)
30
Competencies of adults in the system
District mandates/rules regarding core
instruction, discipline, and support services
Activities and behaviors that are district norms
31
Quality of Core Instruction

• Evidence-based
• Curricula aligned to standards
• Rigorous
• Accurately and consistently implemented
• Closely monitored

Core Instruction
32
School Climate

• Clearly defined social expectations
• Expectations comprehensively taught
• Social expectations systematically reinforced
• Social behavior correction is firm, fair, and
  consistent
• Expectations are culturally respectful and
  socially valid

33
Support Services

• Prevention oriented
• Focused on intervening early
• Needs based
• Closely monitored
• Evidence-based strategies
• Systematic and systemic

34
(No Transcript)
35
Assessing the system

• Requires examining student performance

36
What weve learned from successful schools

• Frequent examination of disaggregated student
  performance data to understand skill gaps of low
  achieving students

37
Student Performance Data

• Ohio Achievement Test
• Curriculum-based Measurement
• Discipline
• Local
• School-wide
• Classroom

38
Graphing School or District Report Card Data
39
Graphing School or District Report Card Data

• Obtain either your district OR school buildings
  Report Card.
• It can be obtained by going to the following
  website http//www.ode.state.oh.us/reportcard/arc
  hives/Default.asp
• Select your school district from the drop down
  list.
• Pick a school building or go down to the District
  option and click on 2005-2006.

40

• Pick a grade level listed (i.e. either 3rd, 4th,
  5th, 6th, 7th, 8th, or 10th grade).
• Go to the first page of the Report card to get
  the Reading performance of ALL students in that
  grade level.
• For ExampleIf I choose 3rd Grade Reading,
• my district or school building had 61.6
• of all students score proficient or above.

41
Graphing School or District Report Card Data

• To look at the subgroups, turn to the TOP of page
  3.

42

• Open the Excel file entitled Graphing My
  District or School Report Card Data. It should
  look like this

43
Follow embedded directions and your data will be
graphed, like this
44
How Healthy is My Cone?

• Disaggregating Formative Data

45
Formative Data Cone

• Assess your students using universal screening or
  formative data, which are frequent assessments of
  student learning that will enable the adjustment
  of instruction in order to meet individual needs
  of students. Examples of such data include
• Curriculum-based measurement (CBM) in specific
  academic skill areas, such as Math, Written
  Expression, and Reading fluency. For CBM
  resources, tools and probes refer to
  http//interventioncentral.com/htmdocs/interventio
  ns/cbmwarehouse.php.

46
Formative Data Cone

• Assess your students using universal screening or
  formative data, which are frequent assessments of
  student learning that will enable the adjustment
  of instruction in order to meet individual needs
  of students. Examples of such data include
• Dynamic Indicators of Early Literacy Skills or
  DIBELS, which can be obtained at
  http//dibels.uoregon.edu/
• Office referrals and other behavioral data. For
  further information regarding setting up
  system-wide positive school supports and data
  refer to http//www.pbis.org.

47
Formative Data Cone

• Decide which students you would like to examine.
  You can pick one or more classrooms, school
  buildings, or districts formative/universal
  screening data. Here are some examples
• At the district level, what percent of your 1st
  through 6th grade students are at low risk, some
  risk, and at risk after collecting Oral Reading
  Fluency of DIBELS.
• At the school-building level, middle school
  teachers in the Math Department use
  curriculum-based measurement assessment data of
  basic math skills that was given to all 7th and
  8th grade students in order to determine which
  are at low risk, some risk, and at risk.
• A high school teacher, looking at her
  disciplinary referrals, wants to know which are
  at low-risk, some-risk, and at-risk in terms of
  behavior.

48
Formative Data Cone

• Use research-based criteria for determining
  whether students are low-, some-, or at-risk
• Curriculum-Based Measurement norms can be
  obtained for the following academic skills at
  http//interventioncentral.com/htmdocs/interventio
  ns/cbmwarehouse.php
• Oral Reading Fluency Norms Grades 1-8 Technical
  Report TIndal/Hasbrouck/Jones Courtesy of BRT
  University of Oregon
• Oral Reading Fluency Norms Grades 1-8 Web Page
  TIndal/Hasbrouck/Jones Courtesy of Read
  Naturally
• Oral Reading Fluency Norms Grades 1-8 Courtesy
  of AIMSWeb
• Math Computation Norms Grades 1-8 Courtesy of
  AIMSWeb
• Written Expression Norms Grades 1-8 Courtesy of
  AIMSWeb

49

• Open Excel file and follow directions embedded
  within

50
Other Student Data Tools

• How does your school measure up E-tool found on
  the Ohio School Leaders website
• http//Ohioschoolleaders.org
• DIBELS Data System
• http//dibels.uoregon.edu
• AIMSWEB
• http//AIMSWeb.com

51
What weve learned from successful schools

• Leadership that works toward the continuous
  improvement of policies and processes

52
A Strategic Planning Assessment Process

• Preventing DISPROPORTIONALITY by Strengthening
  District Policies and Procedures - An Assessment
  and Strategic Planning Process
• Developed by National Center of Culturally
  Responsive Educational Systems (NCCRESt) 2006

53
The NCCRESt Rubric

• An LEA Tool for Examining General Education,
  Early Intervening, Referral, Identification,
  Placement, and Access to the General Education
  Curriculum Processes

54
Purpose of the Rubric

• This tool is designed to help guide and inform
  the assessment, hypothesis building and strategic
  planning process at the district level p.7

55
Use of the Rubric as per NCLB and IDEIA 2004

• NCLB
• Accountability for results
• Evidence-based academic practices

• IDEIA 2004
• Disproportionality
• Calculations (risk ratio)
• Identification of LEAs being disproportionate
• Suspension/Expulsion rates, personnel
  development, program/policy review

56
Rubric consists of 4 Standards

• Core Functions
• Instructional Services
• Individualized Education
• Accountability

57
Rubric details

• Designed to assist district staff in reviewing
  instructional practices, and to identify and
  address areas in which policies and practices may
  contribute to disproportionality
• Results should be used to develop a plan of
  intervention
• The rubric is still a work in progress

58
Recommended Process

• Superintendent selects district-wide team
• Determine data to be used to review and score the
  rubric
• Meet to score the rubric (dyads or triads score
  the rubric independently)
• Reach consensus score based on work of small
  groups
• Report to superintendent for action

59
Rubric Activity

• At your table read the standard assigned.
• Discuss with your colleagues focus areas related
  to your assigned standard in terms of
• Clarity of focus area description
• Ohio examples reflecting the focus area or
  sources of evidence

60

• Quality of Curricula and Core Instruction
• Competencies of administrators and teaching staff
  
• Instructional Environment

• How are the educational experiences different
  for students in high performing
  districts/buildings versus low performing
  districts/buildings?

61

• Instruction
• Matters!

62
Educational Setting Matters

• that two students with the same family
  characteristics going to different schools, one
  with higher and one with lower socio-economic
  profile, could expect to be further apart in
  literacy than two students from different
  backgrounds going to the same school.
• Organization for Economic Cooperation and
  Development (2000).

63
Assessing the system

• Should lead to relevant, responsive, and rigorous
  instruction

64
Phoneme Seg. Fluency Jan to May 05-06 Yr.
Benchmark 35 correct
Ms. N Kdg Class
65
Phoneme Seg. Fluency Jan to May 05-06 Yr.
Benchmark May 35 per minute
Ms. E Kdg Class

66
Nonsense Word Fluency Jan to May 05-06 Yr.
Benchmark 25 correct per minute
Ms. N Kdg Class

67
Nonsense Word Fluency Jan to May 05-06 Yr.
Benchmark 25 correct per minute
Ms. E Kdg Class

68
What questions might the administrator in this
school building/district ask?

• Review the literacy data from Ms. N and Ms. Es
  kindergarten classes
• At your table generate a list of questions based
  on an analysis of the data graphs that an
  administrator might ask.
• Based on the data what are some implications
  regarding
• Professional Development
• Core Instruction
• Other areas

69
Guiding leaders to ask the right questions!

• Scenario Activity
• Disproportionality in Special Education Placement
• Disproportionality in Discipline
• High Achieving and Low Achieving Schools in the
  Same District
• Core Curriculum
• Alternate Assessment

70
Guiding leaders to ask the right questions!

• In groups of four people decide which scenario
  your group will discuss.
• Read the selected scenario
• As a team address the two discussion questions
• What questions would have pose to this team to
  begin their data collection and examination?
• Based on the questions, what sources of data
  would the team need to answer these questions?

71
Assessing the system

• Should lead to changes in adult practices

72
Instructional practices not supported by research

Intervention Effect Size Modality
Matched Instr. (Aud.) .03 Modality Matched
Instr. (Vis.) .04 Right Brain/Left
Brain .?? Cultural Learning Style .00 Adapt
ed from Reschly, Innovations 2006
73
Instructional Practices Supported by Research

• Intervention Effect Size
• Explicit Behavioral/ Social Skills Instruction
  1.00
• Formative Evaluation Using
• CBMGraphing Reinforcement 1.00
• Explicit Instruction Problem Solving .70 to
  1.50
• Comprehension Strategies 1.00
• Kavale (2005), Learning Disabilities, 13, 127-138.

74
Examples of Systems Strategies

• Staff Accountability
• High Expectations for Adults
• Accountability
• Support
• Professional Development
• Job Embedded
• Building Capacity

75
Last Session

• October 27, 2006
• Professional development and technical assistance
  materials
• Building our capacity to support
  districts/schools