Students with Significant Cognitive Disabilities: Creating Measures to Assess Their Progress

1
Students with Significant Cognitive
Disabilities  Creating Measures to Assess Their
Progress

• Teri Wallace, Renata Ticha, Kathy Gustafson
• Research Institute on Progress Monitoring (RIPM)
• University of Minnesota
• walla001_at_umn.edu progressmonitoring.org
• May, 2006

2
The Use of Progress Monitoring with Students with
Significant Cognitive DisabilitiesA White Paper
Prepared for the National Center on Student
Progress Monitoring(www.studentprogress.org)

• Authors Diane Browder, Teri Wallace,
• Martha Snell, Harold Kleinert

3
Topics for Today

• Background, context, definitions
• Some goals and issues related to progress
  monitoring with students with significant
  cognitive disabilities
• Ideas and models to consider
• RIPMs research

4
Progress Monitoring

• Progress monitoring is used to assess students
  academic progress to plan more effective
  instruction and interventions for students as
  needed.
• Such monitoring becomes equally important for
  determining if students are meeting grade level
  expectations.
• It has also been used to predict students
  performance on state standards assessments, in
  some cases.

5
however, progress monitoring has rarely been
applied or evaluated for students with
significant cognitive disabilities.

• Extensive research and knowledge from practice
  supports the notion that progress monitoring can
  enhance instruction and assess students progress
  toward state achievement standards.

6
Defining Significant Cognitive Disabilities

• Students who participate in alternate assessments
  with alternate achievement standards linked to
  state grade level content standards
• Students who are (1) within one or more of the
  existing categories of disability under the IDEA
  (e.g., autism, multiple disabilities, TBI, etc.)
  and (2) whose cognitive impairments may prevent
  them from attaining grade-level achievement
  standards, even with the very best instruction.
• - August, 2005 NCLB non-regulatory
  guidance

7
Some Numbers
The number of students participating in alternate
assessments on alternate achievement standards as
compared to the total population of student
learners and students with disabilities
Chart originally produced by the National
Alternative Assessment Center
8
Some Numbers
The total student population receiving special
education services broken down by disability
category
Chart originally produced by the National
Alternative Assessment Center Source Education
Week analysis of data from the U.S. Department of
Education, Office of Special Education Programs,
Data Analysis System, 2002-03.
9
Students with Disabilities in State
Accountability Systems

• The 1997 Amendments to IDEA required all students
  with disabilities to be included in State and
  district-wide assessment programs.
• Under IDEA, States have been required to include
  students with disabilities in regular assessments
  since 1997, and beginning July 1, 2000, in
  alternate assessments.
• NCLB requires all students be held to grade-level
  achievement standards by taking assessments.
  However..

10
Students with Disabilities in State
Accountability Systems

• The December 9, 2003 regulations provide an
  avenue for students with disabilities to be
  assessed through the following means (as
  determined by each childs IEP team)
• The regular grade-level State assessment
• The regular grade-level State assessment with
  accommodations (setting, timing, response, etc.)
• Alternate assessments aligned with grade-level
  achievement standards
• Alternate assessments based on alternate
  achievement standards

11

• For alternate assessments based on alternate
  standards in grades 3 through 8 and at the high
  school level, the assessment materials should
  show a clear link to the content standards for
  the grade in which the student is enrolled,
  although the grade-level content may be reduced
  in complexity or modified to reflect
  pre-requisite skills. The State may define one
  or more alternate achievement standards for
  proficiency for any grade or cluster of grades.
  
• - August, 2005
  (non-regulatory guidance, NCLB)

12
Alternate Achievement Standards

• There are often different levels of state
  standards (Level A-broad standard),(Level
  B-performance indicators or key elements for
  certain grade levels), (Level C-activities the
  student must do to meet Level B)
• Changes or adjustments can be made to develop an
  alternate standard -
• Expand or extend performance indicators
• Provide functional performance indicators

13
Example -

• Level A - Students will read and write for
  information and understanding.
• Level B - Students will understand and acquire
  new vocabulary and use it correctly in reading
  and writing (5th grade).
• Level C - Extended Indicator - Student will name
  or point to pictures of objects.
• Level C - Functional Performance Indicator -
  Student will identify functional sight words and
  signs.

14

• Modified achievement standards are for those
  students with disabilities who could make
  significant progress during the school year but
  may not reach grade-level achievement standards
  within the same time frame as other students.
• - (Dept. of Education, May
  2005)

15
Progress Monitoring for Students with Significant
Cognitive Disabilities

• GOALS and ISSUES

16
GoalsProgress Monitoring

• For the student to show progress in academic
  content aligned with his or her assigned grade
  level.
• For the student to make progress toward literacy
  or numeracy in general.
• For the student to make progress toward state
  accountability requirements.
• Others?

17
Issues for Consideration

• Flexibility is needed when assessing students who
  may be deaf/blind, have limited physical
  movement, are nonverbal, or who require alternate
  ways of responding to academic content.
• There is limited research demonstrating how to
  define and teach academic skills to students with
  significant disabilities. Most has focused on
  accessing the general education curriculum by
  using functional skills or skills such as
  choice-making.

18
Issues for Consideration

• There is a lack of consensus regarding the
  progress that should be expected for students
  with significant cognitive disabilities in
  general education.
• Curriculum for students with significant
  cognitive disabilities has typically not been
  sequenced by grade level.
• IDEA and NCLB both require determining if
  adequate yearly progress is being made, using
  assessments that are linked to the states
  academic content standards. But, how?

19
Issues for Consideration

• Differentiation of expectations based on the
  degree to which the student is acquiring
  symbolic communication along with academic
  content. (Presymbolic, early symbolic, and
  expanded symbolic.)
• Others?

20
The Question

• What can we use to monitor the progress of
  students with significant cognitive disabilities
  that provides useful information to teachers
  about progress toward IEP goals, grade level
  content standards, academic content, and
  contributes to state accountability systems?

21
Progress Monitoring for Students with Significant
Cognitive Disabilities

• THREE OPTIONS

22
Three Options Progress Monitoring

• Extending data-based decisions research (using
  Mastery Learning) that primarily targeted
  functional life skills for this population to
  academic content
• 25 years of research to support use of data or
  rule-based decision systems
• Farlow Snell, 1989 1994 Brown Snell, 2006
  Browder et al, 2005see reference list

23
Three Options Progress Monitoring

• Building on portfolio models currently in use for
  states alternate assessments to track progress
• Kleinert Kearns, 2004 Kleinert, Kennedy,
  Kearns, 1999see reference list

24
Three Options Progress Monitoring

• Developing ways to apply the research on
  curriculum-based measurement (CBM) for students
  with high incidence disabilities to students with
  significant cognitive disabilities
• 30 years of research more than 200 empirical
  studies
• Deno Mirkin, 1977 Espin Deno, 1994 Fuchs
  Fuchs, 2002see reference list

25
Mastery LearningIntroduction

• Traditionally used with students with cognitive
  disabilities in the functional domain
• Task specific, e.g. using a communication board
  to buy groceries (focus on mastery learning of
  specific objectives)
• Objectives on an IEP
• Using her communication device, Jenny will
  independently ask for groceries in the store by
  the end of the school year.
• Hierarchy of skills observable steps
• Task analysis
• Defined as accuracy, level of independence, level
  of generalization, etc.
• At least 40 states use mastery monitoring within
  their alternate assessment.

26
Mastery LearningAcademic Example

• Article Cushing, Clark, Carter, Kennedy (2005)
• Jennys goal
• Communicating wants and needs verbally or with
  her communication device (p.7)
• Jennys objective
• Jenny will answer questions correctly and
  independently either verbally or by using her
  communication device 80 of the time during three
  consecutive classes (p. 13)

27
Data-Based Decision ModelBased on White paper by
Browder, Wallace, Snell, Kleinert (2005)

• Mastery data can be graphed ( correct, level of
  independence, etc.)
• Data collection method is chosen for target skill
• Start by collecting data for each teaching
  session
• After 3 data points are collected aim line
• After next 5 data points evaluate performance
  (3 out of 5 points above the aim line)
• If performance is inadequate Problem Analysis
  Worksheet

28
(No Transcript)
29
Problem Analysis Worksheet
30
Limitations of Mastery Monitoring

• Mastery monitoring
• Focuses on discrete behaviors
• Limited data on reliability and validity
• Progress cannot be judged over time
• Focus of measurement changes every time mastery
  is achieved
• Steps may be of unequal difficulty
• Every student is measured on a different task
  time consuming for teachers
• - Fuchs (1995)

31
PortfoliosIntroduction

• Systematic collections of student work
• Gathered over time
• Demonstrate student performance and progress on
  targeted skills and knowledge
• Related to state standards (when used as
  alternate assessment)

32
PortfoliosIntroduction

• Greater suitability for students with significant
  cognitive disabilities than traditional
  assessments (given heterogeneous needs of
  students)
• Greater flexibility in documenting wide range of
  student outcomes
• Potential for enhancing student
  self-determination and motivation by including
  students in assessment process

33
Portfolios Applicability for Progress Monitoring

• Include wide range of artifacts as evidence of
  student progress
• Student work samples
• Graphs/charts of student performance data
• Video or audio tapes of student performance
• Sensitive model for assessing breadth and scope
  of student learning (rather than narrow range of
  skills)

34
PortfolioCriteria

• Contents scored according to pre-determined
  criteria
• Criteria related to individual student
• Accuracy of students work
• Degree of independence, progress, competence,
  and/or generalizability of student performance
• Criteria related to system
• Student worked with non-disabled peers and in
  multiple settings
• Student provided opportunities to plan and
  evaluate own work

35
PortfoliosAlternate Assessments

• Portfolios currently used as alternate assessment
  by approximately 46 of states

36
PortfoliosLimitations

• Time-intensive
• For teachers to implement
• For contents to be scored
• Continuing challenge of determining technical
  adequacy of scores reliability and validity

37
Curriculum-Based Measurement (CBM)Introduction

• Curriculum-based Measurement (CBM) is an approach
  to measuring the academic growth of students
  (Deno, 1985).
• It is a General Outcome Measure (GOM) intended to
  serve as an indicator of a students performance
  (in reading).
• It uses simple, repeated measures to monitor a
  students progress over time. For example, the
  number of words a student reads correctly in 1
  minute.

38
Curriculum-Based Measurement (CBM)Introduction

• CBM provides an easy and quick method of
  gathering information about student progress
• Teachers can analyze student scores and adjust
  student goals and instructional programs
• Student data can be compared to teachers
  classroom or school district data
• Its use has expanded to assessing groups,
  aligning CBM performance with high stakes testing
  performance, component of RTI, placement, and
  much more.

39
CBMStandards for Measures (Deno)

• CBM Measures
• General standard Decision validity
• Can the measure be used for the decision to be
  made?
• Specific standards
• Technical adequacy
• Reliable
• Valid
• Practicality
• Repeatable and Brief
• Easy to use
• Easy to understand
• Easy to teach
• Durablility
• Appropriate for range of age / skill levels

40
CBM Sample Reading Tasks

• At Kindergarten
• Letter Sound Fluency or Phoneme Segmentation
  Fluency
• At Grade 1
• Word Identification Fluency
• At Grades 2-3
• Oral Reading Fluency
• At Grades 4-6
• Maze Fluency

41
CBMSample Graph
1. Determine current level of performance
2. Set individual annual goal -- tied
to state standards
4. Evaluate and modify instruction on basis of
rate of progress towards goal
3. Monitor individual progress towards goal
Valid and reliable indicators of performance
42
CBMLimitations

• Timed measures may not work for students with
  significant cognitive disabilities
• Reading from text is not feasible for a student
  who is nonverbal
• Most studies on academic skills for this
  population have targeted teaching either
  functional sight words or money skills (only a
  small portion of the general education curriculum)

43
Summary Statements

• While these three methods of progress monitoring
  are currently available and provide useful
  information on students achievement of academic
  skills related to state content standards, new
  methods are needed that provide better
  information on skills aligned with grade level
  content. This alignment can be difficult to
  comprehend because it may be misunderstood as
  expecting students to achieve at grade level.
  Instead, for students with significant cognitive
  disabilities, the goal is to learn some
  meaningful portion of the content at each grade
  level to access the general curriculum (Browder,
  et al., 2006.).

44

• It would be critical to be sure that the grade
  level outcome expected for students with
  significant cognitive disabilities is articulated
  with sufficient clarity that both the CBM and
  alternate assessment align with the expectation.
  - Browder, Wallace, Snell Kleinert, 2005

45
Future

• New CBM models are needed that align with
  clearly articulated expectations for alternate
  levels of achievement of grade level content.
  Future research is also needed to be sure that
  these CBMs have the technical quality needed to
  make inferences about student progress in grade
  level content.
• - Browder, Wallace, Snell Kleinert, 2005

46
Progress Monitoring for Students with Significant
Cognitive Disabilities

• RIPM Research

47
Some promising work

• Otaiba Hosp (2004) - used letter sounds,
  passage reading and sight words to monitor
  progress while implementing a tutoring model with
  students with down syndrome
• Tindal et al (2003) examined the possibility of
  developing a set of standardized tasks (CBM
  measures) for students with disabilities which
  would measure the same construct as that of large
  scale assessments.

48
RIPMVision for Work related to Students with
Significant Cognitive Disabilities

• To support the inclusion of students with
  cognitive disabilities in state accountability
  systems by creating General Outcome Measures
  (GOMs) that assess students' progress in outcome
  areas that can be linked to state grade level
  content standards. While the measures will
  target academic outcomes by extending our present
  work in progress monitoring, for some students
  additional measures might also assess their
  progress on functional outcomes related to their
  IEP goals. - Advisory Committee,
  Dec. 2005

49
General Outcome Measures (GOM)

• Same principles as CBM
• BUT
• Probes are not necessarily from material in the
  curriculum
• Probes of equivalent difficulty are from the
  level of material student needs to master
• at grade level
• at their own level
• Example
• Isolated word lists based on Dolch words
• The Dolch words are the 220 most frequently found
  words in childrens books

50
Creating pilot measures

• Researched existing literature on CBM for
  students with sig cog disabilities very limited
• Studied early literacy measures including IGDIs
  and DIBELS
• Met with advisory group special ed. teachers
  and researchers
• Met with special ed. teachers in classrooms for
  students with sig cog disabilities
• Kept advisory board informed of our new
  developments

51
RIPM Framework Measures for Students with
Significant Cognitive Disabilities
52
Criteria for pilot measures

• Appropriate format - students required to respond
  only by pointing (no verbal response)
• Characteristics of GOMs not mastery monitoring
• Sensitive to growth over time
• Different levels of stimulus material pictures,
  letters, sight words
• Different levels of administration difficulty
  matching and identification
• Prompting system to avoid student failure

53
Sample Measures (Pilot Study)
54
(No Transcript)
55
?????? to Ponder

• How does level of independence impact progress on
  GOMs? How do we account for level of prompt?

56
Resources

• Research Institute on Progress Monitoring (RIPM)
  - progressmonitoring.org
• National Center on Student Progress Monitoring -
  www.studentprogress.org
• National Center on Alternate Assessment -
  www.naacpartners.org

57
References

• Please see handout provided with this session or
  contact Teri Wallace at walla001_at_umn.edu. Thank
  you.