Defining Reading Proficiency for Accessible Large Scale Assessments

1
Defining Reading Proficiency for Accessible Large
Scale Assessments
Discussion of the Principles and Issues
Paper General Advisory Committee Meeting October
17, 2005 Deborah Dillon
2
Purpose
Our task is to design accessible reading
assessments. Precise definitions of the
constructs being measured is key. Step one
involves developing a definition of reading
proficiency. To that end, this paper presents a
synthesis of information and describes (a) a set
of principles that will guide the research and
development phases of our projects, and
(b) unresolved issues that need to be addressed
related to each principle.

3
Sources for the Development of the Four
Principles and Related Issues

• A review of existing definitions of reading
  proficiency
• A panel of experts to provide input
• Focus groups

4
Overview of Principles

• Principle 1 Students with disabilities deserve
  the opportunity to show their proficiency as
  readers to show what they know and are able to
  do as readers, and this should be reflected in
  the way reading proficiency is defined.
• Principle 2 Reading proficiency must be defined
  by grade level.

5
Principlescont.

• Principle 3 Definitions of reading proficiency
  must include both comprehension and foundational
  skills.
• Principle 4 Reading proficiency must be defined
  in such a way that flexible expressions of
  reading are allowed while preserving the
  essential nature of reading. This is essential as
  we seek to make assessments accessible to
  students with a variety of disabilities.

6
Principle 1 Students with disabilities deserve
the opportunity to show their proficiency as
readers to show what they know and are able to
do as readers, and this should be reflected in
the way reading proficiency is defined.

• Undergirds the other principles re defining
  reading proficiency for accessible large-scale
  assessments no issues follow this principle
• Promotes a definition of reading proficiency that
  requires assessments to reflect what students are
  able to do, not just what students are not able
  to do
• Suggests the need for large-scale assessments
  that do not rely solely on one means of entry to
  demonstrating proficiency

7
Principle 2 Reading proficiency must be
defined by grade level.

• Access to, participation in, and progress in the
  general curriculum is a foundational requirement
• States address areas of reading proficiency and
  component skills through grade-level definitions
  of content standards
• The 2009 NAEP assessment, which is required to be
  used by all states, focuses on grade-level
  cognitive targets and defines these as the
  mental processes or kinds of thinking that
  underlie reading comprehension

8
Issues under Principle 2

• How do the important component reading skills
  vary as a function of grade level?
• How is a determination made about which measure
  is best to use at a specific grade level?
• At what point are grade-level expectations
  inappropriate for individual students or groups
  of students? How are changes made to achievement
  levels while retaining grade-level content?
• How are differences in reading achievement level
  (e.g., modified or alternate achievement levels)
  developed and defined? How are these varying
  achievement levels reflected in definitions of
  reading proficiency?

9
Principle 3 Definitions of reading proficiency
must include both comprehension and foundational
skills.

• The ultimate goal of reading for students with
  and without disabilities is comprehension
  foundational skills can be measured indirectly
  for many students
• If students with disabilities are not proficient
  on comprehension measures, we need to assess
  foundational skills to understand what students
  can do and skills that may impact performance
• The overall goal is to develop flexible
  assessments to assess reading comprehension
  and/or foundational skills based on student
  performance (perhaps technology based ways)

10
Issues under Principle 3

• To what degree can component skills be measured
  independently?
• Comprehension is the primary goal for readers.
  If students are proficient in this area with
  accommodations, do we need to measure the
  foundational skills?
• If comprehension is our primary goal, should the
  comprehension score be weighted more heavily when
  foundational skills are also assessed?
• What feasible techniques are available for
  measuring foundational skills such as fluency or
  phonics knowledge in the context of large-scale
  assessment?
• For students who do not achieve grade-level
  proficiency, what processes can be developed or
  applied to aggregate their performance on
  component skills into an overall measure of
  reading proficiency?
• Some components appear to be problematic for
  certain disabilities (e.g. phonemes and deaf
  students). Do we develop alternate definitions
  based on a better understanding of reading for
  those populations?
• Are some skills less critical to measure than
  others?

11
Issues under Principle 3cont.

• If foundational skills were only assessed after a
  student had performed below proficient on
  comprehension, what proportion of ALL students
  (with and without disabilities) would be assessed
  on each of the foundational skills? Is this
  proportion small enough to assess students in
  small groups, individually, or via computer?
• Can some foundational skills be assessed together
  (e.g., decoding and phonemes)?
• If fundamental skills are going to be measured
  only in students that are not proficient on as
  assessment of comprehension, can accommodations
  be allowed that invalidate the foundational
  skills (e.g., read aloud for decoding or extra
  time for fluency).
• If a student is not proficient on a measure of
  reading comprehension, should listening
  comprehension be assessed prior to measuring
  foundational skills?

12
Principle 4 Reading proficiency must
be defined in such a way that flexible
expressions of reading are allowed while
preserving the essential nature of reading. This
is essential as we seek to make assessments
accessible to students with a variety of
disabilities.

• Proficient readers, including students with
  disabilities, may rely on any set of component
  proficiencies to read and may compensate for some
  skills they lack by drawing on othersdepending
  on their disability (e.g., a student with
  congenital deafness may comprehend what he/she
  reads, but need to deploy alternate strategies to
  understand sound-symbol relationships)
• Public views of what constitutes reading are
  reflected in various ways these are indicated in
  how states allow/do not allow different
  approaches to assessments in reading and if
  assessments are allowed without restrictions

13
Principle 4 Reading proficiency must
be defined in such a way that flexible
expressions of reading are allowed while
preserving the essential nature of reading. This
is essential as we seek to make assessments
accessible to students with a variety of
disabilities.

• Federal statutes allow a range of options in
  types of assessments used and the achievement
  standards applied to students with disabilities.
  Some students can take assessments based on
  modified achievement standards or alternate
  assessments based on grade level standards or on
  alternate achievement standards. Alternate
  achievement standards must be aligned to
  grade-level content standards but may differ from
  grade level achievement standards in breadth,
  depth, or complexity.

14
Issues Under Principle 4

• How can we determine what constitutes reading
  that is still based on grade-level achievement
  standards?
• Does what constitutes reading differ by
  disability category or by needed accommodation?
• When do the concepts of modified and alternate
  achievement standards apply to reading?
• How can we determine the ways that students with
  disabilities compensate for weaknesses in
  specific proficiency components due to their
  disability or multiple disabilities?

15
Summary Comments and Reactions

• Suggestions for other Principles?
• Suggestions for Unresolved Issues?
• Other comments or suggestions?

16
Plans
Principles Issues Paper will be posted on the
NARAP Web site