Georgia Modification Research Study

1
Georgia Modification Research Study Spring 2006
Sharron Hunt Melissa Fincher
2
Georgias Student Assessment Program

• Includes an array of criterion-referenced tests
  (including an alternate) and a norm-referenced
  test
• Built from the ground up to be inclusive
• GA includes teachers of special populations in
  all phases of test development
• GA includes all students in pilots and field tests

3
Georgias Student Assessment Program

• GA provides a menu of traditional accommodation
  options
• Presentation
• Response
• Setting
• Scheduling
• Accommodations are provided during pilots and
  field tests

4
Georgias Student Assessment Program

• Although Georgia is inclusive, some students
  still struggle to access the tests
• Schools sought additional avenues for students
  who were caught between assessments neither the
  general or the alternate were completely
  appropriate for these students

5
Georgias Student Assessment Program

• Georgia looked for avenues to investigate how to
  make assessments more accessible
• Questions to answer
• Who are the students struggling to access the
  general assessments?
• What additional accommodations/modifications
  might assist in making the test more accessible?
• About the same time.
• Enhanced Assessment Instruments Grant announced
  (ultimately pulled back)
• Announcement of the 2 flexibility

6
Georgias Student Assessment Program

• Georgia moved forward with an empirical research
  study designed to answer those questions
• The Testing Division worked with the
• Division for Exceptional Students
• Curriculum and Instructional Services Division

7
Study Purpose

• To investigate the effects of certain
  modifications on the reliability and validity of
  the state testing programs
• Criterion-Referenced Competency Tests (CRCT)
• Georgia High School Graduation Tests (GHSGT)
• Focus on reading and math

8
Study Purpose

• This study will help Georgia determine
• What works
• Who benefits
• The goal of the study is to help Georgia
  operationalize additional options for the
  2006-2007 school year

9
What is a modification?

• an alteration in the administration of an
  assessment that results in a possible change in
  the content or construct being assessed
• typically either through the addition or removal
  of content

10
What is an accommodation?

• an alteration in the administration of an
  assessment that allows students to participate
• how the test is administered or presented
• how the student responds
• Construct (what the test measures) remains the
  same

11
Georgia Modification Study

• Focused on selected grades and content areas
• Students in grades 4 and 7 took the Reading and
  Math CRCT (March 2006)
• Students in grade 12 took the ELA and Math GHSGT
  (April 2006)

12
Georgia Modification Study

• For security reasons, off-level testing was used
• Grade 4 students took the 3rd grade Spring 2005
  CRCT
• Grade 7 students took the 6th grade Spring 2005
  CRCT
• Grade 12 students took the Spring 2005 GHSGT

13
Georgia Modification Study

• Study included both special education students
  and regular education students
• The number of special education students who took
  the test last year at each school was identified
• Tested the number of students identified, if
  possible
• Schools randomly selected an equal number of
  general education students to participate

14
Georgia Modification Study

• Included all special education students, with the
  following exceptions
• GAA students
• Students who require Braille or large print
• Set a broad net to help determine the
  characteristics of students assisted by
  modifications

15
Georgia Modification Study

• Each school assigned a condition for this study
• All students within the school took the test
  under the same condition
• Both special and regular education students

16
Georgia Modification Study

• CRCT (grades 4 and 7)
• Three conditions
• Condition 1 Resource Guides
• Condition 2 Read Aloud/Calculator
• Condition 3 Traditional Administration

17
Georgia Modification Study

• GHSGT (grade 12)
• Two conditions
• Condition 1 Resource Guide
• Condition 2 Traditional Administration

18
Condition 1 CRCT GHSGT

• Resource Guide
• A single sheet (one for each content area) that
  provides key definitions, examples,
  multiplication tables etc.
• A short practice test was provided to help
  introduce the resource guides to students each
  item related to something on the resource guide
• Recommended introducing sample tests and guides a
  couple of days before the administration

19
Condition 1 CRCT GHSGT

• Resource Guide Development
• Committee comprised of Testing, Special
  Education, and Curriculum specialists reviewed
  the tests
• identified concepts students might struggle with
  and discussed appropriate information for
  inclusion on the guide

20
Condition 2 - CRCT

• Read Aloud (Reading CRCT)
• Teacher read the reading passages and questions
  to students
• Teacher reads test verbatim
• Calculator (Math CRCT)
• Basic function calculators provided to schools
  assigned this condition

21
Condition 3 - CRCT

• Traditional Administration
• Students took tests as they have always done
• Special education students received the standard
  accommodations they normally receive
• Regular education students did not receive
  accommodations

22
Condition 2 - GHSGT

• Traditional Administration
• Students took tests as they have always done
• Special education students received the standard
  accommodations they normally receive
• Regular education students did not receive
  accommodations

23
Georgia Modification Study

• A survey was be provided for each special
  education student
• The person most knowledgeable about the student
  completed the survey
• This information will be critical in determining
  who benefits from the modifications

24
Georgia Modification Study

• Survey collected important information about each
  special education student
• Students disability/ies
• Severity of disability/ies
• IEP objectives for reading and math
• Time in mainstream/general ed classroom
• Area/s served
• Instructional services
• Type of accommodation, if any

25
Georgia Modification Study

• Administration Procedures
• Follow all administration protocols as outlined
  in the Examiners Manual
• Complete all student demographics on answer
  documents, including State-Required Codes

26
Georgia Modification Study

• Administration Procedures
• Followed all administration protocols as outlined
  in the Examiners Manual
• Completed all student demographics on answer
  documents, including State-Required Codes
• Complete survey for each special education
  student

27
Georgia Modification Study

• Key Research Questions
• Are there performance differences for students
  with disabilities on the CRCT and GHSGT when
  these assessments are administered under modified
  conditions?
• Are there performance differences for regular
  education students on the CRCT and GHSGT when
  these assessments are administered under modified
  conditions?

28
Georgia Modification Study

• Key Research Questions
• How, if at all, do performance differences on the
  traditional versus modified version of the CRCT
  or GHSGT differ for students with disabilities
  compared to regular education students?
• Are the CRCT and GHSGT reliable and valid
  assessments when administered under modified
  conditions?

29
Georgia Modification Study

• Currently analyzing data to answer the research
  questions
• Analysis at the item level
• Are certain items more likely to function
  differently?
• Analysis at the form level
• How does the test form function under each
  condition (modified vs traditional) for each
  group of students?

30
Georgia Modification Study

• Sharron Hunt, Director of Testing
• shunt_at_doe.k12.ga.us / 404.657.0314
• Melissa Fincher, Assistant Director of Testing
• mfincher_at_doe.k12.ga.us / 404.651.9405