Florida Association of

1

Florida Association of Science Supervisors
2
National Assessment of Educational Progress
(NAEP)

• The Nations Report Card
• NAEPs two major goals are to
• Measure and compare student achievement in states
  and other jurisdictions
• 2. Report changes in achievement of 4th, 8th, and
  12th graders over time

3
NAEP

• Has produced National results for Grades 4, 8,
  and 12 since 1969
• Has produced state-level results at Grades 4 and
  8 since 1990
• Has produced results for the Trial Urban District
  Assessment (TUDA) since 2002
• Does not produce student-, school-, or
  district-level results, except for the 10 TUDAs

4
Current OrganizationalStructure of NAEP
5
Federal Legislative Referenceto NAEP

• No Child Left Behind Act of 2001 (NCLB)
• Section 111(c)(2) of NCLB requires
  states/districts who receive Title I funding to
  participate in biennial State NAEP assessments of
  4th and 8th graders in Reading and Mathematics,
  beginning with the 2002-2003 school year.

6
State Legislative Referenceto NAEP

• Florida State Statute 1008.22 (2)
• The Commissioner of Education shall direct
  Florida school districts to participate in the
  administration of NAEP . . . both for the
  national sample and for any state-by-state
  comparison programs .
• State legislation requires all selected schools
  to participate in all NAEP assessments.

7
NAEP Components
STATE (Public) (Grades 4 8)
8
NAEP Schedule of Assessments
9
NAEP Frameworks

• The American Institute of Research (AIR), with
  the assistance of teachers, develops the NAEP
  frameworks.
• The National Assessment Governing Board (NAGB)
  approves and adopts the frameworks after they
  have been reviewed and critiqued by teachers.
• Each framework describes the content and format
  of the 4th, 8th, and 12th grade assessments.
• There are frameworks for each of the content
  areas assessed by NAEP.

10
NAEP Science Framework

• Current science framework was used to guide the
  1996, 2000, and 2005 assessments. Requires
  assessment in
• Earth science
• Physical science
• Life science
• A new framework, approved in 2005, will be used
  to direct future assessments.

11
NAEP Design Item Development

• Items are based on the subject-area frameworks
  approved by the National Assessment Governing
  Board (NAGB).
• Items are developed by teachers, subject- area
  specialists, and assessment experts.
• Items meet professional standards for testing.
• Experts review items for technical/statistical
  quality, content quality, bias, and sensitivity.

12
How is Floridas NAEP School Sample Selected?

• Step 1 Lists of schools are obtained from the
  National Center for Education Statistics Common
  Core of Data (CCD).
• Step 2 Schools are grouped into strata by two
  primary characteristics
• Type of location (central city, urban fringe,
  rural)
• Minority enrollment (white, African American,
  Hispanic)
• Step 3 Within strata, schools are ordered by a
  measure related to achievement.
• Step 4 A proportional sample is selected
  systematically from the ordered list.

13
Measure of Achievement Used for Floridas NAEP
School Sample Selection

• Grade 4
• Mathematics average scale scores for all Grade 4
  students in 2005
• Grade 8
• Median household income (because the match rate
  for achievement scores for 8th grade was less
  than 70)

14
How is Floridas NAEP Student Sample Selected?

• A list of all 4th or 8th grade students enrolled
  at the sampled schools is submitted
  electronically to Westat from the FLDOE, based on
  the October FTE data. Westat randomly selects
  students chosen to participate in NAEP.
• 90 of selected students at each school must
  participate in the assessment. Make-up sessions
  are scheduled at schools not meeting this
  benchmark.
• Students exempted by parental request are not
  replaced and are not used to calculate the
  assessment participation rate.

15
NAEP 2007

• NAEP 2007 administered in Florida from January
  22 through February 21, 2007.
• Each student takes only a portion of the
  questions, answering two 25-minute sections of
  subject-area questions.
• Students also answer questions related to their
  backgrounds and their home or school experiences
  in the subject of their assessment.
• Assessment is administered by a team from Westat,
  the NAEP administration contractor for the US
  DOE, and the National Center for Education
  Statistics (NCES).
• Parents must be notified prior to the assessment.
  Refusals must be in writing.

16
NAEP Inclusions and Accommodations

• NAEP has included students needing accommodations
  since 1998.
• Students with disabilities (SD) and English
  language learners (ELLs) are to be included in
  NAEP based on an IEP/504 Accommodations Plan/LEP
  Committee decision.
• Most frequent accommodations with NAEP
• Extra testing time
• Individual or small group administrations
• Large print booklets
• Assessment must be completed in one day.

17
NAEP Analysis and Reporting

• National, State, and 10 Trial Urban District
  Assessment (TUDA) results.
• NAEP scores are reported in two ways by Scale
  Scores and by Achievement Levels
• No student, classroom, or school results are
  provided
• No district results are provided except for the
  10 TUDAs
• Subgroup, state, and national comparisons can be
  made
• Relationships between student proficiency and
  certain background variables are available

18
NAEP Scale Scores

• Scale scores provide a comparison of student
  performance with regard to a group (e.g., the
  nation) or a subgroup (e.g., ELL) on a set of
  items.
• Scale scores
• Reading and Mathematics scale score range is
  0-500
• Science and Writing scale score range
• is 0-300
• Scale scores are reported as an average for
  groups of students and may obscure progress or
  problems related to student performance across a
  distribution of scores.

19
National NAEP Average Scale Scores Grades 4, 8,
and 12 Science 1996-2005

• National NAEP science average scale scores were
  4 points higher in 2005 than in 1996 at Grade 4.
  At Grade 12, scale scores were 3 points lower.
  There has been no change at Grade 8. indicates
  score is significantly different from 2005.

20
NAEP 2005 Hands-On Science Tasks

• Approximately half of the students assessed in
  science participated in a hands-on task (kits).
• Grade 4 Seeds, Unknown Powder, Markers
• Grade 8 Markers, Powders, Soil Tests
• Grade 12 Soil Tests, Antacid, Pendulum

21
NAEP 2005 Hands-On Science Tasks

• Students use the kits to conduct an investigation
  and record their findings in their assessment
  booklets
• Kits are self-contained, including any safety
  equipment that may be needed
• Students are shown how to clean up when they are
  finished with their kits
• Certain pieces of the kits are left at the school
  for student use

22
Florida Compared to 43 Other States and the
Department of Defense SchoolsNAEP 2005 Average
Scale Scores Grade 4 Science
Of the 44 states and one jurisdiction that
participated in the NAEP 2005 Grade 4 science
assessment, students average scale scores in
Florida were higher than that of 10, not
significantly different from that of 12, and
lower than that of 22.
23
Florida Compared to 43 Other States and the
Department of Defense SchoolsNAEP 2005 Average
Scale Scores Grade 8 Science
Of the 44 states and one jurisdiction that
participated in the NAEP 2005 Grade 8 science
assessment, students average scale scores in
Florida were higher than that of 3, not
significantly different from that of 8, and
lower than that of 33.
24
2005 NAEP Grade 4 ScienceFlorida and the Nation
Average Scale ScoresRace/Ethnicity
25
2005 NAEP Grade 4 ScienceFlorida and the Nation
Summary of Race/Ethnicity Scale Score Results

• In 2005, the average scale score of Floridas
  African American students was 31 points lower
  than the average scale score of Floridas white
  students. In the nation, the average scale score
  for African American students was 33 points lower
  than the average scale score of white students.
• In 2005, the average scale score of Floridas
  Hispanic students was 17 points lower than the
  average scale score of Floridas white students.
  This performance gap was narrower than that of
  the nation (29 points).

26
2005 NAEP Grade 8 ScienceFlorida and the Nation
Average Scale ScoresRace/Ethnicity
27
NAEP Grade 8 Science 1996 and 2005Florida and
the NationRace/Ethnicity
28
NAEP Grade 8 Science 1996 and 2005Florida and
the Nation Summary of Race/Ethnicity Scale Score
Results

• In 1996 and 2005, the average scale score of
  Floridas African American students was 37 points
  lower than the average scale score of white
  students. The National gap declined from a 39-
  point difference in 1996 to a 36-point difference
  in 2005.
• In 1996 and 2005, the average scale score of
  Hispanic students was 24 points lower than the
  average scale score of white students. The
  National gap increased from 30 points to 32
  points.

29
NAEP Achievement Levels

• NAEP Achievement Levels (ALs) try to identify
  what students should know and be able to do at
  various achievement levels.
• Advanced, Proficient, Basic, and below Basic
  indicate NAEP achievement levels. Each is
  determined by cutpoints established by
  professional educators and NAGB.
• The number or percentage of students in the
  population at large or in subpopulations can be
  reported by the four achievement levels.
• Results show how different groups are performing
  in relation to each other and over time.

30
Florida Compared to 43 Other States and the
Department of Defense SchoolsNAEP 2005 at
Proficient and aboveGrade 4 Science
Of the 44 states and one jurisdiction that
participated in the NAEP 2005 Grade 4 science
assessment, the percentage of students scoring at
Proficient and above in Florida was higher than
that of 8, not significantly different from that
of 15, and lower than that of 21.
31
Florida Compared to 43 Other States and the
Department of Defense SchoolsNAEP 2005 at
Proficient and aboveGrade 8 Science
Of the 44 states and one jurisdiction that
participated in the NAEP 2005 Grade 8 science
assessment, the percentage of students scoring at
Proficient and above in Florida was higher than
that of 3, not significantly different from that
of 11, and lower than that of 30.
32
NAEP Achievement Levels Florida and the Nation
Grade 8 Science 1996 and 2005
Floridas NAEP scores on Grade 8 science have
remained unchanged since 1996.
33
Florida Compared to the Nationat Proficient and
above NAEP 2005 Grades 4 and 8
ScienceRace/Ethnicity
Percentage of States Florida Outperformed
34
Florida Compared to the Nation at Proficient
and aboveNAEP 2005 Grade 8 ScienceRace/Ethnicity
Percentage of States for which Floridas Change
from 1996 to 2005 was Greater
35
NAEP AND FCAT

• FCAT measures student performance on selected
  benchmarks, as defined by Floridas Sunshine
  State Standards (SSS)
• NAEP reports on the performance of groups of
  students at a given time - and across time -
  without specifying how a subject should be taught

36
FCAT and NAEP ParticipationGrade 4 Science 2005
37
FCAT and NAEP ParticipationGrade 8 Science 2005
38
2005 FCAT, Florida NAEP, and National NAEP Grade
4 NAEP and Grade 5 FCAT ScienceGrade 8 NAEP and
FCAT Science
Achievement Levels FCAT Level 3 and above NAEP
Percent Proficient and above
Grade 8
Grade 4
39
2005 FCAT, Florida NAEP, and National NAEP
Grades 4 and 8 ScienceSummary of Achievement
Level Results

• Percentage of students performing at Proficient
  and above in science is very similar for FCAT
  Grade 5 and Grade 4 Florida NAEP and National
  NAEP.
• Percentage of students scoring at Proficient and
  above on the Grade 8 science FCAT is greater than
  on the Grade 8 Florida NAEP and National NAEP.

40
2005 FCAT Grade 5 and Florida NAEP and National
NAEP Grade 4 ScienceRace/Ethnicity
Achievement Levels FCAT Grade 5, Level 3 and
above NAEP Grade 4, Percent Proficient and above
41
2005 FCAT Grade 5 and Florida NAEP and National
NAEP Grade 4 ScienceSummary of Race/Ethnicity
Achievement Level Results

• Grade 5 students in all three racial/ethnic
  groups had higher average scale scores in science
  and a greater percentage achieving Proficient and
  above on the FCAT than Grade 4 students did on
  Florida NAEP and National NAEP.

42
2005 FCAT, Florida NAEP, and National NAEP Grade
8 ScienceRace/Ethnicity
Achievement Levels FCAT Level 3 and above NAEP
Percent Proficient and above
43
2005 FCAT , Florida NAEP, and National NAEP
Grade 8 ScienceSummary of Race/Ethnicity
Achievement Level Results

• A greater percentage of Grade 8 white, African
  American, and Hispanic students scored Level 3 or
  above on the FCAT than scored Proficient and
  above on Florida NAEP and National NAEP

44
The Nations Report Cardhttp//www.nces.ed.gov/na
tionsreportcard/
45
NAEP Websites

• Floridas NAEP website
• http//firn.edu/doe/sas/naephome.htm
• NAEP website
• http//nces.ed.gov/nationsreportcard/
• NAEP Data Explorer
• http//nces.ed.gov/nationsreportcard/nde/
• NAEP Questions Tool
• http//nces.ed.gov/nationsreportcard/itmrls

46
NAEP Coordinator Contact Information

• Michele Sonnenfeld
• NAEP State Coordinator
• Florida Department of Education
• [phone]
• SC 205-0787
• FAX [phone]
• SC 205-0781
• Michele.Sonnenfeld_at_fldoe.org