The New England Common Assessment Program (NECAP)

1
The New England Common Assessment Program (NECAP)
Mathematics Grades 6-8 Including A Brief
Introduction and Overview of the (now) 4-State
Assessment Collaborative Presented by Dan Hupp
April 15, 2009
2
What exactly is NECAP?

• This complete presentation, including audio, will
  be available as soon as possible at
• http//www.maine.gov/education/lsalt/necap/present
  ations.html
• The PowerPoint portion of this presentation is
  currently posted.

3
Which States Participate in NECAP?
4
No Child Left Behind Requirements

• Each State
  Shall
• Develop and implement a statewide accountability
  system that will be effective in ensuring that
  all local educational agencies, public elementary
  schools and public secondary schools make
  adequate yearly progress.

5
What exactly is NECAP?

• NECAP is the assessment used by four New England
  states to meet the testing and accountability
  requirements of the No Child Left Behind Act.

6
Multiple Uses of NECAP Testing

• School Accountability
• Public Reporting
• Program Evaluation and Improvement
• Student Proficiency and Progress

7
Why is Maine Joining NECAP?
All Maine state departments were directed by the
governor to achieve major cuts in future
spending. The Commissioner was directed to
identify significant cost reductions across all
MDOE programs.
8
Why is Maine Joining NECAP?
The state assessment program had only two
realistic options 1) Adopt an all
multiple-choice test format 2) Find a less
costly way to deliver a test that also includes
student constructed responses.
9
Why is Maine Joining NECAP?

• In the fall of 2008, the MDOE conducted content
  comparisons and cost analyses associated with
  joining NECAP.
• Comparison studies between NECAP and Maine
  standards showed adequate correlation (in reading
  and mathematics) to move forward with adoption of
  NECAP in Maine.

10
Why is Maine Joining NECAP?

• Savings were estimated at 1 million dollars
  per year.
• The Maine Department of Education petitioned the
  3-state collaborative for acceptance into the
  assessment group.
• Acceptance was unanimously offered.

11
Where Do the Savings Come From?

• The economies of scale
• The MEA tests were developed, published, scored
  and reported for approximately 15,000
• students /grade level.
• The NECAP program provides the same services for
  approximately 50,000 students/grade level.

12
Why is Maine Joining NECAP?

• To achieve the cost savings, Maine is adopting
  the following for the content areas of reading,
  writing, and mathematics for assessment purposes
• NECAP standards (GLEs) at all grades 2-high
  school,
• NECAP tests at grades 3-8,
• NECAP cut scores at grades 3-8 to determine
  achievement levels, and
• NECAP reports at grades 3-8.

13
2007 MLRs and NECAP Standards
The 2007 Maine Learning Results (MLRs) are
still very much in effect and describe the
breadth and depth of knowledge and skills
expected to be taught and learned at all grades
and in all content areas.
14
2007 MLRs and NECAP Standards

• The MLRs will be modified to include the NECAP
  assessment standards (GLEs) in mathematics,
  reading and writing at grades 2-7 and high
  school, providing a consistent progression along
  the learning continuum.

15
2007 MLRs and NECAP Standards
Until the MLRs are modified, instruction
should be informed by the current Parameters for
Essential Instruction unless there is a
discrepancy between them and the NECAP GLEs. In
those instances, instruction should reflect the
NECAP GLEs.
16
2007 MLRs and NECAP Standards

• The Department is providing content-specific
  presentations that discuss the major differences
  between the NECAP GLEs and the MLRs, which will
  be posted at
• http//www.maine.gov/education/lsalt/necap/present
  ations.html

17
Some Questions Well Try to Answer

• Can you supply a brief history of NECAP?
• What are the major differences between the NECAP
  and the MEAs?
• What are the major differences in reporting?
• How will AYP be affected?

18
NECAP A Brief History

• Developed through a collaborative effort by
  Vermont, New Hampshire and Rhode Island
• Tests were administered for the first time in
  October 2005, assessing Reading and Math in
  grades 3 through 8 and Writing in grades 5 and 8
• Grade 11 Reading, Math and Writing was added in
  Fall 2007

19
NECAP A Brief History

• Participating states share the same NECAP
  standards, tests accommodations, cut scores, and
  reports (for the most part).
• Each state maintains its own alternate assessment
  program
• Each state maintains its own accountability
  system.

20
Maines Alternate Assessment
As in past years, Maines Personalized
Alternate Assessment Portfolio (PAAP) will
parallel the blueprint of the standard state
assessment plan. PAAP alternate standards and
tasks in reading, writing, and mathematics are
currently being revised.
21
Maines Alternate Assessment

• PAAPs will now be administered in Grade 2 for
  the first time.
• The Department is providing a presentation on
  PAAP that addresses revisions to Maines
  2009-2010 Alternate Assessment Standards due to
  the adoption of NECAP standards that will be
  posted at http//www.maine.gov/education/lsalt/ne
  cap/presentations.html

22
NECAP Overview
NECAP and the MEA use the same testing
contractor, Measured Progress, and NECAP has
acknowledged borrowing from the MEA when they
were forming their program. Consequently
(fortunately), there are many similarities in
policies, formats of test and answer booklets,
administration manuals, reports, etc.
23
NECAP Overview - Committees
Maine educators will still be involved in
reviewing and giving input on all test items by
serving on NECAP Item Review and Bias/Sensitivity
Committees. Maine educators, working with
colleagues from New Hampshire, Rhode Island and
Vermont, have recently completed this work for
the October 2009 test.
24
NECAP Overview Teaching/Testing Years
NECAP assesses the learning of one year
(teaching year) at the beginning of the following
year (testing year), so grades 2-7 are assessed
at the beginning of grades 3-8 in reading and
mathematics. Fourth and seventh grade writing
is assessed at the beginning of grades 5 and 8.
Alternate assessments are provided for
students in grades 2-7.
25
NECAP Overview - Test Design
Reading, Mathematics, and Writing tests consist
of multiple-choice (1 point) and
constructed-response (4 points) items. The
mathematics sessions also include short-answer
questions (1 or 2 points). One writing session
includes an extended response item (prompt - 12
points).
26
NECAP Overview Test Design
There are 9 test forms at each grade which
include 3 sessions in both reading and
mathematics of 45 minutes each, with up to an
additional 45 minutes allowed (100 extra time).
Forms contain common, field test and matrix
items. There are integrated test/answer
booklets at grades 3 and 4.
27
NECAP Test Design -Mathematics
Lets Go Math Specific.
28
NECAP Test Design -Mathematics

• Mathematics Test Design Gr. 3 4
• There are 65 common score points at grades 3-4
  distributed among the following item types
• 35-1 point Multiple Choice (MC),
• 10-1 point Short Answer (SA),
• 10-2 point Short Answer (SA),
• 54 MC and 46 Student Generated

29
NECAP Test Design - Mathematics

• Mathematics Test Design Gr. 5-8
• There are 66 common score points at grades 5-8
• 32-1 point MC
• 6-1 point SA
• 6-2 point SA
• 4-4 point Constructed Response (CR)
• 48 MC and 52 Student Generated

30
Illustrating Distribution of Emphasis (PSSM,
2000)
31
Distribution of Emphasis for Mathematics
Mathematics Content Strands 2(3) 3(4) 4(5) 5(6) 6(7) 7(8) 10(11)
Number and Operations 55 50 50 45 30 20 15
Geometry and Measurement 15 20 20 25 25 25 30
Functions and Algebra 15 15 15 15 30 40 40
Data, Statistics, Probability 15 15 15 15 15 15 15
Total 100 100 100 100 100 100 100
2(3) indicates end-of-grade 2 GLEs are tested
at the beginning of grade 3
32
Mathematics Grades 3 4 Test Design Three
45-minute sessions (Schedule 90 minutes each)
Session INo Calculator or tools
Session IICalculator and tool active
Session IIICalculator and tool active
12 multiple choice(one point each)
16 multiple choice(one point each)
5 Short Answer(one point each)
4 Short Answer(one point each)
4 Short Answer(one point each)
6 Short Answer(two points each)
4 Short Answer(two points each)
3 Short Answer(two points each)
All students are allowed an additional 45
minutes without an accommodation. Additionally,
extended time accommodations are available for
all students and must be made prior to testing.
33
Mathematics Grades 5-8 Test Design Three
45-minute sessions (Schedule 90 minutes each)
Session INo Calculator or tools
Session IICalculator and tool active
Session IIICalculator and tool active
13 multiple choice(one point each)
14 multiple choice(one point each)
14 multiple choice(one point each)
3 Short Answer(one point each)
3 Short Answer(one point each)
3 Short Answer(one point each)
3 Short Answer(two points each)
3 Short Answer(two points each)
3 Short Answer(two points each)
2 Constructed Response(four points each)
2 Constructed Response(four points each)
2 Constructed Response(four points each)
All students are allowed an additional 45
minutes without an accommodation. Additionally,
extended time accommodations are available for
all students and must be made prior to testing.
34
NECAP Test Design - Mathematics
So what do the NECAP Assessment Standards (GLEs)
look like. .and are there any practice
materials out there?
35
NECAP Test Design - Mathematics
So what do the NECAP Assessment Standards (GLEs)
look like. http//www.maine.gov/education/lsalt/n
ecap/standards.html .and are there any practice
materials out there? http//www.maine.gov/educatio
n/lsalt/necap/released.html
36
NECAP Test Design - Mathematics
Facilitated Questions What are the major
differences between the NECAP GLEs at your grade
level and your current program mathematics
program?
37
NECAP Test Design - Mathematics
Facilitated Questions Are any of the noted
differences related to material already covered
in your current program? Can understanding be
attached to/from already secure knowledge? For
example probability and fractions
38
NECAP Test Design - Mathematics

• Facilitated Questions
• What actions can be taken to address these
  differences? Who will be responsible for taking
  these actions?
• Short term
• Long term

39
NECAP Test Design - Mathematics
Facilitated Questions What resources and/or
materials might be used to support any identified
deficiencies?
40
NECAP Test Design - Mathematics
Facilitated Questions Have you considered
the many free, on-line resources that are
available to all math teachers? One example
http//illuminations.nctm.org/
41
NECAP Test Design - Mathematics
back to NECAP mathematics protocols and
procedures..
42
NECAP Test Design - Mathematics
Scratch paper is allowed on all math sessions.
(ALL answers must appear in the student answer
booklet.)
43
NECAP Test Design - Mathematics
For every grade, calculators (and other
supplied tools) are not permitted for the 1st
session, but are permitted for the last 2
sessions.
44
NECAP Mathematics

• For calculator sessions, the following resources
  are allowed
• calculators that meet NECAP specifications,
• NECAP Multiplication Tables,
• NECAP Hundreds Charts,
• NECAP mathematics tool kits, protractors and
  rulers.
• 25-50 of mathematics items are released.

45
NECAP Mathematics
Clear plastic rulers are provided to all
students for use in sessions 2 3. The rulers
are marked with both inches and centimeters.
46
NECAP Overview - Accommodations
NECAP accommodations and related policies are
currently being reviewed by educators in all four
states. (Refer to http//www.maine.gov/education/
edletrs/ilethome.shtml for more
information.) There are several differences in
current NECAP accommodations. For instance, in
the reading sessions, the questions and answers
accompanying the reading passage can not be read
to students who have the approved read aloud
accommodations.
47
NECAP Overview - Accommodations
The MEA has not allowed modifications. NECAP
currently allows the following modifications to
the test F-1 Using calculator and/or
manipulatives on Session 1 of the Mathematics
test F-2 Reading the reading test HOWEVER,
all affected sessions using these modifications
are counted as incorrect.
48
NECAP Overview - Accommodations
Also, the responses for students who have been
approved to use a word processor, typewriter, or
computer must be transcribed by school personnel
exactly as written into the students answer
booklet prior to returning materials. NECAP
can not allow computer-generated responses to be
returned in the special handling envelope for
hand scoring due to the volume of returned
materials.
49
NECAP Overview - Accommodations
Following the current review, updated
accommodation policies and a crosswalk to 08-09
MEA accommodations will be available prior to the
start of testing. IEP teams should use the
current 08-09 MEA accommodation policies and
codes for Spring 2009 meetings. These policies
are located at http//www.maine.gov/education/mea
/0809materials/policiesprocedures.pdf Approved
accommodations from the MEA 08-09 list will be
allowed for the 2009 NECAP.
50
NECAP Overview Administration Dates
The NECAP testing window will always begin on
October 1st, or the first school day following
October 1st, each year and is 3 weeks long.
Maines first administration of the NECAP tests
will begin on October 1, 2009 and continue
through October 22nd.
51
NECAP Overview Administration Dates
Testing materials will be shipped from MP the
week of September 14 -18, 2009. They should
arrive in schools during the latter part of that
week or more likely the beginning of the
following week. Following test administration,
the materials will be picked up on October
23rd.
52
NECAP Overview Administration Info.
NECAP test administration presentations
(possibly in this electronic format) containing
more specific information will occur in September
2009.
53
The NECAP Reporting System

• NECAP is very similar to Maines current
  reporting system in that it
• Utilizes an 80-point scale, with an initial digit
  representing the grade being tested.
• Classifies student achievement across 4
  achievement levels with definitions similar with
  Maines present language.

54
NECAP Achievement Levels
Proficient with Distinction (Level 4) Students
performing at this level demonstrate the
prerequisite knowledge and skills needed to
participate and excel in instructional activities
aligned with the GLE at the current grade level.
Errors made by these students are few and minor
and do not reflect gaps in prerequisite knowledge
and skills. (MEA - Exceeds the Standards)
55
NECAP Achievement Levels
Proficient (Level 3) Students performing at
this level demonstrate minor gaps in the
prerequisite knowledge and skills needed to
participate and perform successfully in
instructional activities aligned with the GLE at
the current grade level. It is likely that any
gaps in prerequisite knowledge and skills
demonstrated by these students can be addressed
during the course of typical classroom
instruction. (MEA Meets the Standards)
56
NECAP Achievement Levels
Partially Proficient (Level 2) Students
performing at this level demonstrate gaps in
prerequisite knowledge and skills needed to
participate and perform successfully in
instructional activities aligned with the GLE at
the current grade level. Additional instructional
support may be necessary for these students to
meet grade level expectations. (MEA Partially
Meets the Standards)
57
NECAP Achievement Levels
Substantially Below Proficient (Level 1)
Students performing at this level demonstrate
extensive and significant gaps in prerequisite
knowledge and skills needed to participate and
perform successfully in instructional activities
aligned with the GLE at the current grade level.
Additional instructional support is necessary for
these students to meet grade level expectations.
(MEA Does Not Meet the Standards)
58
The NECAP Reporting System

• NECAP assessment reports are very similar to MEA
  reports, will be available in late January 2010
  and will consist of
• Individual student reports
• School and SAU reports
• Class analysis reports
• They will be delivered on the same secure online
  reporting tool as the MEA reports have been.

59
NECAP Student Report
60
The Item Analysis Report
61
The Item Analysis Report
62
The NECAP Reporting System

• NECAP reporting presentations containing more
  specific information will occur in late January
  and/or February 2010.

63
Maine Accountability Reporting

• 2009-10 Adequate Yearly Progress (AYP)
  determinations will be based on the following
  assessments of learning during the 2008-09
  teaching year
• October 2009 NECAP reports
• 2008-09 PAAP data (except at grade 3)
• first year LEP participation based on 2008-09
  ACCESS administration.

64
Maine Accountability Reporting

• NECAP information, such as GLEs, released items,
  other content-related information, and
  administration information is or will be posted
  and regularly updated on links from the NECAP
  page on the DOE website at
• http//www.maine.gov/education/lsalt/necap/index.h
  tml.

65
Maine DOE Contact Information
Wanda Monthey Team Leader 207-624-6626 Wanda.monthey_at_maine.gov Daniel Hupp Director of Student Assessment 207-624-6827 Dan.hupp_at_maine.gov
Susan Smith MEA/NECAP Coordinator 207-624-6775 Susan.smith_at_maine.gov Kim Schroeter PAAP and Accommodations Coordinator 207-624-6774 Kim.schroeter_at_maine.gov
Diana Dorion ELA Content Specialist 207-624-6823 Diana.dorion_at_maine.gov Patsy Dunton ELA Content Specialist 207-624-6625 Patsy.dunton_at_maine.gov
Sandra McKechnie Program Assistant 207-624-6770 Sandra.mckechnie_at_maine.gov Rachelle Tome Title 1A School Improvement - AYP 207-624-6705 Rachelle.tome_at_maine.gov