NeSA-W Grades 4/8/11 Writing Assessment

1
NeSA-WGrades 4/8/11Writing Assessment

• Nebraska Department of Education Statewide
  Assessment
• Pat Roschewski pat.roschewski_at_nebraska.gov
• Edward Foy edward.foy_at_nebraska.gov
• Julie Barger ESU 16, Ogallala, NE

2
The NeSA-W/DRC Partnership

• Working together to improve teaching and learning
  for Nebraska students.

3
The Nebraska Department of Education and Nebraska
educators continue to direct the process of
statewide assessment.
4
Nebraska Educators continue to be involved in
these aspects of NeSA-W

• Prompt Creation
• Field Testing
• Prompt Selection
• Rangefinding
• Appeals
• Continued training and communication

5
The Rangefinding Process

• Included 7-10 Nebraska educators from a variety
  of educational backgrounds
• Two DRC reps facilitated the work

6
The Rangefinding Process

• Analytically scored, discussed, and came to
  consensus on 90-100 papers from the field testing
  of the test prompt.
• DRC will use the papers selected and scored by
  this group to establish the anchors and train the
  raters prior to the scoring of the assessment.

7
Grade 8 11 Online Assessment

• Grade 8 will remain a descriptive essay.
• Grade 11 will remain a persuasive essay.
• 2. The 8th 11th grades NeSA-W assessment
  will only be available online.
• 3. Students with IEPs or 504 Plans may take the
  test in pencil and paper if documented in an IEP.
• 4. Students responding in Spanish will take a
  pencil and paper assessment.

8

• 5. Paper/pencil responses at 8 and 11 must be
  written in 2 pencil. Papers written in ink will
  not be scored and will result in zero scores.
• 6. In paper/pencil testing student may use only
  the paper provided. If additional sheets of
  paper are added to the booklets, they will not be
  scored.

9

• 7. When making copies of the paper/pencil
  booklets for local scoring or possible appeals,
  do not take the booklets apart.
• 8. Do not copy the entire booklet, but only make
  a copy of the actual students response.

10

• 9. NeSA software is scheduled for release
  November 14th. (All subjects, including writing
  and practice tests.)
• 10. Districts will need to schedule a download
  or update of software prior to administering the
  assessment.
• 11. Students will be allowed to use up to 6,000
  characters in their writing responses
  approximately three pages. A pop-up warning will
  occur at approximately 4,000 characters.

11

• 12. The TAB button is not compatible with the
  software. Student should be advised to space
  over 3-5 spaces. This does not affect scoring.
• 13. The results will be analytically scored.

12
Grade 8 11 Online Assessment

• Printing options are available.
• Both the practice and operational test may be
  printed in 2012.
• The printed copy may be used for local scoring,
  placed in a student portfolio, or shared with the
  student/parents.
• Districts will need the printed copy for appeal
  purposes This copy is used to decide if an
  appeal is warranted and, if so, a copy must be
  attached to the appeal form.

13
Rubric Changes

• Changes have been made to the 8th and 11th
  grade rubrics.
• The rubrics were changed to analytic rubrics in
  the fall of 2010 from the previous holistic
  rubrics.
• The content of each new rubric has been
  tightened, and although most of the previous
  criteria remain, they have been redistributed
  into fewer domains.
• Grade 8 and 11 analytic rubrics will be in use
  this year with the 2012 assessment.

14
Rubric Changes

• The domains of the rubrics will be weighted as
  follows
• Content/Ideas 35
• Organization 25
• Word Choice/Voice/Tone 20
• Sentence Fluency/Conventions - 20

15
(No Transcript)
16
(No Transcript)
17
Rubric Changes

• Changes were made to the rubric for a number of
  reasons.
• A revision of the Language Arts standards
  occurred in 2008-2009.
• An external study determined that the new
  standards and the writing assessment rubrics did
  not match, necessitating a review/revision of the
  rubric.

18
Rubric Changes

• Influence favoring analytical assessment was
  expressed in an effort to provide more specific
  data to data users.
• The use of a computer based assessment reflects a
  real world application of writing.
• Five vendors responded to the RFP (Request for
  Proposals.)

19
Rubric Changes

• The new rubrics are available on the Assessment
  page of the NDE website.
• http//www.education.ne.gov/Assessment/NeSA_Writin
  g.htm

20
NeSA-W Assessment

• NeSA-W testing window is January 23,
  2012-February 10, 2012
• 8th and 11th grade tests will be administered
  online.
• Students with IEPs or 504 plans may take the
  test using paper/pencil if documented in an IEP.
• Students responding in Spanish or a language
  other than English will take a paper/pencil
  assessment.

21
NeSA-W Grades 8 11 On-line

• The recommended testing time is 90 minutes to
  occur on one day.
• The test is not timed, but 90 minutes is
  recommended by NDE as the amount of time to
  schedule for the assessment.
• If a student needs more than 90 minutes, that is
  allowable, the same as for NeSA-R.

22
NeSA-W Grades 8 11 On-line

• Pre-writing may be done on paper furnished by the
  district. Students may choose to prewrite and
  produce a rough draft on paper before entering it
  on the computer, or they may choose to work
  entirely on the computer.

23
NeSA-W Grades 8 11 On-line

• A new thesaurus, dictionary, and cut/paste are
  available. The spell check will not be
  available.
• An online practice test for 8th and 11th grade
  writing will be made available. You can access
  this assessment through the CAL software on
  November 14th.

24
Practice Test Instructions 8 11
25
Practice Test Prompt
26
Practice Test Self Assessment Tool
27
NeSA-W On-line Grade 11Pilot Test Times Data

• Time Span in Minutes in Each Span
• 0 - 30 minutes 6
• 30 - 60 minutes 58
• 60 - 90 minutes 31
• 90 minutes 5

28
Grade 8 11 Online Assessment
Additional Information

• Graphic organizers, story starters, and story
  frames are not to be provided to students as part
  of the state testing program.
• In the testing room, visual aids and clues should
  be removed or covered and remain covered
  throughout the administration of the writing
  assessment.

29
Grades 8 11 Online Assessment
Additional Information

• Teachers are NOT to provide editing assistance to
  students on the NeSA-W.
• No NeSA-W online tests will be provided in
  Spanish. All Spanish tests will be administered
  via paper/pencil test booklets.
• Districts submitted requests for Spanish prompts
  via eDirect.

30
Grade 8 11 Online Assessment
Additional Information
The composite score is calculated from the domain
scores as 1. Composite 1.4D1 1.0D2 0.8D3
0.8D4 D1, D2, D3, and D4 are the sums of the
two reader scores for the four domains. The table
below illustrates the calculation for a student
who received a 2 or 3 from each reader on each
domain.
R1 R2 DR1R2 W WD
Ideas / Content 3 3 6 1.4 8.4
Organization 2 3 5 1.0 5.0
Voice / Word Choice 3 2 5 0.8 4.0
Sentence Fluency / Conventions 2 2 4 0.8 3.2
Composite Score 20.6
The corresponding Scale Score can then be
retrieved from the table supplied by DRC.
31
Grade 8 11 Online Assessment
Additional Information

• Current information about the NeSA-W assessment
  and all NeSA assessments can be found on the NDE
  website.
• Go to http//www.education.ne.gov
• On your left, click on Statewide Assessment.
• Click on NeSA Writing.

32
The Process of Analytical Scoring

• The New Statewide Process

33
Holistic vs. Analytical Scoring

• Analytic scoring involves a score being given
  for each domain and its criteria. These
  individual weighted scores are combined to
  calculate the final score.

34
(No Transcript)
35
(No Transcript)
36
The Analytical Scoring Process

• The Process
• First read focus on Ideas/Content and
  Organization--assign a score to each domain of 1,
  2, 3, or 4.
• Then review paper focus on Word Choice/Voice and
  Sentence Fluency/Conventions-- assign a score to
  each domain of 1, 2, 3, or 4.

37
The Analytical Scoring Process

• The Process
• Each paper is scored by two readers.
• Agreement Exact matches or adjacent scores .
• A Third Read by the table leader is required for
  each domain if scores are not in agreement.

38
The Analytical Scoring Process

• Rules for obtaining the final score for a student
  after the third read
• If the third score is an exact match to one of
  the original scores, the two matching scores are
  used.
• If the third score is adjacent to one of the
  original scores but not the other, the third
  score and the score adjacent to it are used.
• If the third score is adjacent to both scores,
  the third score is used twice.

39
The Analytical Scoring Process

• Possible impact of analytical scoring
• Information Each student report will include
  four sub scores and one scale score. These
  reports are shared with students and parents.
• Trends Over time, data from this assessment can
  indicate domain strength or weakness at a
  classroom, school, and/or district level.

40
Scoring Practice
41
Think about your dream house.  It could be any
place you wish.       Write an essay describing
your dream house by creating a vivid mental
picture using sensory details for the readers so
that they can see it, experience it, and be able
to understand why you would want to live there. 
8th Grade Prompt
42
My dream house is located . . .

• 8 see top right

43
Analytical Scoring

• Ideas Content 2
• The picture of what is being described is
  somewhat limited some digressions from the
  topic.
• Organization 3
• Intro, body, conclusion are functional pacing is
  controlled transitions functional paragraphing
  is generally organized.
• Voice Word Choice 3
• Wording is generally expressive language
  generally precise adequate vivid words and
  phrases.
• Sentence Fluency Conventions 3
• Sentences generally vary in length or structure
  occasionally the phrasing sounds unnatural
  fragments do not distract the reader grammar,
  usage, punctuation do not distract.

44
Have you ever wanted to see, . . .

• 6 see top right

45
Analytical Scoring

• Ideas Content 2
  
• Limited sensory details limited picture of what
  is being described.
• Organization 2
  
• Structural development of an intro, body,
  conclusion is limited pacing is inconsistent
  transitions are weak.
• Voice Word Choice 1
  
• Wording is inexpressive and lifeless little
  sense of the writer language is neither
  specific, precise, or varied few vivid words or
  phrases.
• Sentence Fluency Conventions 2
• Sentences seldom vary, phrasing occasionally
  unnatural, conventions do not distract the reader

46
Grade 11 Scoring Guide

• The packet you have in your handout represents
  a shortened version of the actual scoring packet
  that raters use when scoring Nebraska papers. In
  the actual raters packet two sample papers are
  included for each score point in each domain,
  totaling 32 papers that are used by raters as
  anchors for the rubric. Also note the rationale
  that is provided for the score on each paper.

47
Grade 11 Scoring Guide

• Ideas/Content 4 example
• The writer conveys a clear opinion regarding the
  benefits of taking a class in Medical
  Terminology. The content is well focused
  throughout the response and the reasoning is
  logical and compelling. The supporting examples
  are numerous and relevant.

48
Grade 11 Scoring Guide

• Organization 2 example
• A limited structural development is evident in
  this essay about the need for a weights class. A
  weak introduction and a brief conclusion are
  present, but the main body is a list of somewhat
  random ideas regarding the benefits of a class,
  which are connected using a few weaker and
  repetitive (also) transitions. Paragraphing
  while present is not successful and pacing is
  somewhat inconsistent.

49
Grade 11 Scoring Guide

• Voice/Word Choice 3 example
• The writer demonstrates a general commitment to
  the merits of a gym class. While the voice is
  appropriate for the purpose and audience, it is
  not particularly enthusiastic or passionate about
  the topic and projects only a generally
  persuasive tone. The language is generally
  specific and engaging and the writer generally
  anticipates the readers questions.

50
Grade 11 Scoring Guide

• Sentence Fluency/Conventions 1 example
• Frequent grammar, usage, punctuation and spelling
  errors distract the reader through this response.
  Sentences seldom vary in length and structure,
  and the run-on sentences confuse the reader.
  Phrasing sounds awkward and unnatural.

51

• What scores would these papers receive in other
  domains?

52
Questions?Nebraska Department of Education
Statewide AssessmentPat Roschewski
pat.roschewski_at_nebraska.govEdward Foy
edward.foy_at_nebraska.gov