Deborah Swensen, Ph. D. Assessment Director

1
Deborah Swensen, Ph. D. Assessment Director

• Assessment Directors
• November 13, 2008

2
UALPA

• Constructed Reponses will be scored by contractor
• Districts must meet the deadlines
• Problems from last year

3
AMAOs

• Accountability
• 2006-2007 AMAO II and III
• 2007-2008 AMAO I, II, and III
• AMAO I
• New calculations
• AMAO II
• AMAO III

4
AMAO I

• Numerator All ELL students who moved
• Denominator All ELL students with 2 score
  points on UALPA
• 7 ways to make progress
• Between proficiency levels
• With E and I (low and high)
• Maintaining A
• 2008 29
• 2009 32.5
• Every year after increase 2.5

5
AMAO II and III

• AMAO II
• Increase for 2007-2008 school year
• 2006-07  20.1 (K-6) 26.2 (7-12)
• 2007-08  24.9 (K-6) 30.7 (7-12)
• AMAO III
• 2008-2009 another increase in AMOs
• Possible change in AMO target

6
USTAR

• Under the direction of Curriculum
• Curriculum is gathering additional information
• Configurations of schools/programs
• Windows of instruction
• Number of students
• Courses being taught

7
Trainings

• Utilize data available
• Cognos
• Access data for CRT tests and enrollment numbers
• Drill down to school, grade, and demographic
  levels
• Export the information to PDF or Excel
• Look-up tables for UALPA and UBSCT
• Available to schools and districts

8
Nolan FawcettUBSCT Secondary Math Specialist
9
Printing Problem From October Test
10
What Happened!!!!!

• After USOE approval Manuals are sent to
  contractor for publication.
• Contractor put them through a pre-flight
  process that looks for problems with font or
  graphics issues.
• Manual was suppose to be checked by contractor or
  changes introduced by pre-flight
• This review was missed in the process.

11
Letter from Measured Progress
12
Dates for Results

• November 21st AD Preliminary Results
• (Remember this is Rough Data Results)
• November 24th Preliminary Results to Schools
• December 15th Slice File and Fix-it File
• January 6th Paper Reports
• Late November 2008 Release Oct Test

13
Preliminary Results Website

• Website is a secure system with usernames and
  passwords assigned at the district and school
  level.
• Passwords, a user guide, and Helpdesk support
  information was distributed to District
  Assessment Directors at the end of November.
• Measured Progress was responsible for supporting
  and maintaining the website. http//iservices.meas
  uredprogress.org

14
Getting Your Results

• Using Internet Explorer, navigate to 
  http//iservices.measuredprogress.org
• Once at the web page, select Utah Basic Skills
  Competency Test from the drop-down box and click
  Enter.
• Click on the View UBSCT Preliminary Results link
  on the left of the page.

15
Results Continued

• On the Log In screen
• Enter your districts user name in the User Name
  box (your districts user name is your 2-digit
  district code for example, Alpine districts
  user name is 01).
• Then, enter your secure password in the Password
  box. 
• If you have any questions or need assistance,
  please contact the UBSCT Helpdesk,
  1.866.502.2742.

16
Preliminary Results Website Screen Shots

• Login Screen

17
You will be directed to a screen labeled,
Home.  After reading the text on this page,
click on Reporting on the menu at the top of the
page.

• Welcome Screen

18
In the section labeled, Basic Filtering
Options, select the school, grade, and subject
for which you would like to view student
information. (Please note that only results for
reading and mathematics are available.)

• Reporting
• Screen

Filter Options
Report File Download Options (PDF, Excel, CSV)
19

• After you have made a filter selection, select
  your preferred download file type. To do this,
  click on the icon for Adobe PDF, Excel, or CSV. A
  file will be created in the format you have
  selected. Student records in the file will appear
  in alphabetical order by school.
• Repeat the three steps above to view student data
  records for other schools, grades, and subtests
  within your district by changing your selections
  in the Basic Filtering Options section.

20

• For more information about the data provided on
  the roster reports, click on Documentation on the
  menu at the top of the page. You will find a PDF
  file for reading and mathematics.

21
Preliminary Results

• EMAIL PROBLEMS
• We had several locations block access to Measured
  Progresss emails.
• Please make sure that you fix the problem so that
  the following emails can come through.
• Measured Progress
• Andrea Chan - [phone] ext 2298 E-Mail
  chan.andrea_at_measuredprogress.org
• Tinamarie Skillings [phone] ext 2146
  E-mail skillings.tinamarie_at_measuredprogress.org

22
Calculator Reminder

• Algebra 2 students will need to have their own
  graphing calculator for the CBT.

23
Rita Brock ELL Specialist

• UALPA 2008-09

24
UALPA 2008-2009 Administering Utahs Academic
Language Proficiency Assessment
25
(No Transcript)
26
(No Transcript)
27

• Overview What, Why, Who, When
• District Assessment Coordinators and Test
  Administrators Roles Responsibilities
• III. Structure and Format of the Assessment
• Test Administration Procedures
• After Testing

28
I. UALPA What, Why

• Statewide assessment of all identified ELL/LEP
  students
• Mandated by the No Child Left Behind Act of 2001

29
Who

• Students to be tested include
• All newly enrolled LEP students
• All LEP students whose proficiency level was
  Advanced or below, as measured by the 2007-2008
  UALPA

30
Who

• Kindergarten Testing with UALPA
• Only test those students identified as English
  Language Learner students with UALPA

31
When

• Testing window
• January 12, through May 8, 2009
• (for both traditional and year-round schools)
• ELLs should be tested annually until they are
  determined to be at the Fluent proficiency level.
  

32
District Testing Schedules

• January 12 through March 2nd 2009
• Reports will be returned in April
• March 2nd through May 8th 2009
• Reports will be returned in June
• New students enrolled AFTER April 24, 2009 DO NOT
  need to be tested on the UALPA

33
II. Roles and Responsibilities of the District
Assessment Coordinator the Test Administrator
34
District Assessment Coordinator

• Before testing
• Creates master list of LEP students to be
  assessed.
• Orders test materials inventories and confirms
  materials received.
• Selects and trains Test Administrators using
  Testing Ethics policy material required by the
  Utah Board of Education. Information and forms
  can be found at http//www.schools.utah.gov/arc/AA
  EthicsPolicy.pdf
• Trains Test Administrators on the administration
  of the UALPA
• Orders answer documents and pre-printed barcode
  labels, as needed.

35
Pre-printed Student Barcode Labels

• District submits a pre-print file to USOE
  Computer Services at least 10 working days prior
  to date the student barcode labels and answer
  documents will be needed.
• Before this is done, ELL students will need to
  have been identified as receiving Level 1
  Beginner or Level 2 -Intermediate.
• Test Administrators will apply the labels to the
  answer documents of those students they are
  assessing.

36
New students without labels
For new students without pre-printed labels

• Student information will need to be hand bubbled
• An SSID number must be assigned before UALPA
  administration

37
District Assessment Coordinator

• During testing
• Be available to answer Test Administrator Site
  Test Coordinator questions.
• Distribute additional materials to schools if
  needed.

38
Test Administrator

• During testing
• Administer the one-on-one Speaking tests first.
• Administer the other 3 tests in a group setting.
• Continually monitor students.
• Maintain and oversee the security of all test
  materials.
• Arrange make-up testing sessions for absentees.

39
IV. Test Administration Procedures A.
General
40
Test Site

• Individual Testing
• Quiet one-on-one environment
• Seating
• Group Testing
• Quiet room
• Do Not Disturb sign on door
• Desks must be cleared

41
Test Security

• Responsibility of both the District Assessment
  Coordinator and Test Administrator
• All test materials must be accounted for and
  returned to the District Assessment Coordinator.

42
Testing Ethics

• No questions from the UALPA may be taught to or
  reviewed with students.
• Copying any test material is prohibited.
• All Test Administrators must be trained under the
  direction of the District Assessment Director.

43
Grouping Students for Testing

• As long as the group is not too large, all
  students within a grade span, taking the same
  form may be tested together in a group setting.
• Group size
• Optimal group size 10
• Maximum group size 20
• Take into account the maturity of the students,
  the number of available proctors, and the
  acoustics of the room.

44
Prompting

• In general, prompting is not allowed.
• Exceptions
• To clarify a students response
• If student responded in another language
• Test Administrator may repeat a question if
• There was a distraction or interruption
• Student did not yet begin to respond and asks for
  question to be repeated

45
Translating Directions

• No item directions or item content may be
  translated. The script must be read in English
  exactly as printed in the Test Administration
  Manual.

46
?
Timing

• The UALPA is an untimed test.
• During individual testing, Test Administrators
  should wait approximately 15 seconds for a
  student to begin a response.
• During group testing, Test Administrators should
  use their best judgment in allowing sufficient
  time for students to finish multiple-choice and
  extended responses.

47
Frustration Level

• Frustration level is defined as when a student
  has made 5 consecutive scores of 0 or is
  obviously unable to respond to test prompts.
• If frustration level is reached on a Form 1
  Speaking test, administer all other tests
  individually.
• If frustration level is reached on a Form 2
  Speaking test, then switch to a Form 1 test.

48
Frustration Level

• If frustration level is reached on any test, mark
  the FL bubble in the Office Use Only box on the
  answer document.
• Determination of frustration level cannot be made
  during group testing.

49
Accommodations

• Decisions regarding accommodations must be made
  ahead of time by the IEP, ELL or Section 504
  team and documented in the students file.
• Test Administrators and/or teachers should not
  make accommodation decisions at the time of
  testing.
• Braille and Enlarged Print versions of the test
  are available, but must be ordered 2 months in
  advance.

50
?
Non-allowable Accommodations

• Certain accommodations are not allowed during
  the UALPA. These include
• Reading directions or questions aloud to a
  student (except as indicated by the Test
  Administrator script).
• Translating directions or questions for a
  student.
• Providing bilingual word lists to a student.

51
Scoring Guides

• Oral responses on the Speaking test are scored by
  Test Administrators at the time of testing.
• Responses are rated using the Scoring Guides in
  the flipchart.
• The Test Administrator must study the Scoring
  Guides before giving the test for the first time.

52
Pre-printed Student Barcode Labels

• District submits a pre-print file to USOE
  Computer Services at least 10 working days prior
  to date the student barcode labels and answer
  documents will be needed.
• Before this is done, ELL students will need to
  have been identified as receiving Form 1 or Form
  2.
• Test Administrators will apply the labels to the
  answer documents of those students they are
  assessing.

53
New students without labels
For new students without pre-printed labels

• Student information will need to be hand bubbled
• An SSID number must be assigned before UALPA
  administration

54
Testing Absentees

• All LEP students should be administered all four
  tests.
• If a student is absent for a particular testing
  session, schedule a make-up test for that student
  as soon as possible.

55
Kindergarten Test Administration

• All four tests are administered individually
• All responses are recorded by the Test
  Administrator on the students Form A answer
  document

56
Grade-Span 1-2 Test Administration

• The Speaking test is administered individually
  using the Speaking test flipchart, and is always
  administered first.
• Speaking responses are recorded by the Test
  Administrator on in the consumable booklet.
• The other three tests are administered to small
  groups ideally of not more than 10 students, with
  students marking their answers in the consumable
  booklet.

57
Grades 3-12 Test Administration

• Speaking Test is administered individually using
  the flipchart, and must be administered first.
  Speaking responses are recorded by the Test
  Administrator on the students answer document
• The other 3 tests are group administered (20
  student maximum) using the script in the Test
  Administration Manual.
• These responses are marked or written on the
  student answer document by the student.

58
V. After Testing
59
Test Administrator

• After testing
• Checks answer documents for completeness proper
  bubbling.
• Checks to see if any applicable testing
  accommodations are marked.
• Organizes test materials.
• Returns test materials to the District Assessment
  Coordinator.

60
District Assessment Coordinator

• After testing
• Organizes scoring of constructed responses.
• Returns completed answer documents to USOE.
• Collects and securely destroys all 2008-2009 test
  materials at end of school year.

61
Returning completed answer documents to USOE

• Verify that all answer documents have had the
  constructed responses scored.
• Separate each grade within boxes by
  teacher/school header sheets.
• Use packing material to avoid movement of
  materials inside box. Do NOT use foam as this
  affects scan ability of documents.
• Tape box securely with heavy-duty tape (not
  cellophane or masking tape).
• Ship to USOE via UPS, FedEx, or other carrier
  that tracks shipments. Do not send by U.S. Mail.

62
Key Dates for UALPA 2008-2009

• January 12, 2009
• Test materials shipped to districts
• January 12 May 8 2009
• Testing Window
• February 15th 2009, May 1st 2009
• Deadline for Districts to send Constructed
  Responses to USOE

63
Key Dates for UALPA 2008-2009

• District Deadlines for return of answer
  documents to USOE
• March 2, 2009 (50) 1st Scoring
• May 8, 2009 (50) 2nd Scoring

64
Reports returned for answer documents submitted
for 1st and 2nd scoring window

• April 13 (approximately) for 1st scoring window.
• June 22 (approximately) for 2nd scoring window

65
If you have questions...
66
SHARON MARSH COMPUTER SERVICES

• Raw Reports and Error Correction

67
Pre-Print UALPA Uploads The How- Who- When

68
The How
69
Obtaining a User ID and Password

• LEAs must submit a New User Agreement Form to
  USOE in order to request a user ID. The New User
  Agreement form must be filled out and faxed to
  Sharon Marsh at
• [phone]. After checking the contents and
  forms for accuracy, the administrator will create
  a user ID for the requestor

70
User ID And Password Format

• Your User ID is made up using your LEA number,
  first initial, last name.
• The password for the first time that you log on
  to the testing web-site is Password1
• Passwords must meet the following criteria
• Be at least 8 characters in length
• Contain at least 1 number (0-9),
• Contain at least 1 uppercase letter (A-Z),
• Contain at least 1 lowercase letter (a-z)

71
Logging into the Assessment Website

• https//cs.schools.utah.gov/assessment
• First screen is the Login screen
• Example User ID
  99smarsh
• Example Password
  Password1
  

72
Logging into the Assessment Website

• https//cs.schools.utah.gov/assessment
• When you look at the URL and see the
  //cs.schools.utah.gov/assessment
• You can tell that you are logged onto the testing
  website and not the UTOS site for ordering test
  materials.
• Because we store student level data we dont
  identify our login page further.
  

73
Login Page
99smarsh
Password1
74
Home Page
Links to various parts of website and contains a
Calendar of Events
75
Support Page
76
Who Do I call For Help?
Call Sharon 538-7915
76
10/30/2009
Assessment Directors Meeting
77
If Locked Out Of Testing Website Dont Call Help
Desk Call Sharon Marsh 538-7915 or Your Data Tech
78
Before Uploading

• Before uploading a file to the Assessment system,
  the file content should be named as outlined the
  Preprint File Name Specification. Files that fail
  validation or are named improperly will NOT be
  accepted by the website. The Preprint File
  Validation Application and the Preprint File
  specification is on the Support page.

79
Steps for Uploading Files

• Create an LEA Preprint file for each test
• Use the provided Preprint File Validation
  Application tool to validate the files contents.
• Login to the Assessment website using a valid
  user ID and password.
• Click on the Upload File link on the Home page
  under File Maintenance section. The upload File
  screen will appear.

80
Upload File Screen
This is in your handout
81
Selecting Your File

• Click on the Browse button to open the choose
  file window then navigate to where you have your
  Preprint file.
• After locating the correct Preprint file, select
  it and click on Open
• The Preprint File that was selected along with
  its path, should now appear in the File Name box.

82
Locating Correct Preprint File
This is in your hand-outs
83
Review Your Files After they Are Sent
84
Processing Files With Errors

• If errors were found during validation, a status
  of Failed will appear next to the file name. The
  following options are available for files with
  errors.
• Click on the filename with status of failed
• View errors on the file by clicking on the File
  Summary button. Some of the file errors will be
  displayed. The user may print these out.

85
Processing Files With Errors
86
Processing Files With Errors

• Cancel the Failed file to put it at the bottom of
  the file list.
• Files with errors must be fixed by the LEA and
  re-uploaded for processing.
• Most errors are SSID miss-matches.

87
The Who
88
Which Students Test For UALPA?

• All students that are A, P, E, or I
• They will be flagged in your SIS systems so that
  when you create you files to upload you should be
  able to upload only those students.

89
Which Students Test For UALPA?
A Advanced PPre-Emergent EEmergent
IIntermediate
90
Testing Levels

• Kindergarten
• Grade 1-2
• Grade 3-6
• Grade 7-8
• Grade 9-12

91
The When
92
When Do I Send My Files?

• If the test window for UALPA is in January you
  will want to start building your files and
  sending them in 10 days before your want to test.
  
• Because all of the documents for UALPA are
  multiple pages you will receive bar code labels
  instead of having the students information
  printed on the answer document.
• Please make sure that your teachers place the
  labels on the answer documents within the
  brackets

93
Will I Receive Blank Documents?

• Because we know that you might have new students
  that move in after you have uploaded your files
  we will provide you with a few blank forms to
  use.
• When filling in the student information use the
  local LEA number and not the SSID number.

94
More Information

• In our December training I will have more
  information on new answer documents.
• Last minute information on pre-print uploads
• Changes if any

95
QUESTIONS
96
Margaret Young DWA Specialist

• Direct Writing Assessment2008-09 Administration

97
DWA

• Scoring rubric and student response examples
  www.schools.utah.gov/assessment/info_dwa.aspx
• One prompt for 6th, one prompt for 9th
• Ensure students have as much time as needed
  during the one administration session (untimed
  assessment).
• Assessment window is determined by school closing
  date.

98
DWA

• Closing date DWA window
• May 20-22, 2009 February 9-27, 2009
• May 26-29, 2009 Feb. 16-March 6, 2009
• June 2-5, 2009 Feb. 23-March 13, 2009
• June 9-12, 2009 March 2-20, 2009
• If your schools closing date is not listed
  above, please contact Margaret Young at
  margaret.young_at_schools.utah.gov.

99
DWA

• 2 lead pencil, no pens
• Districts are responsible for transcribing onto
  the answer document all Braille and other
  accommodated responses.
• For ordering information, please contact Sarah
  Moore at sarah.moore_at_schools.utah.gov.

100
Julie Quinn CBT Specialist

• CBT Update

101
CBT Participation Projections

• Final CBT participation numbers are due December
  12, 2008.
• 2009 Projections as of November 10
• 117,000 more tests than 2008
• (approx. 62 of eligible tests)
• and rising.

102
December Trainings

• Please RSVP as soon as possible,
• to ensure availability.
• Available slots will be released December 4,
• for additional individuals from your LEA.
• Many LEAs have not yet registered.

103
Load Testing

• Change in statewide load testing plan
• January 19 through January 30
• Simulate the heaviest two weeks of testing
• Please have every student possible take training
  tests during this window
• Email will be sent asking for when you will plan
  to participate, to balance loads

104
Load Testing

• Local load testing is critical.
• Mid-December through early January
• Focus on T1 schools and less robust wireless
  networks
• Any problems that impact the CBT projections must
  be relayed immediately (paper testing order
  impact)

105
Questions? Concerns?
106
Sarah Moore Production Manager

• Production / Distribution

107
Utah Test Ordering System

• UBSCT February
• DWA
• UALPA
• CRT

108
Trimester / Semester CRT

• No Algebra 2 available for trimester/semester
  
• assessments
• Ordering closes November 30th

109
UBSCT February

• Ordering closes DECEMBER 20th
• 10th graders, 11th and 12th grader
• retakes

110
Direct Writing Assessment

• Ordering Closes January 15th
• 6th and 9th grades
• Materials to Order
• Student Directions
• TAMS

111
Utah Test Ordering System

• CRT / CBT Spring
• CBT currently closed
• PBT projections scheduled to pre-populate within
  UTOS December 1st

112
Pre-Populate?

• Based off last years test totals in each
  grade/subject
• Minus current CBT projections
• Review December 1st

113

• Call With Questions Regarding UTOS
• pre-population!
• We can reopen for you to alter counts if needed.