Michigan Merit Examination

1

• Michigan Merit Examination
• Edward Roeber
• Michigan Department of Education

2
Issues Leading Up to the MME

• High school principals organization felt MEAP
  testing took too long (days and hours)
• MEAP assessment program suffered an operational
  meltdown in 2003
• Although students can receive up to 3,000
  towards post-secondary education, student didnt
  take the MEAP test seriously
• Participation in the MEAP assessment was too low
  - especially in suburban districts

3
Proposed MME Legislation

• Students would take the following tests
• College-entrance test and a work skills
  assessment acceptable to employers, plus and
  Michigan items needed to secure Federal approval
• Testing would occur in the spring of the junior
  year
• Retesting in fall and spring of senior year
• One free retake
• College-entrance readiness tests (PLAN and PSAT)
  used for dual enrollment eligibility purposes

4
Arguments in Favor of MME

• Students will receive a free college-reportable
  score on a college entrance test - this would
  motivate students to participate in and do well
  on assessments
• Students will receive regular work skills scores
  in reading and mathematics - useful in employment
• Michigan Merit Exam scores could still be used to
  award post-secondary scholarship in Michigan
• Lower cost and less testing time

5
Arguments Against MME

• Little standards-based information from a college
  entrance and work skills tests - no data for
  student or school improvement
• Test administration procedures are very rigid
• Can all students score above average?
• Since ACT test scores have not changed much over
  the past decade, how will schools make AYP?
• MME is far more expensive - 73 versus 19/student

6
MME Legislation

• PA 592 of 2004 - MME inserted into Career and
  Technical Preparation Act
• PA 593 of 2004 - Replaced high school MEAP with
  MME in School Aid Act
• PA 594 of 2004 - Replaced high school MEAP with
  MME in Postsecondary Enrollment Options Act
• PA 595 of 2004 - Replaced high school MEAP with
  MME for determining Merit Award Scholarships
• PA 596 of 2004 - Replaced the high school MEAP
  with MME in the State School Code

7
Steps Used to Implement MME

• Visited potential vendors to share ideas
• Competitively bid the MME program
• Reviewed the bids
• Selected new contractor(s) - Pearson/ACT
• Determined the content of each MME test
• Conducted pilot test of 37 high schools
  (statistical alignment)

8
Steps Needed to Implement MME

• Conducted additional content alignment studies to
  select MME items
• Prepared revisions to Michigans Accountability
  Workbook
• Submitted application to USED in July, 2006
• Conducted MME standard setting for NCLB purposes
  in July
• Set PLAN/PSAT cuts for dual enrollment

9
Steps Needed to Implement MME

• November 1, 2006 was the drop-dead date for the
  MME versus MEAP decision for Spring 2007
• Receive approval from USED prior to drop-dead
  date, OR
• continue MEAP High School Assessment (if approval
  is not received by that date)
• If approved by USED, implement the MME in grades
  11 in the 2006-07 school year maintain MEAP HST
  for Grade 12 only

10
Steps Needed to Implement MME

• Was the USED approved the MME?
• November 1, 2006 letter from USED
• . . . It is likely to be fully compliant. .
  However, additional evidence is needed to show
  how Michigans new assessments meet the standards
  and assessment requirements under ESEA.

11
Evidence That Must be Provided

• Evidence of SBE adoption of achievement standards
  for the MME
• Documentation of technical quality indicators,
  except standard setting and materials previously
  submitted, as listed in the Peer Review Guidance
  for the MME as administered in spring 2007.

12
Evidence That Must be Provided

• Final technical manuals for the 2006-07 MME and
  MI-Access assessments.
• Actual MME assessment reports of results at the
  student, school, and state levels that have been
  provided.

13
Soits a Go!

• First MME administration - Spring 2007
• Students to be assessed Grade 11
• All students enrolled in Grade 11 during March
  2007 take the complete Michigan Merit Exam
• Including students who previously took the MEAP
  in grade 10 to qualify for dual enrollment

14
MME Assessment Design

• Grade 10 College entrance readiness test (PLAN
  and PSAT - local school/student decision)
• Grades 11 and 12 Michigan Merit Examination
• English Language Arts - ACT, WorkKeys and a
  Michigan Social Studies item
• Mathematics - ACT mathematics, a few items from
  the ACT Science section, WorkKeys and some
  Michigan items
• Science - ACT test and Michigan items
• Social Studies - all Michigan items

15
MME Assessment Design
Assessments Administered Contributes to an English Language Arts Score Contributes to a Mathematics Score Contributes to a Science Score Contributes to a Social Studies Score
ACT Plus Writing WorkKeys Applied Mathematics Reading for Information And Michigan Mathematics Michigan Science and Social Studies Total ELA Score Reading ACT Reading (MC) WorkKeys Reading for Information items (MC) Writing ACT English (MC) ACT Writing (CR) Michigan Social Studies CR item -writing score ACT Mathematics (MC) WorkKeys Applied Mathematics (MC) Selected ACT Science items (MC) Michigan items (15 MC) ACT Science (MC) Michigan Science (55 MC) Michigan Social Studies (57 MC and 2 CR - social studies score) Note The number of Michigan items may increase in order to field-test new items to replace those released annually.
16
MME Test Organization MME Test Organization MME Test Organization MME Test Organization MME Test Organization    
Day Test Subject Session Parts Total Items or Prompts Testing Time (minutes) Total Admin Time
Day 1 ACT Plus Writing English 5 75 45
Day 1 ACT Plus Writing Reading 5 40 35
Day 1 ACT Plus Writing Writing 5 1 30
Day 1 ACT Plus Writing Mathematics 5 60 60
Day 1 ACT Plus Writing Science 5 40 40
Day 2 WorkKeys Reading for Information 3 33 45  
Day 2 WorkKeys Applied Mathematics 3 33 45  
Day 2 Michigan Mathematics 3 14 20  
Days 2,3 or 4 Michigan Science 1 46 45  
Days 2,3 or 4 Michigan Social Studies 2 32 50  
Days 2,3 or 4 Michigan   2 27 50  
TOTAL Minutes 465
TOTAL hours 7.75
17
Special Populations

• Students with Disabilities
• Students with Section 504 Plans
• English Language Learners

18
Accommodations

• ACT-approved
• Available to students with disabilities and
  Section 504 plans
• Apply only to the ACT
• State-allowed
• Available to students with disabilities, Section
  504 plans, and English language learners
• Apply to all parts of the MME

19
Accommodation Requests

• ACT-approved versus State-allowed
• For the ACT
• 7,934 ACT-approved
• 456 denied
• 1,565 state-allowed materials requested
• For State-allowed accommodations no approval is
  required

20
MME Accommodation Summary Table
21
MME Schedule for 2006-2007

• MME assessment administration dates
• MME administered statewide
• March 13 - ACT
• March 14 - WorkKeys and Michigan Mathematics
• March 14-16 - Other Michigan tests
• Make-up testing scheduled two weeks later
• March 27 - ACT
• March 28 - WorkKeys and Michigan Mathematics
• March 28-30 - Other Michigan tests
• Auxiliary test centers offered for non-public
  students

22
MME Activities for 2006-2007

• September-October, 2006
• Conducted high school video test briefing on
  September 12
• Established 1,111 high schools as test centers -
  public schools, charter schools, and private
  schools
• Schools identified key testing staff
• Test Supervisor
• Backup Test Supervisor, and
• Test Accommodations Coordinator

23
MME Activities for 2006-2007

• November, 2006-February, 2007
• High school Test Supervisor, Backup Test
  Supervisor, and Test Accommodations Coordinator
  training at 12 sites (about 3,400 testing staff)
• Schools apply for accommodations for students
  with disabilities and English language learners
  (both ACT- and state-approved ones)
• ACT reviews accommodation requests and determines
  those eligible for college-reportable or
  state-reportable scores
• Auxiliary test centers set up for non-public
  school students who attend a non-public school
  that was not established

24
MME Schedule for 2006-2007

• February-March, 2007
• Tests (ACT and Michigan) produced
• Accommodations determined and materials ordered
• Non-secure and secure materials shipped to high
  schools
• Students complete the demographic and
  non-cognitive sections of the answer folders for
  the MME
• High school Test Supervisors train room
  supervisors and proctors
• MME assessments administered

25
MME Schedule for 2006-2007

• April-July, 2007
• Students and schools receive reports of results
• Regular ACT reports (college-reportable)
• WorkKeys reports
• MME score reports
• MME scores used for state accreditation and NCLB
  AYP accountability purposes

26
MME Schedule for 2006-2007

• August-November, 2007
• Retake vouchers produced, sent to schools, and
  given to students
• Students register for MME retake via ACT
• Retake on a single fall nationally-scheduled ACT
  testing Saturday administration - fourth Saturday
  in October
• WorkKeys and Michigan tests administered the
  following week in schools

27
MME Issues and Challenges

• Getting USED to approve an ACT-based assessment
  for NCLB AYP purposes
• How much augmentation is needed?
• Non-alignment of the WorkKeys tests
• Alignment of the MME tests to Michigans
  standards - determining needs to be added
• Changing nature of the ACT tests
• Preparing to administer the ACT in Michigans
  schools
• Test center establishment
• Testing staff training
• Accommodations

28
MME Issues and Challenges

• Preparing the IT work for blending multiple
  assessments into a coherent assessment
• Different psychometric models
• Two contractors systems with Michigans systems
• Assuring adequate assessment administration
  preparation - for very secure tests (ACT)
• Moving from an ACT-administered test to Michigan
  educator-administered tests

29
MME Issues and Challenges

• Accommodations for students with disabilities
• What accommodations will be offered and under
  what conditions?
• How much work is it to get accommodations
  approved?
• Will MI-Access need to be modified and/or
  expanded?
• Participation of English language learners
• What accommodations will be offered and under
  what conditions?
• Will any ELL accommodations result in
  college-reportable ACT scores?
• Why cant ELLs use the same accommodations as
  SWDs but not receive college-reportable scores?

30
MME Issues and Challenges

• Working through the logistics of testing
• Multiple (3) answer folders
• Inability to pre-ID students
• Blended reporting of multiple tests
• Alignment of the tests to a states standards
• Working with ACT
• Attempting to modify established procedures
• Electronic registration of students, key testing
  staff and schools
• Updating information on staff and schools
  electronically
• Considering modern psychometric methods
• Blending ACTs procedures with those of states

31
MME Issues and Challenges

• States using ACT have recently formed an ACT
  Users Group
• Identified a number of challenges that states
  would like to work on together
• Identified several issues that states would like
  to work on with ACT
• Next meeting is May 17-18 in Detroit

32
For Questions and Comments

• Edward Roeber
• [phone] voice
• Roebere_at_michigan.gov
• Email Questions to mme_at_michigan.gov
• MME Website www.michigan.gov/mme