RFPs for Online Testing

1
RFPs for Online Testing

• EIMAC, October 2006

2
Some Key Policy Decisionsthat may impact cost

• Ease of use
• Uniqueness
• Reliability
• Volume
• Scope
• Timeline
• What is essential vs nice to have?

3
Additional Staffing Requirements

• 1 Project Management Institute (PMI) Certified
  manager or a project manager with a minimum 5
  years experience on projects exceeding 1m,
• 1 senior network engineer with at least 3 years
  experience in routing and with Tier 1 and Tier 2
  internet service providers.
• 1 senior application developer with at least 7
  years application development experience and also
  at least 3 years experience with the programming
  language's used by Contractor.

4
Global Considerations

• Delineate what is currently available versus
  might be available
• Vendor should have experience or subcontract with
  another with each aspect of the process

5
Technical Objectives

• Orient around current infrastructure available in
  schools to support (i.e. T1 lines)
• Scalability (do you expect to be testing more in
  the future)
• Availability? 24 x 7?
• Reliability fail over systems? Backup data,
  backup power. Disaster recover plan with a
  timeframe (i.e. 24 hours)
• Security of facilities (i.e. intrusion), virus,
  hacking and denial of service attacks.

6
Technical Objectives

• System development and life cycle (i.e. change
  management, version control)
• Number of concurrent users?
• Any controls in the system to prevent overage?
• What are the repercussions of going over?
• Beware of words like user
• Just English or other languages? Has database
  implications
• Item types supported?
• ADA accommodations (i.e. zoom, read aloud text)

7
Technical Specifications

• Must it be a browser or can it be some other
  medium of connection (i.e. java)
• Will browser be secure. If so, how secure is
  secure?
• What platforms (mac, windows, Linux)?
• What versions must be supported (i.e. windows 98,
  mac os 9.2) and What is the process for ending
  support for OS? Learn from the Mac Intel
  Incident!
• What is the minimum CPU and Memory required and
  what is the process for increasing
  specifications?

8
Item and Test Design

• Importing items into the system
• Transferring from another source
• Inputting new items
• Editing Items (should have development and
  production sites)
• Process for field testing
• Process for reviewing tests
• Test attribute information/reports required?

9
Test Administration

• Can test be interrupted?
• Can students move backward and review answers?
• Can they access the same items more than once?
• What happens if they access the wrong test?

10
Data and Scoring

• Hosting versus Scoring
• Student IDs?
• When will results be available?
• Which site will contain the official reports?

11
Data and Scoring

• Who has access to the data and how will security
  roles be established
• Can teachers assign students to classes. If so,
  how?
• What is the format of the item and examinee data?
• FERPA compliant of course..

12
Training and Support

• Training for the SEA
• Training for the LEAs and Schools
• Help desk support (tier 1)? tier 2?
• User manuals and reference guides
• Sample tests
• FAQs
• Technical manual describing application functions

13
Remedies ?
14
Implementation Plan

• Step I - Project Planning and Fit/Gap Analysis
• Step II - System Prototyping and Application
  Development
• Step III - System Implementation
• Step IV - Post-Implementation Support.

15
Evaluation

• Establish committee with sufficient expertise
• Review Proposals
• Review demonstration
• Item loading
• Item/test psychometrics
• Test design
• Login and error recovery
• District and State Administration screens
• Test Administration
• Reporting
• Use pc with minimum specifications and all
  expected operating systems