Test Design

1
Test Design

• Helen Zsohar, PhD, RN
• Associate Professor (Clinical)
• College of Nursing

2
Issues we will address

• What  kind  of  test  should  you  use?  How  
• do  you  write  a  really  good  multiple
   choice exam?  How  can  you  make  sure  that
   your  exams  test  what  you  had  intended?
   This  
• session  will  provide  you  with  some  Dos  
• and  Donts  of  test  preparation.  

3
So what is the purpose of testing?
http//www.cartoonstock.com/search.asp?actionsear
chxakeywordtestsCategoryEducation2FTeaching
BooleanOrArtistNotSelected
4
Linking testing to goals and outcomes

• Close relationship between the objective of
  instruction and test
• Instructional methods consistent with desired
  outcomes

5
Why do we test?

• Measure performance (may be in cognitive,
  affective or psychomotor realms)
• Provide additional learning opportunity
• Assess teacher effectiveness

6
Best practices in developing tests

• Analyze the domains to be tested
• Develop a test blueprint
• Develop test items
• Test the test items (often done in real time)
• Evaluate effectiveness of the test

7
Example Blueprint on Test Design
Adapted From http//www.utexas.edu/academic/cte/s
ourcebook/tests.pdf
8
Types of Tests

• Selected Response
• Constructed Response
• Performance Response (skills and behavior)

9
Selected Response Exams

• Multiple choice
• Matching
• True/false

10
Multiple choice tests-delivery methods

• Paper and pencil
• Computers (Blackboard)
• Student response systems

11
Organizing multiple-choice exams

• 50 items per hour testing generally conventional
• Place incentive items first and throughout test
  (10 for total test)
• Sort items into logical groupings
• Test banks OK if consistent with instruction
• Be careful about use of humor in items
• Stem and 4 alternatives (one correct)
• Avoid multiple-multiple items

12
Example of Incentive Item

• The pulse rate on a patient is equivalent to
• number of heart beats per minute
• respirations every 30 seconds
• systolic blood pressure
• central venous pressure

13
Example of Incentive Item

• The pulse rate on a patient is equivalent to
• number of heart beats per minute
• respirations every 30 seconds
• systolic blood pressure
• central venous pressure

14
Constructing Multiple-choice Items

• Item stem
• Phrase as question if possible
• Put repetitive information in stem
• Avoid irrelevant information
• Avoid negatively phrased questions

15
Constructing Multiple-choice Items

• Correct response
• Be sure it is correct
• Read with stem to make sure correct
• Put in random position on responses

16
Constructing Multiple-choice Items

• Distractors
• Write plausible and discriminable responses
• Read with stem to make sure correct
• Put in random position on responses
• Make sure of parallel structure
• Avoid all or none

17
Advantages of selected response exams

• Control of sampling of knowledge domain
• Scoring ease
• Versatility
• Objective evaluation of test (item analysis)

18
Disadvantages of selected response exams

• Test recognition rather that recall
• Allow for guessing
• Difficult to construct

19
Constructed Response Tests

• Fill in the blanks
• Essay
• Problem-solving exercise

20
Developing Constructed Response Tests

• Identify method for sampling appropriate domain.
• Identify a format that is conducive to sampling
  the domain
• Write the criteria for evaluation prior to
  administering the test (rubric)
• Write items where integration of background
  knowledge not memorization are the focus.

21
Advantages of constructed response tests

• Relatively easy to construct
• May require a higher level of cognition from
  learners (measure higher-order goals)
• Require higher-order skills from learners (verbal
  and oral discourse)
• Provide good information to instructors regarding
  teaching effectiveness of content.

22
Disadvantages of constructed response tests

• more subjectivity in grading than selected
  response
• take longer to grade than selected response
• style over substance may be inadvertently
  rewarded
• inter-rater reliability may be affected

23
My personal Top 10 List of Dos and Donts in
Test Design

• Dos List
• Congruence between instruction and testing
• Test blueprint prior to test development
• Adequate time to develop test
• Key or rubric developed prior to test
  administration
• Adequate direction for how to make responses

24
My personal Top 10 List of Dos and Donts in
Test Design

• Donts
• Randomly take questions from a test bank
• Make the test too short or too long
• Discuss the test with learners right after the
  test is submitted
• Dont ask irrelevant information on test
• Make typing or syntax errors on test