MultipleChoice Items

1
Multiple-Choice Items

• Glen Hammond, RRC, CAE, CIE

2
Multiple Choice Items 550

• Are selection/recognition/choice items
• are composed of a question or an incomplete
  statement STEM and a list of alternative
  responses, OPTIONS Incorrect responses are
  DISTRACTERS or FOILS. The correct answer is the
  KEY.

3
Example 1 550

• Which of the following is a selection-type item?
• multiple choice
• extended essay
• completion
• restricted essay

4
Example 2

• The highest level of Blooms Cognitive Taxonomy
  listed here is
• synthesis.
• knowledge.
• application.
• analysis.

5
M.-C. Parts Functions p. 50

• Stem set task or problem to be performed
• Alternatives suggest responses
• Distracters appearing plausible to uninformed
• Key correct answer should be apparent to
  those who know course well

6
Directions 551

• For each of the following stems, select the
  correct or best answer from among the choices
  presented. Place the letter of your choice in
  the blank in front of the question circle the
  letter, underline the answer, mark the
  corresponding letter or space on your answer
  sheet.

7
Considerations 551

• They limit testing to what a student should do
  rather than what one might do if asked to supply
  own response.
• They cannot generally measure problem-solving
  skills. We get the answers, but not the process
  the student has gone through to arrive at the
  answer.

8
Considerations 551

• They cannot test organization and presentation of
  ideas. Use essay for those items
• They cannot test synthesis as cannot any other
  recognition format, as synthesis is an individual
  or group creation.
• They are difficult to construct and make
  believable.

9
M-C may be used to 551

• measure many different learning outcomes.
• present a list of similar choices for
  alternatives.
• ensure one idea only is being tested.
• reducing chances of successfully guessing the
  correct answer .25 as opposed to .5.
• reduce the incidence of patterns of answers as
  might happen in T-F.

10
Use M-C to test 551

• knowledge of facts, methods, terminology,
  principles, procedures, systems, frameworks
• interpretation or comprehension of information
• application of information or generalizations
• analysis or evaluation limited degree

11
M-C Construction Rules 551-2

• Test important learning outcomes.
• Present a single clearly-formulated task in the
  stem of the question.
• State the stem in clear, simple language.
• Put as much of the wording as possible in the
  stem.
• State the stem in the positive form, whenever
  possible.

12
M-C Construction Rules

• Emphasize NEGATIVE PHRASING whenever it is used
  in the stem. Preferred form is All of the
  following are synonyms for curriculum EXCEPT ---.
• Ensure that the intended answer is correct or
  clearly the best answer.
• Make all alternatives grammatically consistent
  with the stem of the item and parallel in form.

13
M. C. Construction Rules

• Avoid verbal clues that might enable students to
  select the correct answer or to eliminate an
  incorrect alternative.
• similarity of wording in both the stem and
  correct answer.
• stating the correct answer in textbook language
  or stereotyped phraseology.
• including absolute terms in the distracters
  never, always, totally, without exception.

14
M. C. Construction Rules

• including two responses that are all-inclusive
  all of the above or none of the above.
• including two responses that have the same
  meaning.

15
M. C. Construction Rules

• Make the distracters plausible and attractive to
  the uninformed
• Use common misconceptions as distracters.
• State the alternatives in the language of the
  student.
• Use good-sounding words accurate, important,
  key, etc. in the distracters as well as in the
  correct answer.
• Make the distracters similar to the correct
  answer both in length in complexity of wording.

16
M. C. Construction Rules

• Use extraneous clues in the distracters, such as
  stereotypical phrasing, scientific sounding
  answers, and verbal associations with the stem.
• Make the alternatives homogeneous, but in doing
  so beware of fine discriminations that are
  educationally significant.

17
M. C. Construction Rules

• Vary the relative length of the correct answer to
  eliminate length as a clue.
• Avoid using the alternative all of the above
  and use none of the above with extreme caution.
• Control the difficult of the item problem in the
  stem, changing the alternatives, or increasing
  the homogeneity of the options.

18
M. C. Construction Rules

• Vary the position of the correct answer in random
  order. You may do this by opening a book
  anywhere and using the numbers on the right as a
  guide 1 A 3 B 5 C 7 D 9 E.
• Ensure each item is independent of the other
  items in the test.

19
M. C. Construction Rules

• Use an efficient item format alternatives listed
  below each other letters rather than numbers for
  indicating different alternatives if stem is a
  phrase, begin alternatives with lower-case
  letters.

20
Uses Observing Directly

• discriminate and make correct choices
• comprehend concepts, principles, and
  generalizations
• make judgments about choices
• infer reason
• compute
• interpret new information
• apply information knowledge

21
Only Minimally Useful for Inferring

• recall with minimum prompts
• producing expressing unique ideas
• articulating explanations and giving examples
• organizing personal thoughts
• displaying thought processes or patterns of
  reasoning

22
Advantages of

• versatility in assessing a variety of
  instructional objectives
• reduction of opportunities for bluffing or
  dressing up the answer
• focus on reading and thinking and not on writing
  under examination conditions
• reduction of chances for obtaining correct answer
  by blind guessing

23
Advantages of

• utility in obtaining diagnostic information
  when distracters are based on common student
  errors and misconceptions.

24
Criticisms of

• no opportunity to express own thoughts
• items may focus on trivia
• penalize those who see outside the box
• reduction but not elimination of guessing
• belief that they lead to standardized
  conceptions -gt oversimplification of ideas

25
Not Recommended
for

• problems or questions requiring a single-word or
  single-number answer
• computational problems in general
• questions with only two plausible answers
• items for which writing the answer doesnt take
  any longer than marking an answer to a
  multiple-choice item.

26
Criticize this item

• 50. multiple choice
• An undetailed example is a. always used singly,
  b. ineffective, c. used to generate a sharing
  experience with the audience, d. usually lengthy.

27
Key

• unnecessary to label each item as to type
• incomplete stem no clear task or problem
• avoid tandem side-by-side arrangement use
  parallel one below other format
• watch exact phrasing in textbook a sharing
  experience capitalized on clang expressions and
  rote memory rather than understanding and
  comprehension

28
Key

• verbal clues undetailed can be lengthy
• correct answer c is longer than the rest of the
  items, thus providing a clue
• specific determiner in a make that item easy to
  eliminate

29
Key

• heterogeneous not homogeneous options
• how examples are used
• judgment about desirability
• function of examples
• characteristics of an example

30
Word Distracters So That They

• permit those with the minimum desired degree of
  knowledge to select the correct answer
• do not permit those who lack the prerequisite
  knowledge to select the correct alternative on
  the basis of testwiseness or superficial clues.

31
Making Distracters Plausible

• Use the students common errors.
• Use important-sounding words significant,
  accurate that are relevant to the item stem.
• Use words that have verbal associations with the
  item stem political, politician.
• Use textbook language or other phraseology that
  has the appearance of truth.

32
Making Distracters Plausible

• Use incorrect answers that are likely to result
  from student misunderstanding or carelessness
  forgets to convert from one unit to another.
• Use distracters that are homogeneous and similar
  in content to the correct answer e.g., all
  distracters are capital cities, prime ministers,
  etc.

33
Making Distracters Plausible

• Use distracters that are parallel in form and
  grammatically consistent with the items stem.
• Make the distracters similar to the correct
  answer in length, vocabulary, sentence structure,
  and complexity of thought.

34
Key Understanding

• Distracters should distract the uninformed, but
  they should not result in trick questions that
  mislead knowledgeable students Avoid inserting
  NOT in a correct answer to make it a distracter!.

35
Key Requirements

• Note

Parallel in form Homogeneous in content
36
Parallel in form Homogeneous in content
37
Assignment

• 10 regular multiple-choice. Try to ensure that
  some are at the higher levels of cognition.