Constructing Test Items

1
Constructing Test Items
2
MULTIPLE CHOICE ITEMS
A multiple-choice item consists of one or more
introductory sentences followed by a list of two
or more suggested responses, also called
alternatives, choices, and options. These
responses should be arranged in a meaningful way
(logically, numerically, alphabetically, etc.).
3
Stem part of the item that asks the question
instead of asking a question, it may set the task
a student must perform or state the problem a
student must solve.

Sample question Who invented the sewing machine?
a.      Fulton c. Singer b.      Howe d. White
4
Keyed alternative the correct answer or best
answer among the alternatives also called keyed
answer, or simply key.  

The remaining incorrect alternatives are called
distractors or foils.
5

• Varieties of Multiple-Choice Items
•  
• The correct-answer variety
• Which of the following, combined with the
  revolution of the sun, causes the seasons?
• A. The frequency of sunspot occurrences
• B. The gravitational pull of the moon
• C. The intensity of light emitted by the sun
• D. The tilt of the earths axis
•  

6
(2) The best-answer variety What was the basic
purpose of the Marshall Plan? A.  militarily
defend Western Europe B.  reestablish business
and industry in West Europe C.  settle
United States differences with
Russia D.  directly help the hungry and homeless
in Europe
7
(3) The multiple-response variety What factors
are principally responsible for the clotting of
blood? A. contact of blood with a foreign
substance B. contact of blood with injured
tissue C. oxidation of hemoglobin D. presence of
unchanged prothrombin
8
(4) The incomplete statement variety The new
president of Indonesia is A. Susilo Bambang
Yudhoyono B. Gloria Macapagal-Arroyo C. Thaksin
Shinawatra D. Abdullah Ahmad Badawi
9
 (5) The negative variety Which of these is
NOT true of viruses? A. Viruses live only in
plants and animals. B. Viruses reproduce
themselves. C. Viruses are composed of very
large living cells. D. Viruses can cause
diseases.
10
(6) The combined response variety In what order
should these sentences be written in order to
make a coherent paragraph? I.    A sharp
distinction must be drawn between table manners
and sporting manners. II.   This kind of
handling of a spoon at the table, however, is
likely to produce nothing more than an angry
protest against squirting grapefruit
juice about. III.   Thus, for example, a fly ball
caught by an outfielder in baseball or a
completed pass in football is a subject for
applause. IV.   Similarity, the dexterous
handling of a spoon in golf to release a
ball from a sand may win a championship
match. V.    But a biscuit or a muffin tossed and
caught at the table produces scorn and
reproach.   A.      I,
II, III, IV, V B.      I, III, V, IV,
II C.      I, V, III, IV, II D.      II, V,
IV, III, I
11
ADVANTAGES OF MULTIPLE-CHOICE ITEMS

• Major strength discriminatory power (content
  mastery)
• Minimize the opportunity for less knowledgeable
  students to bluff or dress-up their answers.
• The distractor a student chooses may give a
  diagnostic insight into difficulties the student
  is experiencing.

12
DISADVANTAGES OF MULTIPLE-CHOICE ITEMS

• Poorly written multiple-choice items can be
  trivial and limited to factual knowledge.
• Students must choose from among a fixed list of
  options rather than creating or expressing their
  own ideas.

13
ALTERNATIVE-RESPONSE ITEMS
Sometimes, or commonly known as true-false
items Items consist of a declarative statement
that the student is asked to mark true or false,
right or wrong, correct or incorrect, yes or no,
fact or opinion, and agree or disagree. Random
guessing sometimes called blind guessing, which
by definition is random responding.
14
VARIETIES OF ALTERNATIVE-RESPONSE ITEMS
1) True-False Variety presents a proposition
that a student judges true or false.   Example
The sum of all the angles in any four-sided
closed figure equals 360 degrees. T
F   2) Yes-No Variety asks a direct question,
to which a student answers yes or no.   Example
Is it possible for a presidential candidate to
become President of the Republic of Indonesia
without obtaining 50 of the votes cast on the
election day? Y N
15
VARIETIES OF A/R ITEMS
3) Right-Wrong Variety presents a computation,
equation, or language sentence that the student
judges as correct or incorrect (right or
wrong)   Examples Example assessing an
arithmetic principle 5 3 x 2
16 R W Example assessing grammatical correctness
Did she knew who he was? C I
16
VARIETIES OF A/R ITEMS
4) Correction Variety requires a student to
judge a proposition, as does the true-false
variety, but the student is also required to
correct any false statement to make it
true.   Example Read each statement below and
decide if it is correct or incorrect. If it is
incorrect, change the underlined word or phrase
to make the statement correct.   The new
lecturer, who we discussed with just now, hails
from Philippines. C I
17
VARIETIES OF A/R ITEMS
5) Multiple True-False Variety looks similar to
a multiple-choice item however, instead of
selecting one option as correct, the student
treats every option as a separate true-false
statement, and each choice is scored as a
separate item.   Example Under the Bill of
Rights, freedom of press means that
newspapers 1. have the right to print anything
they wish without restrictions. T F 2. can be
stopped from printing criticisms of the
government. T F 3. have the right to attend any
meeting of the executive branch of the federal
government. T F
18
ADVANTAGES OF A/R ITEMS

• They are relatively easy to write.
• 2. They can be scored easily and objectively.
• 3. They can cover a wide range of content within
  a relatively short period.

19
DISADVANTAGES OF A/R ITEMS

• Can be ambiguously worded
• 2. Can be answered by random guessing
• 3. May encourage students to study and accept
  only oversimplified statements of truth and
  factual details

20
DEVELOPING MATCHING TYPE ITEMS
Matching item requires the students to use some
association criterion in order to match the words
or phrases that represent ideas, concepts,
principles, or things. A two-column format is
commonly used for the matching item, although
variations can be used.
21
DEVELOPING MATCHING TYPE ITEMS
The two columns of matching item are usually
called the premises (which consist of the item
stems) and the responses (which consist of the
available options). The entire set of premises
and responses comprises the item. Matching
items should contain homogenous content so that
all responses must be considered plausible
answers. Do not mix dissimilar items as persons
and places in a single exercise.
22
DEVELOPING MATCHING TYPE ITEMS

• Limit a matching exercise to not more than 10-15
  items. Longer items require too many comparisons
  and the task becomes tedious and time consuming.
  Shorter items also tend to have more homogenous
  content.
• Matching exercises may have more responses than
  premises, more premises than responses, or an
  equal number of each (perfect matching type).

23
ADVANTAGES OF M/T ITEMS
1. Space-saving, compact, and objective way to
assess a number of important learning targets
(e.g., ability to identify association or
relationships between two sets of things) 2.
Assess ability to identify relationships and
classify things
24
DISADVANTAGES OF M/T ITEMS
1. Format seems to be limited to the assessment
of memorized factual information 2. Difficult in
trying to find homogenous premises and responses
25
DEVELOPING SHORT-ANSWER ITEMS
Short-answer items require a student to respond
to each item with a word, short phrase, number,
or symbol. Sometimes called supply-type test
items
26
GENERIC TYPES 1. Question asks a direct
question Sample item How many microns make up
one millimeter? (1,000) 2. Association consists
of a list of items or a picture for which
students have to recall numbers, labels, symbols,
or other terms Sample item On the blank next to
the name of each chemical element, write the
symbol used for it. Barium Ba Chlorine Cl 3.
Completion requires a student to add words to
complete an incomplete statement Sample item The
capital city of Malaysia is ____________.
27
ADVANTAGES OF SHORT-ANSWER ITEMS

• Easy to construct
• Can be scored objectively efficient in terms of
  testing time and scoring time
• Reduces guessing (do not prevent students from
  attempting to guess the answer the items only
  lower the probability of guessing correctly

28
DISADVANTAGES OF SHORT-ANSWER ITEMS

• Spelling errors
• Grammatical errors
• Legibility tends to complicate the scoring
  process

29
DEVELOPING ESSAY ITEMS
Essay items require the examinee to write a
somewhat lengthy response to a question or
problem student is required to structure a
response, usually some detail and length,
consisting of up to several paragraphs of
narrative (the response is basically an essay.)
30
Two types of essay questions
a. Restricted-response type usually limits both
the content and the response the content is
usually restricted by the scope of the topic to
be discussed. Limitations on the form of response
are generally indicated in the question. Sample
items Describe two situations that
demonstrate the application of the law of supply
and demand. Do not use those examples discussed
in the class. Describe, in a paragraph of no
more than fifty words, the three most common
causes of HIV infection among teenagers.
31
b. Extended-response type allows students to
write essays in which they are free to express
their own ideas and interrelationships among
their ideas and to use their own organization of
their answers. Sample items Compare
developments in international relations in the
administrations of Prime Minister A and Prime
Minister B. Cite examples when possible.
Evaluate the coping mechanisms of the Malaysian
government during the economic crisis of 1990s.
32
SCORING ESSAY ITEMS
1. Check your scoring key against factual
responses. Once the grading has begun, the
standards should not be changed, nor should they
vary from paper to paper or reader to reader. 2.
Be consistent in your grading. Once the scoring
criteria have been checked out against actual
responses and once the grading has begun,
teachers should occasionally refer to the first
few papers graded to satisfy themselves that the
standards are being applied consistently.
33
3. Use the scoring rubric that is most
appropriate. One type is analytic (focuses on one
characteristic at a time and is especially useful
in providing students with specific feedback
about aspects of their work) and the other
holistic (more useful when the focus of the
assessment is on overall content understanding
rather than writing skill, per se).
34
SAMPLE RUBRIC
Score General Elaboration
Focus/Organization Impression
6 Exceptional Extended Clearly stated
main idea achievement elaboration Unified
focus and of one
organization main point Effectively orients
reader 5 Commendable Full Stated or
implied main achievement elaboration
idea of one Focused and organized main
point Effectively orients reader
4 Adequate Moderate Main idea present but may
achievement elaboration not maintain
consistent of one focus main
point Some orientation of reader
35
SAMPLE RUBRIC (contd)
Score General Elaboration
Focus/Organization Impression
3 Some Restricted Main idea not
clear evidence of elaboration Usually on
topic, but with achievement of one some
digressions main point 2 Limited Limited
Vague indication of main evidence
of elaboration idea or focus achievement of
one Significant digressions main point
No sense of closure
1 Minimal No No
apparent main idea evidence of
Elaboration No apparent plan
or achievement of one coherence main
point
36
4. Grade only one question at a time for all
papers. To reduce the halo effect (the quality
of the response to one question influencing the
readers evaluation of the quality of the
response to the next question), it is strongly
recommended that lecturers grade one question at
a time. 5. Grade the responses anonymously. The
lecturer should not know whose paper is being
graded. 6. The mechanics of expression
(legibility, spelling, punctuation, and grammar)
should be judged separately from what the student
writes.
37

• Try to score all responses to a particular
  question without interruption.
• Provide comments and correct errors, if possible.
• Set realistic standards (Tests should not be
  punitive they are instructional and evaluation
  tools).

38

• If especially important decisions (awards,
  scholarships, special training, and the like) are
  to be based on the results, obtain two or more
  independent ratings.
• 11. Avoid the use of optional questions in an
  essay test. (It is difficult to construct
  questions of equal difficulty. It also introduces
  scoring difficulties.)

39
ADVANTAGES OF ESSAY ITEMS

• Measure higher-level or complex learning
  outcomes provide the students to organize,
  analyze, and synthesize ideas
• Emphasize integration and application of thinking
  and problem-solving skills
• Enable the direct evaluation of writing and
  self-expression skills response can be evaluated
  with respect to English usage, sentence
  structure, spelling, and penmanship

40
DISADVANTAGES OF ESSAY ITEMS

• Scoring and inconsistencies are the primary
  disadvantage
• Irrelevant factors, such as neatness and
  penmanship, may also influence scoring.
• Essay items may limit sampling of content/topics
  covered.

41
TO BE SUBJECTIVE OR OBJECTIVE?
The question is purely RHETORICAL. Not only does
it have no answer, but it deserves none. Very few
lecturers will ever find themselves in situations
where they must always either one form of test or
the other. Some class situations, particularly
where size is a factor, may lend themselves more
readily to objective test formats in other
situations, essay formats are better sometimes a
combination of both may be desirable. The
important point is that each has advantages and
disadvantages. Good lecturers should endeavor to
develop skills necessary for constructing the
best possible items in a variety of formats
without becoming passionate advocates of one over
the other.
- Lefrancois (2000, p. 513)