PREPARING EFFECTIVE MULTIPLE CHOICE EXAMINATION QUESTIONS

1
PREPARING EFFECTIVE MULTIPLE CHOICE EXAMINATION
QUESTIONS

• PATRICK DUFF, M.D.

2
LEARNING OBJECTIVES

• Describe the levels of complexity of an
  examination question
• Describe the types of examination questions
• Describe desirable and undesirable features of
  examination questions

3
KEY REQUIREMENTS FOR AN EXAMINATION
REQUIREMENT DEFINITION
Validity Measures what it is supposed to measure
Reliability Provides reproducible and consistent results
Discrimination Distinguishes between the excellent, very good, and fair student
4
COMPLEXITY OF EXAMINATION QUESTIONS
CLASSIFICATION DESCRIPTION
Recall Recall of a specific fact
Interpretation Identification and interpretation of illustrative material
Problem solving Recall, interpretation, and practical application
5
TYPES OF EXAMINATION QUESTIONS
TYPE REMARKS
C either, or, both, neither Unacceptable
K select among several alternatives Unacceptable
True false Unacceptable
6
TYPES OF EXAMINATION QUESTIONS
TYPE REMARKS
A select the one best answer Ideal question appropriate for recall, interpretation, problem solving
B matching Appropriate for recall, interpretation, and simple management problems
7
EXAMPLERECALL

• Which of the following is the most important risk
  factor for squamous cell carcinoma of the lung?
• A. Age gt 50
• B. Diabetes
• C. Smoking
• D. Obesity

8
EXAMPLEINTERPRETATION

• A 24 year old nulliparous woman at 16 weeks
  gestation had the following values on her serum
  analyte screening test
• AFP below the mean
• HCGabove the mean
• Estriolbelow the mean
• Inhibinabove the mean

9
EXAMPLE INTERPRETATION

• Which of the following chromosome abnormalities
  is her baby most likely to have?
• A. Triploidy
• B. Trisomy 21
• C. 45, X
• D. Trisomy 18

10
EXAMPLEINTERPRETATION

• A term neonate has hepatosplenomegaly,
  thrombocytopenia, and the diffuse rash noted in
  the photograph.

11
EXAMPLEINTERPRETATION

• Which congenital infection is most likely?
• A. Parvovirus
• B. CMV
• C. HIV
• D. Mumps

12
EXAMPLEPROBLEM SOLVING

• A 22 year old male has a 48 hour history of
  malaise, fever (39 deg C), shaking chills, and
  cough productive of rust-colored sputum. On
  physical examination, there is dullness to
  percussion in the lower left lung field, and
  tubular breath sounds can be heard in this area.

13
EXAMPLEPROBLEM SOLVING

• Which of the following is most appropriate for
  treatment of this patient?
• A. Isoniazid
• B. Fluconazole
• C. Ceftriaxone
• D. Acyclovir

14
APPROPRIATE MIX OF EXAMINATION QUESTIONS
15
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• Determine a single clear objective prior to
  writing the question
• Have only one objective per question
• Make sure the question is at an appropriate level
  of difficulty for the target audience

16
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• Correct response should be obvious to the
  well-informed examinee before the answers are
  read
• If the examinee has to read all of the answers
  before deciding, the question is probably a
  K-type disguised as an A-type

17
TYPE A QUESTIONGOOD EXAMPLE

• Which of the following is the most common
  complication of multiple pregnancy?
• A. Gestational diabetes
• B. Placenta previa
• C. Postpartum hemorrhage
• D. Preterm delivery

18
BEWARE THE PSEUDO-A QUESTION

• All of the following statements about preterm
  delivery are true except
• The frequency is 10 .
• It can be prevented by use of progesterone.
• It is more common in whites than in
  African-Americans.
• It may result in neonatal RDS.

19
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• Include sufficient information in the stem, but
  avoid unnecessary verbiage
• Be certain the question has a clear, unambiguous
  answer

20
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• Include 4 to 5 possible answers
• Be certain the incorrect answers are reasonably
  plausible

21
TYPE A QUESTIONBAD EXAMPLE

• What is the risk of pregnancy loss after a second
  trimester amniocentesis?
• A. 0.5
• B. 10
• C. 25
• D. 50

22
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• Do not use none of the above or all of the
  above as distractors
• Confusing
• Rewards guessing

23
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• Avoid clues in the responses such as answers of
  unequal length or repetition of key words

24
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• Avoid compound answers, ie, answers that include
  multiple actions

25
TYPE A QUESTIONBAD EXAMPLE

• Which of the following is the most appropriate
  treatment for community-acquired pnemonia?
• A. Clindamycin
• B. Vancomycin
• C. Valacyclovir
• D. Administer oxygen, check blood gases, and
  administer levofloxacin

26
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• Be wary of except type questions, especially
  when a double negative is present

27
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• In certain instances, an except type question
  may be appropriate and may accurately assess the
  breadth of an examinees knowledge

28
TYPE A QUESTIONGOOD EXAMPLE

• All of the following are risk factors for
  gestational diabetes except
• A. Obesity
• B. Positive family history
• C. Age lt 25 y
• D. History of a macrosomic child

29
GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTION

• Be certain that illustrative materials are clear
  and that they reproduce well
• Vary the correct response among the letters a - d

30
GUIDELINES FOR AN EFFECTIVE TYPE-B QUESTION

• Clearly identify the nature of the items in
  column A and column B
• Include at least 1 or 2 unused distractors

31
GUIDELINES FOR AN EFFECTIVE TYPE-B QUESTION

• Indicate if answers in one of the columns may be
  used only once, more than once, or not at all
• Be certain the matching items are reasonably
  related to one another

32
GUIDELINES FOR AN EFFECTIVE TYPE-B QUESTION

• Matching items are particularly useful for
  testing interpretation skills
• X-ray
• ECG
• Sonogram
• Histology

33
TYPE B QUESTIONGOOD EXAMPLE
COLUMN A Antibiotics COLUMN B Target Organisms
1. Clindamycin a. Bacteroides species
2. Penicillin b. Staphylococci
3. Gentamicin c. E. coli
d. Group B streptococci
34
TYPE B QUESTIONBAD EXAMPLE
DRUG PRINCIPAL SIDE EFFECT
1. Nifedipine a. Marrow suppression
2. Ampicillin b. Decreased libido
3. Fluoxetine c. Rash
d. Hypotension
35
CONCLUSIONS

• Use only type-A and type-B questions
• Minimize recall questions and maximize questions
  that test interpretive and problem-solving skills

36
CONCLUSIONS

• Be certain that each question has a single and
  specific objective
• Be certain the questions can be answered in the
  time allotted