ABAASA JOINT COUNCIL ON INTRAINING EXAMINATIONS

1
ABA/ASA JOINT COUNCIL ON IN-TRAINING EXAMINATIONS

• 2004 REPORT TO SAAC/AAPD
• Glenn P. Gravlee, MD
• Chair, Joint Council
• The Ohio State University Medical Center
• Department of Anesthesiology

2
JOINT COUNCIL MEMBERS

• ABA
• David Chestnut
• Douglas Coursin
• Glenn Gravlee
• Patricia Kapur
• Mark Rockoff (Jim DiNardo)
• Raymond Roy
• Kenneth Tuman

• ASA
• Arnold Berry
• John Cooper (John Rowlingson)
• Jeffrey Gross (V.C.)
• Philip Lebowitz
• Charles Otto
• Patricia Petrozza
• Mark Rosen

3
50 Question Writers

• Title Junior Editors

4
IN-TRAINING EXAMINATION EDITORS

• Steven Allen, Audree Bendo, James Di Nardo,
  Carter Dodge, Sylvia Dolinski, John Ebert, John
  Emhardt, Robert Gaiser, Eric Kitain, Bruce
  Kleinman, Elliott Krane, Larry Krenis, Larry
  Kushins, Cynthia Lien, Vinod Malhotra, Donald
  Martin, Roger Mecca, John Moyers, Julia Pollock,
  Lindo Jo Rice, Robert Sladen, Richard Stypula,
  Richard Teplick, Helen Westman, Thomas Wolfe

5
IN-TRAINING EXAMINATION

• 3 exams in 1
• In-Training Exam
• Requalifying Exam
• ABA Written Exam
• Subset of 300 questions from ITE
• Plus Recertification Exam
• Well-performing A-types from ITE form the
  question pool

6
Demographics of In-Training Examinations
Total Number of Examinees
Registrants
7
DEMOGRAPHICS OF IN-TRAINING EXAMINATIONS - AMG
IMG
8
Exam Content340 Questions 100 are Links
15
16
4
8
9
47
Number of Questions
9
Disease States Breakdown163 Questions
Number of Questions
7.2
5.7
5.7
4.9
5.1
4.6
4.6
4.6
4.3
10
Item Difficulty (Mean Logit SE)
11
In-training ExamAverage Scaled Score
All CA3 Residents
12
Gaps in Knowledge CA3s

• 8 Surprising Gaps Identified
• Report being sent by ASA office

13
Examination Registration/Behavior

• Government Issue Picture ID
• NO Cell Phones or Pagers
• Calculators are NOT OK
• Consequences of Cheating
• Proctors are in charge

14
Joint Council Finances

• Joint Oversight by ASA/ABA
• Largely sustained by Exam fees
• In the Red (50-100K) when N was low

15
Joint Council Finances

• Still losing 20-50K/year with strong N
  increased proctor compensation, NBME fees mainly
• Exam cost _at_ 85 since 1996
• Joint Council approved increase in exam fee to
  100 effective for 2005 exam

16
Content Outline Revision

• 1996 version used for 2004 Exam
• 2003 version used for 2005 Exam
• New Format Organocentric
• Both versions on ASA website
• 2004 version disappears 12/31/04
• www.asahq.org/publicationsandservices/contentoutli
  nerev2003

17
FAQs/Future Issues

• Computerized Exam?
• Phase out K-types?
• Introduce R-types and question clusters (G types)
  in 2005

18
Q. Which of the following foods grows on trees?

• Bananas
• Tomatoes
• Apples
• Freedom (French) Fries
• A1,2,3 B1,3 C2,4 D4 E1,2,3,4
• Answer B

19
K-type evolution

• Always on exam approx 40 in 1970s and 1980s
• Gradual decrease to current 20
• Elimination being considered primary rationale
  for evaluating R-types
• NBME dislikes, controversial among Joint Council
  members

20
Disadvantages to K-types

• No longer on USMLE
• Take longer to answer (approx 60 vs 50 sec)
• Slightly more difficult than A-types
• Examinees dislike them

21
Advantages to K types

• Question performance has been good
• Discriminates stronger vs weaker examinees pretty
  well, ideal K-type requires greater depth of
  understanding
• Anesthesiology doesnt always lend itself to
  single best answer
• May take 3-4 A-types to test the same information
  as a single well-written K-type

22
K-type (interim) conclusions

• See how R-types perform
• Hold the line at 15-20 for now, reassess over
  next two years
• Lets base K-type fate more upon the need for
  knowledgeable anesthesiologists than upon
  question popularity among examinees
• Remember that they have 3-4 years to practice

23
Extended Matching (R-type)

• Acute DIC
• Hemophilia A
• Platelet function defect

• Idiopathic thrombocytopenia purpura
• von Willebrands disease
• Antithrombin III deficiency

1. 70 year-old man undergoing CABG/AVR is
oozing after CPB

• A 70 year-old man undergoing CABG/AVR
• develops a coagulopathy after CPB.

Answer C
24
Extended Matching (R-type)

• Acute DIC
• Hemophilia A
• Platelet function defect

• Idiopathic thrombocytopenia purpura
• von Willebrands disease
• Antithrombin III deficiency

An active 30 year-old man with a history of
hemarthroses and excessive bleeding after small
cuts presents for appendectomy.
Answer B
Continue with additional questions
25
R-types and G-types

• Examples soon will appear on ASA Web site

26
Nonstandard Examinations

• Extended time, quiet room, reader, etc.
• Application processing takes more time (4 months
  before exam)
• Process described in ABA Booklet of Information,
  Section 7.01
• Same process for ITE as ABA written exam
• Once approved by ITE, ABA usually rubber-stamps
  it (same committee)

27
Nonstandard Exam Concerns

• Numbers gradually increasing
• Candidates often wait for ABA exam to request
  accommodation
• Candidates/examinees often allow insufficient
  time
• Candidates/examinees presume that a previous
  accommodation automatically merits an ABA/ASA or
  ABA accommodation

28
Nonstandard Exam Concerns

• Recognize that A.D.D. does not automatically
  qualify for extended test time
• Recognize that psychological testing must be lt5
  years old
• Comments from candidates and from our expert
  consultants at times scare the Nonstandard Exams
  Committee

29

• When a resident fails to measure up to the
  academic standards and/or professional behavior
  expected, it is incumbent upon the program
  director to document these failures during the
  Residency Review Committee-mandated semiannual
  review.
• Miller SH, Plastic Reconst Surg 1990

30
Communications

• gravlee.1_at_osu.edu
• 614-293-9081
• Therese ODonnell-Leonard