Training Surgeons

1
Training Surgeons Ability to Judge Operative
Risk Using Feedbackfrom Judgment Analysis

• Rosamond Jacklin, Nick Sevdalis, Ara Darzi,
  Charles Vincent

2
Judgment Analysis

• Participants provide risk estimates for a series
  of scenarios
• Differ with respect to risk factors
• Estimates modelled quantitatively
• Multiple linear regression
• Beta weights
• Model fit
• Overcomes limitations of self report
• Comparison to gold standard

3
Variables affecting operative mortality in study
Judgment from Surgeon A

• ASA status
• Urgency of
• operation
• Age of patient

Estimate from gold standard1
Judgment from Surgeon B
1. Donati et al, 2004
4
Study Design

• 2 groups of participants
• surgical trainees (n69) PGY 1-3
• medical students (n35) Final year
• Ranking of variables (cues) by importance
• First judgment analysis task
• Individualised feedback prepared and given
• Second judgment analysis task

5
Feedback

• Personalised feedback
• Anchors highest and lowest values
• 2. Overall correlation between participant
  estimates and the gold standard
• 3. Influence of ASA, urgency of operation and age
• 4. Goodness of fit of regression model

6
Measures

• Accuracy
• Correlation
• Mean signed difference
• Mean absolute difference
• Cue weightings
• Observed (implicit) cue weightings
• Explicit cue weightings

7
Mean Absolute Difference
Absolute Mean Difference
Pre-feedback
Post-feedback
8
Observed (implicit) Cue Weightings
9
Declared (explicit) cue weightings

• Pre-feedback
• 18.4 participants correctly rank ordered the
  variables
• Post-feedback
• 95.2 participants correctly rank ordered the
  variables

10
Conclusions

• Can measure performance at surgical risk
  estimation
• Basis for graphical feedback
• Task design must be appropriate for learner
• Explicit learning of importance of risk factors
• ? ability to integrate risk factors into
  judgments of operative risk

11
Acknowledgements

• Royal College of Surgeons of England
• Rosetrees Foundation
• Grand Lodge 250th Anniversary Fund