ARC WM Blog Content
Academic Performance is a Better Predictor of Unprofessional Behaviour than the Situational Judgment Test
The first ever paper written by the ARC WM Director, while still a medical student, concerned the admission criteria for medical school.[1] Subsequently, we showed that psychological tests were extremely poor predictors of performance and they are easily gamed.[2]
Celia Brown, ARC West Midlands collaborator, has now co-authored a paper that throws further doubt on the value of situational judgment tests. [3] The authors created a national retrospective cohort to examine the association between the different types of test (situational judgement and educational achievement) in the medical entrance examination and the probability of subsequent disciplinary action in the first five years of professional practice.
While an increase of one standard deviation in the educational performance measure reduced the risk of disciplinary action by about 50%, there was no correlation between the situational judgment result and risk of disciplinary action. The finding that educational achievement is associated with lower risks of antisocial behaviour (in addition to future academic attainment) corroborates an extensive previous body of work referenced in the article.
What is the mechanism? High academic achievement is a marker of self-discipline and commitment, attributes that are also necessary for good conduct. It turns out that academic achievement is the best test of overall fitness to practice. Michael Sandel has argued forcibly against ‘merit’ in selection for high status professions, arguing instead for a lottery system. [4] Brown and colleagues show what poor policy that would be – patients would pay the price in the case of medical care.
Richard Lilford, ARC WM Director
References:
- Lilford RJ. The doctor’s role in terms of fundamental human needs. Leech. 1972; 41 (1-2): 7-8.
- Hemmerdinger JM, Stoddart SD, Lilford RJ. A systematic review of tests of empathy in medicine. BMC Med Educ. 2007; 7: 24.
- Sam AH, Bala L, Westacott RJ, Brown C. Is Academic Attainment or Situational Judgment Test Performance in Medical School Associated With the Likelihood of Disciplinary Action? A National Retrospective Cohort Study. Acad Med. 2021; 96(10): 1467-75.
- Sandel MJ. The Tyranny of Merit: What’s Become of the Common Good? London: Penguin Books Ltd; 2020.