Certification Exam Scores for New Doctors are Associated with Fewer Deaths and Hospital Readmissions
Collaborative study by researchers from Harvard Medical School, Beth Israel Deaconess Medical Center and ABIM concludes that high exam scores are linked to better patient outcomes for newly trained internists, yet no link was found for the residency evaluations
You can draw a straight line from good performances on American Board of Internal Medicine (ABIM) board certification exams taken soon after residency and good patient outcomes.
That’s the conclusion from a study published in the Journal of the American Medical Association (JAMA) that looked at half a million hospitalizations and the performance of nearly 7,000 newly trained hospitalists who treated Medicare fee-for-service beneficiaries over three years.
The study examined the relationship between how both end of residency evaluations—known as Milestone ratings (designed to rate physician competencies ranging from knowledge and patient care to professionalism)—and initial ABIM internal medicine certification exam scores (designed to assess medical knowledge) were related to risk of death and costly hospital readmissions.
The study concludes that higher certification exam scores, not Milestone ratings—even the Milestone rating for medical knowledge alone which was designed to measure the same competency as the exam—are associated with fewer patient deaths and readmissions.
To put these findings in context, being cared for during a hospital stay by a newly trained internist who scored in the top 25%, versus the bottom 25% of all physicians who took the exam was associated with an 8% reduction in the risk that a patient will die within seven days of admission and a 9% reduction in the risk of being re-admitted to the hospital within seven days of discharge. No associations were found for similar Milestone ratings comparisons.
“Associations of Internal Medicine Residency Milestone Ratings and Initial Certification Exam Performance for Newly Trained Hospitalists with Outcomes of Hospitalized Patients,” published today in JAMA, analyzed how 6,898 newly trained hospitalists treated Medicare fee-for-service beneficiaries over 455,120 hospitalizations between 2017–2019. The hospitalists completed training between 2016 and 2018.
“These results confirm that certification exams are measuring knowledge that directly translates into improved outcomes for patients,” said Bruce Evan Landon, MD, MBA, Harvard Medical School, and a coauthor. “Though Milestone ratings might be helpful to programs, these ratings don’t appear to be a strong indicator regarding the quality of recent graduates of internal medicine training programs.”
Lead author Bradley Gray, Ph.D., a Senior Health Researcher at ABIM, added, “These findings support the growing belief that a high-quality standardized exam can provide more meaningful feedback than measures of competency, regardless of how well constructed, that are based on subjective judgments of evaluators.”
The paper is one of a series of research studies done to measure the value of ABIM certification and Maintenance of Certification (MOC). Previous studies have shown hospitalized heart attack and heart failure patients die less frequently when treated by ABIM-certified doctors. They also show a strong link between a physician’s knowledge, as measured by the ABIM MOC exam, and reductions in likelihood of patient death, emergency room visits or hospitalizations as well as a reduced chance of receiving an inappropriate opioid prescription.
