Associations Between Medical Knowledge and Patient Outcomes
By Furman S. McDonald, MD, MPH, Senior Vice President for Academic and Medical Affairs
When I talk with colleagues across the profession, I’m often asked about Maintenance of Certification (MOC) and whether there is evidence associating it with better patient care.
I appreciate these questions. Physicians today have so many things to do that MOC can feel like “just one more thing,” and knowing the time and effort invested in earning and maintaining certification, it’s fair to question its value in health care and to ask if there is any evidence suggesting that it is associated with patient relevant outcomes.
Recently, I was honored to be invited by Dr. Lee Berkowitz, whom I have long admired as a leader in graduate medical education, to give Grand Rounds at the University of North Carolina where I spoke to these questions in a talk entitled, “Evidence Based Education: Medical Knowledge, Acquisition and Associations with Patient Relevant Outcomes.” I discussed some research I had done on medical knowledge acquisition before joining the ABIM staff, and then described some of the more recent studies that have found associations with demonstrated medical knowledge and patient outcomes. I invite you to watch it here:
Regarding research, some have called for randomized controlled trials (RCTs) in relation to certification and MOC. In fact, there is a robust body of evidence in the educational literature spanning more than a century, including well-designed RCTs, demonstrating that testing enhances learning. However, in terms of evidence-based medicine (EBM), certification and MOC are not “therapies” for the outcome of learning, they are assessments of physician knowledge which have prognostic implications for patients who are the most important frame of reference. As described in the famous JAMA series, the highest levels of evidence for prognosis are not RCTs; rather prognosis is best investigated using well-designed cohort studies.
A large and growing body of evidence from well-designed cohort studies has demonstrated that patients who are cared for by doctors who demonstrate more medical knowledge through certification and MOC are associated with better prognosis on a host of important outcomes including mortality, ED visits, hospitalizations, myriad process of care measures, costs of care and state medical board disciplinary actions. The totality of the evidence now includes studies collectively involving hundreds of thousands of physicians and millions of patients, often involving the near complete population of physicians and patients they serve, making this conclusion highly generalizable.
The key verb in this assertion about physician knowledge and patient outcomes is “demonstrate.”As explained in the presentation, there are also good data that physician participation in learning activities without the ability to demonstrate medical knowledge does not have these associations with better patient prognosis. This is evidence for summative assessment, i.e., assessment of what has been learned, in ABIM Certification and MOC.
All of the ABIM-related studies have been done using traditional “longform” or “10-year” exams. The new Longitudinal Knowledge Assessment (LKA®) will be studied in the future and I hypothesize will show similar, if not stronger, associations since it is designed to be both a summative and formative assessment. (Formative assessment is “assessment for learning” and summative assessment is “assessment of learning.”) So, the LKA is an ongoing formative learning activity with a summative assessment built into it. However, we’ll need to wait until the LKA has been utilized longer to be able to assess its associations with patient-relevant outcomes.
I encourage readers of this blog to watch the Grand Rounds presentation or a shorter version we’ve entitled “A Whirlwind Tour of the Evidence.” For those that want to look into any of the studies in more detail, we provide the references on this website and also suggest that they check out the infographics on our website that provide visual summaries of some of the research presented in the videos. With so many things being asked of physicians it is very fair to ask whether certification and MOC are associated with better patient outcomes. For me, it is encouraging to know that the efforts we make to acquire and retain medical knowledge that can be demonstrated are associated with patient-relevant outcomes in so many studies, and I anticipate we’ll continue to see this already large body of evidence continue to grow.
