Diplomates to Receive Additional Feedback on their Exams – Incorporates Diplomate Recommendations (Part I)

We’ve heard from many physicians who, when they get their exam results, want to have more information so they can better understand their results. Working with input from ABIM Board Certified physicians, we are launching a new score report that we hope addresses many of the improvements physicians have asked for. The new score report is now distributed electronically and features clearer graphical explanations and more detailed feedback on performance.

Through our conversations with physicians, we learned together how the exam question feedback could be improved: Physicians want more specific information regarding the exam questions they missed. At the same time, we heard that the report needed to be simplified to be most useful. I am optimistic the improved report addresses both these important concerns.

Prototyping and Usability Study with Physicians

The process to revise the exam score report was rigorous. To identify best practices, ABIM staff researched similar testing organizations, bringing the insights gained to a focus group of randomly sampled ABIM Board Certified physicians. The focus group participants used those insights to inform an in-depth discussion on what physicians seek in a score report and what information might be missing, and to identify what was confusing about the old report. The focus group discussion led to the development of two initial prototypes, iteratively refined, based on comments from more physicians. A “hands-on” usability study—point-by-point review of each prototype—was then conducted by randomly selected ABIM Board Certified physicians. The ultimate draft design—a blend of the two prototypes—was presented by electronic survey to physicians representing all of the ABIM exam committees for final input and further refinement.

What We Learned

As an ABIM staffer, the honest face-to-face conversations with physicians were invaluable. While it was sometimes hard to hear that the old score report had too much technical jargon and missed the mark, it was wonderful to see that the new design is jargon-free and lengthy explanations have been replaced with hyperlinked, supplemental Web material.

There were so many voices heard in the development of the new report that this process can serve as a model for ways we can continue to work closely with the physician community to enhance our products and programs. We appreciate the feedback we receive from the internal medicine community, and I encourage you to continue sharing your thoughts and ideas with ABIM’s CEO at rbaronmd@abim.org.

At a later date, I will be sharing a more detailed update on the new score report sections.

 

See below for sample pages from the new report: