Ensuring ABIM’s Exams Fairly Assess Physician Knowledge

Once a year, a group of the country’s leading experts on testing and measurement gather to discuss, debate and provide feedback on the scientific underpinnings of ABIM’s assessments. The Technical Advisory Committee (TAC) reviews the work of ABIM’s measurement staff and provides feedback on the technical psychometric procedures that ensure ABIM assessments are a fair and valid measure of physician knowledge.

ABIM invites some of the worlds most respected experts in educational measurement and psychometrics to form its Technical Advisory Committee (TAC). This group of scholars serve as independent external advisors who are brought together to analyze, debate and critique the scientific underpinnings of ABIM’s assessments, with an eye toward ensuring they are as good as they can be.

“Some assessment organizations are unwilling to shine a light on the assessment work performed by their psychometricians, but ABIM has sought out some of the brightest minds in assessment to ensure that we are continuing to deliver defensible state-of-the-art assessments,” said Jerome Clauser, Ed.D., Sr. Director of Research & Innovations at ABIM. “I am so proud of the expertise that is on display within this committee, and so pleased that ABIM is choosing to subject itself to this sort of scrutiny when many testing organizations choose not to.”

The TAC is comprised of experts from around academia and the testing industry, invited for their expertise and willingness to provide honest feedback on ABIM’s assessments.

During a recent meeting, committee members provided feedback and engaged in lively debate on a broad range of topics, including everything from aligning assessment with practice by including patient characteristics on multiple choice questions to aggregating scores for summative assessments.

The committee is part of the holistic process that ABIM follows to ensure all assessments are meaningful and accurately measure physician knowledge.

“ABIM designs and builds all of our assessments to ensure they are fair for examinees and accurate for our stakeholders,” said Brad Brossman, Ph.D., Vice President of Psychometrics at ABIM. “The TAC provides an outside second opinion to guarantee that our assessments are meeting the highest standards in certification testing.”

TAC member Neil Dorans, Ph.D., said he joined because he believed that his “four decades of experience with the theory and practice of assessment could benefit ABIM in their efforts to identify physicians who possess both the requisite knowledge of internal medicine and essential skills needed for ensuring proper patient care. Measurements collected under proper measurement conditions are as essential to accurate assessment of the skills and knowledge possessed by physicians as are the measurements that inform physicians about the health of their patients.”

TAC members:

• Kurt Geisinger, Director of the Buros Center for Testing and Distinguished Professor at the University of Nebraska-Lincoln
• Neil Dorans, former Distinguished Presidential Appointee at ETS
• Hariharan Swaminathan, Professor Emeritus at the University of Connecticut
• Mike Kane, Messick Chair in Validity at ETS
• Suzanne Lane, Professor Emeritus at the University of Pittsburgh
• Matthias Von Davier, Director, International Study Center at Boston College