Document Type

Article

Publication Date

Winter 12-15-2011

Abstract

Objective. To compare the reliability and credibility of Angoff-based, absolute criteria derived by faculty, alumni, and a combination of alumni and faculty judge panels.

Methods. Independently, faculty, alumni, and mixed faculty-alumni judge panels developed pass/fail criteria for an 86-item test. Generalizability and decision studies were performed. Root mean square errors (RMSE) and 95% confidence intervals were calculated for reliability and credibility assessment. School graduate performance upon the North American Licensure Examination (NAPLEX) was the comparator for credibility assessment.

Results. RMSEs were 1.06%, 1.42%, and 2.32% for the alumni, faculty, and mixed judge panels respectively. The school's NAPLEX pass rate was 97.5%. This rate triangulated well with the faculty judge panel (pass rate = 93.9%, CI95% = 87.1% - 98.2%), but did not with either mixed judge or alumni judge panels.

Conclusions. Faculty-derived criteria offer superior pass/fail decision defensibility relative to both alumni derived and mixed faculty-alumni derived criteria.

Comments

The copy of record is available from the publisher at http://www.ajpe.org/doi/abs/10.5688/ajpe7510200. Copyright © 2011 American Journal of Pharmaceutical Education. Reprinted with permission. All rights reserved.