Document Type
Article
Publication Date
Winter 12-15-2011
Abstract
Objective. To compare the reliability and credibility of Angoff-based, absolute criteria derived by faculty, alumni, and a combination of alumni and faculty judge panels.
Methods. Independently, faculty, alumni, and mixed faculty-alumni judge panels developed pass/fail criteria for an 86-item test. Generalizability and decision studies were performed. Root mean square errors (RMSE) and 95% confidence intervals were calculated for reliability and credibility assessment. School graduate performance upon the North American Licensure Examination (NAPLEX) was the comparator for credibility assessment.
Results. RMSEs were 1.06%, 1.42%, and 2.32% for the alumni, faculty, and mixed judge panels respectively. The school's NAPLEX pass rate was 97.5%. This rate triangulated well with the faculty judge panel (pass rate = 93.9%, CI95% = 87.1% - 98.2%), but did not with either mixed judge or alumni judge panels.
Conclusions. Faculty-derived criteria offer superior pass/fail decision defensibility relative to both alumni derived and mixed faculty-alumni derived criteria.
Recommended Citation
H. Glenn Anderson Jr and Arthur A. Nelson (2011). Reliability and Credibility of Progress Test Criteria Developed by Alumni, Faculty, and Mixed Alumni-Faculty Judge Panels. American Journal of Pharmaceutical Education: Volume 75, Issue 10, Article 200. doi: 10.5688/ajpe7510200
Included in
Other Pharmacy and Pharmaceutical Sciences Commons, Pharmacoeconomics and Pharmaceutical Economics Commons, Pharmacy Administration, Policy and Regulation Commons
Comments
The copy of record is available from the publisher at http://www.ajpe.org/doi/abs/10.5688/ajpe7510200. Copyright © 2011 American Journal of Pharmaceutical Education. Reprinted with permission. All rights reserved.