Title: Various Generalizations of Upper-Lower Index and Their Critical Values

Authors: Lubomir Stepanek, Patricia Martinkova


Affiliation: First Faculty of Medicine, Charles University in Prague;
Institute of Computer Science, Czech Academy of Sciences

Abstract:
While item response theory (IRT) is the most used approach for description of
item difficulty and discrimination, classical test theory indices may provide
an advantage of a quick and easy-to-understand calculation using basic
arithmetic. One popular index for item discrimination, the Upper-Lower Index
(ULI) is defined as a difference between proportions of correct answers to an
item in the upper and lower group of test takers, as defined by their total
score. For ULI, practical questions arise regarding the optimal size of the
upper and lower groups and regarding the critical values used to identify
relatively poor items for elimination or revision (Engelhart, 1965).  In this
study, we revisit ULI definitions and their generalizations (Brennan, 1972).
We use analytical derivations as well as simulations to get optimal group
sizes and to define critical values for cases motivated by university
admission tests (Martinkova et al., 2017). 

References:
Engelhart, M. D. (1965). A comparison of several item discrimination indices.
Journal of Educational Measurement, 2: 69-76. doi:
10.1111/j.1745-3984.1965.tb00393.x. 

Brennan, R. L. (1972). A generalized upper-lower item discrimination index.
Educational and psychological measurement, 32: 289-303. doi:
10.1177/001316447203200206. 

Martinkova, P., Stepanek, L., Drabinova, A., Houdek, J., Vejrazka, M., Stuka,
C. (2017). Semi-real-time analyses of item characteristics for medical school
admission tests. 189-194. doi: 10.15439/2017F380.