Screening Items for Bias: An Empirical Comparison of the Performance of Three Indices in Small Samples of Examinees
OTHER
Jeffrey D. Kromrey, Cynthia G. Parshall
Abstract
A Monte Carlo study was conducted to compare the performance of three statistical indices of test item bias in small samples of examinees. The statistical indices compared were the Delta method, the Mantel-Haenszel (MH) method, and the Standardization method. Sample sizes of 50, 100, and 200 were examined. One thousand samples of each size were drawn with replacement from each of three archival data files from three teacher subject area tests (in the areas of elementary education, early childhood education, and specific learning disabilities). Each sample was drawn so that 80% of the examinees were sampled from a reference group and 20% were sampled from a focal group. Item bias was experimentally controlled in the study, and the effectiveness of the indices was evaluated as the proportion of such biased items appropriately identified. Previous research suggesting that item bias indices such as the MH and Standardization methods should only be applied to large samples may have been overly conservative. Results support the use of statistical screening for item bias, even with samples as small as 50 examinees, and with only 10 focal group members in each sample. The MH is the best performer of these three indices, although both the MH and Standardization methods are preferable to the Delta method. Three tables present simulation data. An 11-item list of references and 3 tables are included. (SLD)
Citation
Kromrey, J.D. & Parshall, C.G. Screening Items for Bias: An Empirical Comparison of the Performance of Three Indices in Small Samples of Examinees. Retrieved August 14, 2024 from https://www.learntechlib.org/p/145081/.
This record was imported from
ERIC on March 21, 2014. [Original Record]
ERIC is sponsored by the Institute of Education Sciences (IES) of the U.S. Department of Education.
Copyright for this record is held by the content creator. For more details see ERIC's copyright policy.
