Detecting cluster bias in a multilevel item response model: A Monte Carlo evaluation of detection methods and consequences of ignoring cluster bias

Abstract

Cluster bias in a multilevel item response model can be investigated by testing whether the within-level item discriminations are equal to the between-level item discriminations. However, in most multilevel item response model applications, the possibility of cluster bias is often ignored. Cluster bias detection methods using a multilevel item response model (the likelihood ratio test, Wald test, AIC, BIC, and saBIC) and the consequences of ignoring cluster bias are illustrated and discussed. Simulation results showed that all criteria performed well in detecting global cluster bias except the BIC with small sample sizes and high ICCs when some portion of the items exhibited cluster bias. For item cluster bias, the AIC outperformed the other criteria in the presence of partial cluster bias. When cluster bias was ignored, accuracy of item discrimination estimates and standard errors was mainly problematic. Implications of the findings and limitations are discussed.