Results of large-scale assessments of student achievement are sensitive to students’ persistence in maintaining a constant level of effort and precision over the course of a test. Low persistence is indicated, for example, by item position effects (IPE) that reflect decreases in the probabilities of correct responses being given towards the end of a test. IPEs are commonly modeled on the basis of assessment designs with rotated item positions by means of IRT models in which the items’ difficulty parameters are related to their positions in the test. In these models, the strength of this relationship is typically allowed to vary between individuals. Therefore, IRT models for IPEs allow individual differences in IPEs to be related to ability and to covariates. However, a drawback of the commonly used IRT approach is that it assumes that the students’ response process does not change across positions.
In this paper we present an alternative representation of test-taking persistence. We assume that students might change their response behavior from an effortful response mode to random guessing behavior. Drawing upon the HYBRID IRT model, we propose a model that can be applied to rotated assessment designs. The suggested model combines a two parameter logistic (2PL) part with a latent class model, whereby the latent classes represent the first item positions in which individuals have changed their response behavior. Latent class membership is expressed as a function of an underlying normally distributed continuous variable that reflects the individuals’ switching points to random guessing behavior. This specification enhances the estimation of latent class proportions, and allows for a straightforward assessment of the relationships of switching points with ability and covariates. The model can be estimated with standard software by means of maximum likelihood estimation via the expectation maximization algorithm.
To demonstrate the model’s utility, we applied it to a reading comprehension test (with 32 item positions) administered to fifth-grade students (n = 2,774) by means of a rotated matrix design. Compared to the commonly used IRT model for IPEs, the newly proposed model showed a better fit to the data. Results derived on the basis of the proposed model indicated that higher ability was associated with later onset points of random guessing behavior (r = .46). In addition, students’ switching points were clearly related to a test of decoding speed (r = .32). The standard IRT model for IPEs did not indicate any relationship between ability and IPEs and revealed a rather weak relationship between IPEs and decoding speed.
These findings suggest that, at least in the area of reading assessments, students’ test-taking persistence might be better represented by qualitative changes in response behavior. Under these circumstances, the proposed extension of the HYBRID model provides a promising tool for assessing test-taking persistence and studying its relationships with ability and covariates.