Research has shown the benefits of taking into account the correlation among dimensions when estimating latent trait scores in computerized adaptive tests (CATs). Multidimensional CATs (MCATs) could further improve measurement precision/decrease test length as compared to using separate unidimensional CATs for each domain, especially if domains are highly correlated.
In this study, we systematically evaluate the impact of a number of important design factors on CAT performance, using realistic example item banks. Two main scenarios are compared: health assessment (polytomous items, small to medium item bank sizes, high discrimination parameters) and educational testing (dichotomous items, large item banks, small to medium-sized discrimination parameters). Measurement efficiency is evaluated for both between-item multidimensional CATs (MCAT conditions) and separate unidimensional CATs for each latent dimension (UCAT condition). We focus on fixed-precision CATs since it is both feasible and desirable in health settings; but to date most research regarding CAT has focused on fixed-length testing. This study shows that the benefits associated with fixed-precision multidimensional CAT hold under a wide variety of circumstances.
MCAT has great potential when it comes to reducing test length and improving accuracy and precision of latent trait scores, both in health and educational measurement. We will discuss how the incremental value of MCAT depends on factors like adequate targeting, the size of the correlations, item bank size, and item parameters.