International large-scale (educational) assessment (ILSA) studies, such as PISA, measure the abilities of students across countries in different subjects such as mathematics or reading. The main goal of ILSA studies is to make accurate inferences about populations. For various reasons, it is common in ILSA studies to administer different sets of questions to the sampled students. Expressing this additional degree of uncertainty of measurement for an individual is one of the reasons that ILSA studies usually provide several possible scores for each individual, called plausible values. These are used to estimate the statistics of interest as well as to estimate the amount of measurement error in said statistic. Among the usual statistics of interest, especially proportions suffer from a larger amount of measurement error due to the granular nature of plausible values; they are either above or below a specified standard, essentially dichotomizing the continuous plausible value. The measurement error is further inflated once the plausible value for an individual is multiplied by their sampling weight.
In this study, a method called de-weighting is introduced to prevent this inflation of measurement error in ILSA population estimates due to sampling weights. In short, the de-weighting of plausible values entails imputing as many plausible values for a sampled individual as their sampling weight, as opposed to simply multiplying a single imputed plausible value by their sampling weight. The variability of the individuals in the population who are not sampled and therefore represented through a sampling weight is thus taken into account. The findings suggest that the proposed method greatly reduces measurement variance, thereby also decreasing the standard error of the estimates. As hypothesized, the effect is greatest when estimating proportions. The possible implications of using the de-weighting method for ILSA studies are discussed.