Click here to attend the defence via Zoom
Trial lecture
Subscores on Educational Tests: Validity issues and Practical implications
Adjudication committee
- 1st opponent: Professor Stephen Sireci, University of Massachusetts, USA
- 2nd opponent: Professor Anil Kanjee, Tshwane University of Technology, South Africa
- Committee Chair: Researcher Dr. Trude Nilsen, University of Oslo
Chair of defence
Centre Director Professor Sigrid Blömeke
Supervisors
- Adjunct Professor David Rutkowski, University of Oslo
- Adjunct Professor Leslie Rutkowski, University of Oslo
Summary of the dissertation
In spite of a body of research into subscale score reporting at the individual level, there exists a paucity of research into subscale score estimation in international large-scale assessment (ILSA). This doctoral thesis aimed at evaluating the typically available methods for subscale score estimation in order to identify a model that was suitable for (a) item parameter estimation; (b) population score estimation; (c) reporting valuable subscale scores. This dissertation further examined the models in order to identify the better fitting model. The key motivation of this dissertation was to provide practitioners with general guidelines when it comes to estimating subscale scores under different test specifications.
This thesis was based on two simulation studies and an empirical study. Simulation studies 1 and 2 were designed to resemble the SACMEQ and TIMSS data. The difference between the two simulation studies was that one did not employ matrix sampled test booklets and latent regression methods in score estimation whilst the other did. Within each of the simulation studies, data were simulated assuming, the data comprised of single- and multiple-groups. The empirical investigations were based on data from TIMSS 2015’s eighth grade mathematics test.
As subscale scores have become increasingly relevant for guiding educational policy and practice, this study informed test practitioners as to the selection of the most appropriate subscale score estimation method. This thesis argues that different subscale score estimation methods may be more optimal under different test conditions and sample composition. In addition, this thesis argues that the choice of model may depend on the practitioner’s primary concern. This study also contributes to informing the choice of model when the sample of participants becomes more diverse with regards to performance.
The work on this thesis was carried out at the Centre for Educational Measurement, University of Oslo (CEMO). The research project was part of the Norwegian Research Council-funded project, “Embracing Heterogeneity in International Surveys”.