QUINT just completed training on the PLATO classroom observation instrument for the round of data collection under the LISA Nordic Study. Data collection is now completed in all subjects in Iceland and Denmark. Social Science lessons will be recorded in
Sweden and Norway in autumn 2019. Data collection in Finland is planned to start in January 2020.
As part of this effort, in an attempt to better understand how training on classroom observation instruments leads raters to be more accurate and effective scorers, Centre Director Kirsti Klette and QUINT Postdoctoral fellow Mark White, conducted a small pilot study.
In this study, they had raters who were undergoing the PLATO training score Norwegian language arts lessons both before and after the PLATO training. Data collection for this study has just wrapped up and the raters generally reported this to be an interesting experience. When scoring PLATO pre-training, the raters struggled to understand PLATO dimensions, expressing a great deal of uncertainty about what classroom behaviors might actually count under each element and where the boundary between a score of a 2 and a 3 should lie. Despite this uncertainty, on most dimensions, raters wound up providing fairly similar scores, which may reflect their common background as students in Nordic schools, teacher trainees in Nordic colleges of education, and teachers in Nordic schools. Post-training, raters were far more certain in their scores and were more fluent in their ability to justify and explain their scoring, though it is important to note that uncertainty in how to interpret and understand some classroom events remained. Mark White concludes:
We are just now starting the process of diving into the data collected to explore whether raters are providing more consistent and accurate ratings, as well as exploring the way rater talk and think about their ratings might elucidate ways that PLATO may privilege instructional forms that are less appropriate for the Nordic context.
