LISA Nordic session | Standardized instruments and observation manuals as lenses into teaching quality: findings from Nordic classrooms using student perception surveys and the PLATO manual
In a special session at QUINT Conference 2022, “Standardized instruments and observation manuals as lenses into teaching quality: findings from Nordic classrooms using student perception surveys and the PLATO manual”, three key researchers from LISA Nordic presented papers on insights gained from the project. LISA Nordic Study is one of the central projects in the QUINT centre, coordinated by Centre Director Professor Kirsti Klette (University of Oslo) and Professor Michael Tengberg (Karlstad University).
The session included three papers focusing on teaching quality in Language Arts classrooms in the Nordic countries, as well as comments from the discussant, Professor Pamela Grossman (University of Pennsylvania).
Paper 1 (Klette et al.) reported on a standardized classroom observation manual (PLATO) and discussed challenges linked to analyzing teaching quality. Paper 2 (Roe et al.) focused on how student perception surveys (the Tripod survey) might inform researchers about features of teaching quality. And Paper 3 (Gisladottir et al.) reported on the stated purpose and use of feedback (based on PLATO scorings) in Icelandic classrooms.
Paper 1: “Standardized Observation Manuals as Lenses into Teaching Quality: Findings from Nordic Classrooms Using the PLATO Manual”
Presenters: Professor Kirsti Klette and QUINT Postdoctoral Fellow Camilla G. Magnusson (QUINT/University of Oslo).
Co-authors: Astrid Roe, Marte Blikstad-Balas, Jennifer M. Luoto (QUINT/University of Oslo), and Michael Tengberg (QUINT/University of Karlstad).
The paper reported on a predefined observation protocol in a Nordic context, to decompose teaching into key elements and identify key patterns and features for improvement, and to check for variation, using the Classroom Discourse element (CD) as an example.
The presentation focused on the research design for the LISA Nordic study as well as providing insight into the PLATO manual (Protocol for Language Arts Teaching Observations) and the findings across Nordic classrooms. The findings showed that in language arts, there was a general pattern of low scores on scaffolding elements and high scores on classroom environment elements. However, scores on the classroom discourse element (CD) varied across countries, and in-depth analyses of high scores in CD revealed common features across classrooms: use of questions, teachers revoicing and prompting, and often related to literature discussions. The paper’s authors concluded that PLATO works well within a Nordic context, that there are strong similarities in PLATO scores across Nordic LA classrooms, however, there is variation in scores on some elements (CD as an example). They highlighted that PLATO is able to capture features of teaching quality in a Nordic context, and that teachers’ instructional patterns across contexts are similar rather than different, although they find variation on the element level.
Paper 2: “Student perceptions as an indicator of teaching quality: A report from Nordic lower secondary language arts classrooms”
Presenter: Astrid Roe (QUINT/University of Oslo).
Co-authors: Michael Tengberg (QUINT/Karlstad University), Berglind Gísladóttir (QUINT/University of Iceland), and Anders Stig Christensen (QUINT/UCL University College in Denmark).
This paper reported on patterns of prevalent instructional practices between Nordic countries, and to explore validity aspects of the student responses as an indicator of teaching quality. The study reported on results from the Tripod student survey in five Nordic contexts (Norway, Sweden, Denmark, Iceland, and Finland), measuring student perceptions of teaching quality based on seven dimensions: care, control, clarity, challenge, captivate, confer, and consolidate, on a five-point Likert scale. The results focused mainly on score distributions. Across all countries, scores ranged from a total mean of 3.56 on the dimension “confer” to 3.99 on the dimension “care”. The total mean by country ranged from 3.61 in Iceland to 4.27 in Finland. Variations between countries on the different dimensions revealed that Sweden scored a little higher on “care”, Iceland a little lower on “control” and “clarify”, and Finland a bit lower on “challenge”. There were large variations between classrooms.
The paper’s authors concluded that there is some evidence suggesting that the Tripod survey may provide reliable data on Nordic students’ perception of teaching quality, but critical aspects of validity are yet to be explored. There are some recurrent differences between countries, but variation within countries is also considerable. However, in general, Nordic students appear to be satisfied with the quality of language arts teaching, especially with the quality of instructional explanations and the provision of care and support.
Paper 3: “Stated purpose and use of feedback in Icelandic lower secondary classrooms. Results from video-recordings”
Presenter: Berglind Gísladóttir, QUINT/University of Iceland
Co-authors: Birna María Svanbjörnsdóttir and Sólveig Zophoníasdóttir (QUINT/University of Akureyri)
The aim of this paper was to provide insight into teaching quality in Icelandic lower secondary schools and to assess the quality indicators “purpose” and “feedback” in language arts lessons and mathematics. The researchers reported on 72 video-recorded lessons that were analysed using the PLATO protocol. Their findings show some evidence regarding the quality of teaching in lower secondary classrooms in Iceland, related to the feedback students receive from their teachers and the coherence of the lessons observed. In contrast to previous findings from the Nordic countries, the researchers found limited evidence for a clear connection between the elements of purpose and feedback. However, similar to previous Nordic studies, they found that in a majority of the observed lessons, purpose and feedback were at the lower end of the PLATO observation protocol, indicating limited evidence of quality feedback to students and coherent lessons.