The fifth annual QUINT conference, "Analyzing Teaching Quality – instruments, improvements and implications. What have we learned?", brought together world leading scholars with a keen interest in discussing the many aspects of measuring teaching quality.
Over the course of the last five years the Nordic Centre of Excellence QUINT – Quality in Nordic Teaching, has produced significant findings on teaching quality. The centre has been an international hub for research on teaching quality, particularly in the form of its annual conference.
In this, the final QUINT conference, education researchers from around the world gathered to discuss the analysis of teaching quality, with a particular focus on the instruments of measurement, and on improving teaching quality.
Over three days participants heard from top academics from the US and Europe, and were presented with findings from the QUINT Centre’s extensive research on teaching quality in the Nordic countries.
This year's conference hosted more than 120 guests from 13 countries.
The three-day even concluded with a panel discussion by education authorities from the municipal and national levels in the Nordic countries.
Keynote summaries
Courtney Bell
In her keynote presentation, Professor Courtney Bell discussed the measurement of teaching quality by looking at it in its historical and scientific context.
“Teaching quality matters—in every country, for all students,” Bell said. “But the field does not agree on or use measures of teaching quality that satisfy common considerations for reliability, validity, and practicality.”
Bell emphasised the importance of measuring teaching quality in context-specific ways.
“Your decision about how best to measure teaching quality will depend on the context in which you are measuring it, how much it will cost and what kind of burden it will be to your research team and the schools you will be working in […] And of course, it will depend on the technical quality of the measures themselves.”
She argued that there is a need for more work on the reasoning behind choosing specific measures of teaching quality.
“We have to consider the social implications of particular measurement decisions [...] we [also] have to take account of the purpose, empirical data, our experiences with those measures, and the social context in which we are operating. There is very little thinking around this, and we could use more.”
Bell presented provisional findings from a forthcoming literature review that investigated the reliability and the strength of the validity evidence in different measurement approaches. One of the findings was that definitions of teaching quality are not clearly specified in much of the literature.
Bell also identified potential lines of new research for QUINT collaborators, and the broader international research community.
“[An] enduring tension of measuring teaching quality concerns how we can empirically account for our increasingly complex, nuanced theories and understandings of teaching and learning […] It would be great to have scholars working on those problems as a community – problems of theory, statistical modeling, and placing students’ learning into systems of education – community and school based.”
Hannah Bijlsma
In her keynote, Bijlsma presented findings from a recent study from the Netherlands carried out by the University of Twente and the Dutch School Inspectorate.
The study developed an instrument to collect the perceptions of observers (school inspectors), students and teachers on teaching quality.
The findings showed that school inspectors in the Netherlands had a more critical perception of the quality of teaching in a given classroom than the teachers or students from that same classroom.
Bijlsma also discussed the use of feedback to teachers, gathered from studies like these, to work on improving teaching quality.
The floor was then opened for comments from Ole Espen Rakkestad from the Norwegain teacher’s Union; Knut Erik Brandvald, a school leader from Oslo, and Anna Kristín Sigurðardóttir, a QUINT researcher from Iceland.
The panel discussed questions related to teaching quality measurement from a school-level perspective.
Drew Gitomer and José Felipe Martínez
In their joint presentation, Drew Gitomer and José Felipe Martínez discussed the challenges associated with the reliability and validity of observation metrics that are used to measure teaching quality.
Gitomer & Martínez have previously contributed to QUINT Observation Systems Seminar series.
“Teaching is more complex than ever,” Gitomer and Martínez said. “The methods and techniques used to study it are substantially refined, but evidence is still inconsistent at best.”
This problem of reliability is inherent to what is being measured, Gitomer and Martínez said. ‘Teaching,’ as they pointed out, is not a natural category (with physical features), so measuring it results in inconsistent categorisations.
“It was unrealistic to expect that these kinds of observation protocols would have produced more reliable and valid scores. The greatest promise of observational methods is to contribute to improving teaching through professional and institutional development.”
Gitomer and Martínez proposed two general approaches to the reliability problem of observation measures.
The first is to re-conceptualize ‘measurement error’ in observation.
“Many facets of what has been conceptualized as measurement error can be rich sources of information critical to understanding and engaging in the support and improvement of teaching” they said.
The second is to exploit the possibilities that AI offers for classroom observation.
While Gitomer and Martínez expressed scepticism at the many myriad claims made by commercial AI actors that their services will revolutionize teaching observation, they nevertheless see possibilities for applying the technology in classroom research.
One area where they see this potential is in automated measurement of 'low-inference measures', like classroom discourse. Another is as a supplementary tool for researchers’ analysis that can, for example, select and transcribe classroom videos.
Gitomer and Martínez concluded by underlining the importance of the policy context in measuring teaching quality, such as national teacher evaluation schemes, or popular views on the desired outcomes of education.
Pam Grossman
In her keynote presentation, Professor Pamela Grossman focused on research on core practices and teacher development. She began by mapping the current landscape of core practices in education research.
“On one hand, we have definitely made progress in coming up with ways to define teaching. But in some ways, this has come at a price. As lists of core practices have proliferated, we still lack consistent terminology to describe what counts as a core practice.”
Grossman gave an overview of how core practices have been taken up in teacher education, despite this lack of consistency in the terminology.
“A number of teacher-ed programs have reorganized their curriculum around a set of core practices, and a number of scholars have written about the process. What seems important is the initial agreement around a set of core practices that faculty can agree upon.”
Grossman also pointed out that there are relatively few studies that connect the quality of core practices to student learning. She referenced emerging research that shows the importance of decomposing core practices in teacher education, prior to teaching students tying to enact these practices. There is evidence that teaching students that spend time decomposing a practice are more likely to grow in that practice.
Professor Grossman gave special mention to the work of QUINT researchers, including the use of the use of the PLATO observation protocol to overcome some of the challenges associated with studying core practices in teacher education. In the QUINT studies, PLATO, Grossman said, “provided a common technical language for researchers and teachers” and “provided evidence that descriptions of classroom practices can travel across national borders.”
Grossman concluded her talk with questions about core practices for the field to address, one of which was the problem of grain size when defining core practices, which she described as a ‘Goldilocks problem.’ ‘Creating a Positive Classroom Environment’ for example was too big to be a core practice, whereas ‘use of wait time’ was too small.
Panel Discussion
The panel discussion, led by Marte Blikstad-Balas, tackled the question of how to work systematically around teaching quality in the Nordic countries.
The panellists from different levels of the school sector in Denmark, Finland, Iceland, Norway, and Sweden shared their perspectives on how to approach this complex challenge.
One of the questions raised during the discussion was the role of assessment in efforts to improve teaching quality. How much, and in what ways, should students and teachers be assessed? And how should assessment information be used to work on teaching quality?
There were no quick fixes proposed during the discussion, but there was agreement among the panellists that willingness to work systematically on teaching quality was not a barrier, all levels of the school systems in the Nordic countries agree that it is important.
The panellists also discussed a then-unreleased video from QUINT, which summarised the findings of the LISA Nordic study. That video can be viewed below.
Conference photos
Photos: Misha Jemsek/UiO