About the book
Complete book (pdf)
Recent research suggests that there is a direct link between quality of teaching and
teacher's instructions and student achievement scores. Additionally, there is a strong correlation between levels of teaching quality and differences between schools and classrooms. However, measuring teaching quality has proven to be difficult and scholars strive to decide on the what and the how when aiming to measure teaching quality. This book discusses the many dilemmas of measuring teaching quality, be it substantial, theoretical, or methodological. This edited volume presents eight chapters, assigned authors provide updated and new knowledge on the many challenges linked to defining what teaching quality is and how it can be measured.
This collection of chapters originates from the discussions at the QUINT Conference 2019: Analysing Teaching Quality: Perspectives, Potential, and Pitfalls, organized by the Nordic Centre of Excellence: Quality in Nordic Teaching 18−20 June 2019 at the University of Oslo.
While there is broad agreement that teaching quality matters and that teachers’ instructional repertoires in classrooms are key requisites for students’ learning, measuring instructional quality has proven to be challenging. Scholars around the world strive to agree on the ‘what’, ‘how’, and even ‘why’ when measuring teaching quality. Against this backdrop, the Nordic Centre of Excellence: Quality in Nordic Teaching, QUINT, organized a conference with the theme “Analysing Teaching Quality: Perspectives, Principles and Pitfalls”. The conference brought together scholars from around the world, and made available different perspectives and approaches to teaching quality. In this book, we have invited some of the key contributors from the conference to provide insights into the matter of instructional quality. The book consists of eight chapters that together address both methodological, theoretical and substantial aspects when measuring teaching quality. While all chapters touch upon issues of theory, methodology and empirical findings, they are organized according to one these three main themes depending on the overall focus of the chapter.
Chapters
Why – and How – Should We Measure Instructional Quality?
Authors: Marte Blikstad-Balas, Michael Tengberg, and Kirsti Klette
Practice, Feedback, Argument, Measurement: A Frame for Understanding Diverse Perspectives on Teaching Assessments
Authors: Courtney Bell, and Robert Mislevy
Abstract: Every time we observe a lesson or create a rating of teaching, we are “assessing” teaching. Assessment is a complex enterprise. In addition to the issues that readily come to mind when we think of assessment—e.g., the degree to which an instrument captures the complexity of teaching, the meaning of scores, rater reliability, and the consequences associated with assessments—stakeholders frequently do not agree on the goals of assessing teaching. In this chapter, we describe four metaphors for understanding assessments of teaching: assessment as a cultural practice, a feedback loop, an evidentiary argument, and measurement. We illustrate how the use of different metaphors draws attention to various aspects of measuring and improving teaching. This has implications for what we study and how we carry out research on teaching. The analysis helps us understand why researchers, policy makers, and practitioners often have incompatible views of how to gather and use teaching assessment data.
The Surplus of Quality: How to Study Quality in Teaching in Three QUINT Projects
Author: Nikolaj Elf
Abstract: The QUINT ambition is to investigate quality in teaching through a systemic and differential framework. This chapter explores what this means with regards to three QUINT projects. I ask: What are the commonalities and differences when comparing the LISA, Connected Classrooms and Quality Literature Education projects’ theoretical and methodological approaches to investigating teaching quality in Nordic classrooms? One commonality is that all three projects investigate quality in everyday teaching practices. This unit of analysis is construed, however, in different ways that imply a multidimensional framework for understanding teaching quality.
A Validity Framework for the Design and Analysis of Studies Using Standardized Observation Systems
Author: Mark White
Abstract: I present a validity framework for comparing instructional quality across contexts using standardized observation systems. The framework explicitly breaks down the steps in operationalizing instructional quality and sampling instruction. In doing so, I highlight various levels to which observation scores can be generalized and the processes affecting generalization. The explicitness of the framework helps to structure potential validity arguments that are necessary to support study conclusions.
Multi-Group Measurement Invariance and Generalizability Analyses for an Instructional Quality Observational Instrument
Authors: Armin Jentsch, Hannah Heinrichs, Lena Schlesinger, Gabriele Kaiser,
Johannes König, and Sigrid Blömeke
Abstract: Based on a theoretical framework of instructional quality, we developed an observational instrument with which generic and subject-specific dimensions were measured in two samples of secondary mathematics classrooms (N = 76). As a step towards validating our assumptions on observer ratings, we combine generalizability and measurement invariance analysis to investigate the psychometric properties of the observational instrument.
Instructional Quality: A Review of Conceptualizations, Measurement Approaches, and Research Findings
Authors: Bas Senden, Trude Nilsen, and Sigrid Blömeke
Abstract: This chapter reviews and discusses conceptualizations, measurement approaches, and research findings of instructional quality. Although agreement on how to conceptualize and measure instructional quality is rare, some common ground can be found. In addition, research findings indicate that the role of instructional quality for student learning might vary across contexts, hinting towards the importance of differential effectiveness for instructional quality.
Observational Scores as Predictors for Student Achievement Gains
Authors: Kirsti Klette, Astrid Roe, and Marte Blikstad-Balas
Abstract: The present study draws on video data from 180 Norwegian language arts (LA) lessons and 171 mathematics lessons in eighth-grade classrooms in Norway and achievement gains in reading and numeracy for all students in these classes. Both language arts and mathematics lessons were analyzed using the Protocol for Language Arts Teaching Observations (PLATO) to score 12 elements of LA and mathematics instruction on a scale from 1 to 4. The students’ achievement gains in reading and numeracy are based on results from the national reading and numeracy tests at the beginning of eighth and the beginning of and in ninth grade (before and after the video recordings). The aim of the study is to investigate possible correlations between PLATO scores and achievement gains, for all students and for sub groups of students. Our analyses suggest that (a) instructional facets might support students differentially; (b) high-achieving students might profit more from the indicators of instructional quality as measured though PLATO and (c) instructional quality may encompass different features in different subjects.
Cognitive Activation Potential of E&S Tasks at Commercial Vocational Schools in German-Speaking Switzerland
Author: Eva Weingartner
Abstract: This study focuses on tasks in Economy & Society used in a video study at commercial vocational schools and analyses the objective and realized cognitive activation potential. The results show that the objective cognitive activation potential is generally on a low to medium level and rarely changed through implementation in class. The insights of this study are useful for future teacher trainings.
Exploring the Potential in Using Teachers’ Intended Lesson Goals as a Context-Sensitive Lens to Understanding Observational Scores of Instructional Quality
Authors: Jennifer Maria Luoto and Alexander Jonas Viktor Selling
Teachers’ perspectives of their intended lesson goals are often not a key focus in classroom observation research on instructional quality, but we argue that insight into teachers’ goals can help us better understand quality in classroom practice. This study explores the potential in using teachers’ intended lesson goals as complementary lenses to better understand patterns of instructional quality derived by a standardized observation system. Three Finnish-Swedish 9th grade mathematics teachers were observed during a sequence of three–four consecutive lessons, and interviewed about their lesson goals. The lessons were analyzed with the observation system Protocol for Language Arts Teaching Observations (PLATO), resulting in patterns of instructional quality that were contrasted with teachers’ intended lesson goals. Findings indicate that patterns of instructional quality vary greatly across teachers and within each teacher’s lesson, depending on the goals they have for the activities in the lessons. As expected due to the limitation of any observation system, several of the identified teachers’ goals were not captured with PLATO, while the goals that were captured generally reflected teachers’ practice in high PLATO scores. This indicates that teachers’ lesson goals can shed light on patterns of instructional quality, and that lesson goals may be used to target which aspects of instructional quality teachers find relevant for certain activities and lessons. We discuss the potential but also some of the difficulties in combining standardized observations with context-sensitive aspects such as teachers’ lesson goals—to better understand instructional quality, as well as how observation scores can be used to assist teachers in reaching their intended goals.
Full info
Blikstad-Balas Marte, Kirsti Klette & Michael Tengberg (2021) Ways of Analyzing Teaching Quality: Potential and Pitfalls, Universitetsforlaget. DOI: https://doi.org/10.18261/9788215045054-2021