Our latest publications

• Ye, Wangqiong; Scherer, Ronny & Blömeke, Sigrid (2024). Teachers’ and principals’ perceptions of school emphasis on academic success: measurement invariance, agreement, and relations to student achievement. Large-scale Assessments in Education. ISSN 2196-0739. 12. doi: 10.1186/s40536-024-00207-w.
• Ulitzsch, Esther; Buchholz, Janine; Shin, Hyo Jeong; Bertling, Jonas & Lüdtke, Oliver (2024). Using a novel multiple-source indicator to investigate the effect of scale format on careless and insufficient effort responding in a large-scale survey experiment. Large-scale Assessments in Education. ISSN 2196-0739. 12(1). doi: 10.1186/s40536-024-00205-y.
• Scherer, Ronny & Blömeke, Sigrid (2024). Teachers’ and principals’ perceptions of school emphasis on academic success: measurement invariance, agreement, and relations to student achievement. Large-scale Assessments in Education. ISSN 2196-0739. doi: 10.1186/s40536-024-00207-w.
• Ye, Wangqiong; Olsen, Rolf Vegar & Blömeke, Sigrid (2024). More money does not necessarily help: relations of education expenditure, school characteristics, and academic resilience across 36 education systems. Frontiers in Education. ISSN 2504-284X. 9. doi: 10.3389/feduc.2024.1368642.
• Mahler, Daniela; Bock, Denise; Schauber, Stefan & Harms, Ute (2024). Using longitudinal models to describe preservice science teachers’ development of content knowledge and pedagogical content knowledge. Teaching and Teacher Education : An International Journal of Research and Studies. ISSN 0742-051X. 144. doi: 10.1016/j.tate.2024.104583. Full text in Research Archive
• Arencibia Aleman, Jose Manuel; Sandsør, Astrid Marie Jorde; Zachrisson, Henrik Daae & Blömeke, Sigrid (2024). Teacher-assigned grades and external exams: sources of discrepancy. Assessment in Education: Principles, Policy & Practice. ISSN 0969-594X. doi: 10.1080/0969594X.2024.2338764.
• Mutak, Augustin; Krause, Robert; Ulitzsch, Esther; Much, Sören; Ranger, Jochen & Pohl, Steffi (2024). Modeling the Intraindividual Relation of Ability and Speed within a Test. Journal of Educational Measurement. ISSN 0022-0655. doi: 10.1111/jedm.12391.
• Veletic, Jelena; Mejía-Rodríguez, Ana María & Olsen, Rolf Vegar (2024). A systematic literature review of TALIS secondary research: Trends and future directions. Review of Education. ISSN 2049-6613. 12(1). doi: 10.1002/rev3.3469. Full text in Research Archive
• Acar, Selcuk; Lee, Lindsay Ellis & Scherer, Ronny (2024). A Reliability Generalization of the Torrance Tests of Creative Thinking-Figural. European Journal of Psychological Assessment. ISSN 1015-5759. doi: 10.1027/1015-5759/a000819.
• Adnan, Müge; Tondeur, Jo; Scherer, Ronny & Siddiq, Fazilat (2024). Profiling teacher educators: ready to prepare the next generation for educational technology use? Technology, Pedagogy and Education. ISSN 1475-939X. doi: 10.1080/1475939X.2024.2322481.
• Jentsch, Armin; Benecke, Kirsten; Blömeke, Sigrid; König, Johannes & Kaiser, Gabriele Agnes Hildegard (2024). Effects of observation mode on ratings of teaching quality in secondary mathematics classrooms: Authors and their affiliations. ZDM: Mathematics Education. ISSN 1863-9690. doi: 10.1007/s11858-024-01557-z.
• Van Laar, Saskia; Chen, Jianan & Braeken, Johan (2024). How Randomly are Students Random Responding to Your Questionnaire? Within-Person Variability in Random Responding Across Scales in the TIMSS 2015 Eighth-Grade Student Questionnaire. Measurement: Interdisciplinary Research and Perspectives. ISSN 1536-6367. 22(1), p. 90–107. doi: 10.1080/15366367.2023.2203972.
• Chen, Jianan; Steinmann, Isa & Braeken, Johan (2024). Competing explanations for inconsistent responding to a mixed-worded self-esteem scale: Cognitive abilities or personality? Personality and Individual Differences. ISSN 0191-8869. 222, p. 1–6. doi: 10.1016/j.paid.2024.112573. Full text in Research Archive
• Steinmann, Isa; Chen, Jianan & Braeken, Johan (2024). Who Responds Inconsistently to Mixed-Worded Scales? Differences by Achievement, Age Group, and Gender. Assessment in Education: Principles, Policy & Practice. ISSN 0969-594X. doi: 10.1080/0969594X.2024.2318554. Full text in Research Archive
• Scherer, Ronny; Siddiq, Fazilat & Nilsen, Trude (2024). The potential of international large-scale assessments for meta-analyses in education. Large-scale Assessments in Education. ISSN 2196-0739. 12(1). doi: 10.1186/s40536-024-00191-1. Full text in Research Archive
• Zhang, Maoxin; Andersson, Björn & Jin, Shaobo (2024). Fast estimation of generalized linear latent variable models for performance and process data with ordinal, continuous, and count observed variables. British Journal of Mathematical & Statistical Psychology. ISSN 0007-1102. doi: 10.1111/bmsp.12337. Full text in Research Archive
• Campos, Diego Gonzalez; Fütterer, Tim; Gfrörer, Thomas; Lavelle-Hill, Rosa; Murayama, Kou & König, Lars [Show all 9 contributors for this article] (2024). Screening Smarter, Not Harder: A Comparative Analysis of Machine Learning Screening Algorithms and Heuristic Stopping Criteria for Systematic Reviews in Educational Research. Educational Psychology Review. ISSN 1040-726X. 36(1). doi: 10.1007/s10648-024-09862-5. Full text in Research Archive
• Karimi, Roya; Mallah, Narmeen; Scherer, Ronny; Rodríguez-Cano, Rubén & Takkouche, Bahi (2023). Sleep quality as a mediator of the relation between depression and chronic pain: a systematic review and meta-analysis. British Journal of Anaesthesia. ISSN 0007-0912. doi: 10.1016/j.bja.2023.02.036.
• Schauber, Stefan & Stensløkken, Kåre-Olav (2023). No knowledge gap in human physiology after remote teaching for second year medical students throughout the Covid-19 pandemic. BMC Medical Education. ISSN 1472-6920. 23(1). doi: 10.1186/s12909-023-04959-x. Full text in Research Archive
• Kristensen, Jarl Kleppe; Torkildsen, Janne von Koss & Andersson, Björn (2023). Repeated mistakes in app-based language learning: Persistence and relation to learning gains. Computers & Education. ISSN 0360-1315. 210. doi: 10.1016/j.compedu.2023.104966. Full text in Research Archive
• Chen, Jianan; van Laar, Saskia & Braeken, Johan (2023). Who are those random responders on your survey? The case of the TIMSS 2015 student questionnaire. Large-scale Assessments in Education. ISSN 2196-0739. 11(1). doi: 10.1186/s40536-023-00184-6. Full text in Research Archive Show summary

  A general validity and survey quality concern with student questionnaires under low-stakes assessment conditions is that some responders will not genuinely engage with the questionnaire, often with more random response patterns as a result. Using a mixture IRT approach and a meta-analytic lens across 22 educational systems participating in TIMSS 2015, we investigated whether the prevalence of random responders on six scales regarding students' engagement and attitudes toward mathematics and sciences was a function of grade, gender, socio-economic status, spoken language at home, or migration background. Among these common policy-relevant covariates in educational research, we found support for small group differences in prevalence of random responders (OR ≥ 1.22)(average prevalence of 7%). In general, being a student in grade 8 (vs. grade 4), being male, reporting to have fewer books, or speaking a language different from the test language at home were all risk factors characterizing random responders. The expected generalization and implications of these findings are discussed based on the observed heterogeneity across educational systems and consistency across questionnaire scales.

• Campos, Diego Gonzalez & Scherer, Ronny (2023). Digital gender gaps in Students’ knowledge, attitudes and skills: an integrative data analysis across 32 Countries. Education and Information Technologies : Official Journal of the IFIP technical committee on Education. ISSN 1360-2357. 29, p. 655–693. doi: 10.1007/s10639-023-12272-9. Full text in Research Archive
• Van Laar, Saskia & Braeken, Johan (2023). Prevalence of random responders as a function of scale position and questionnaire length in the TIMSS 2015 eighth-grade student questionnaire. International Journal of Testing. ISSN 1530-5058. doi: 10.1080/15305058.2023.2263206. Full text in Research Archive
• Cabellos, Beatriz; Siddiq, Fazilat & Scherer, Ronny (2023). The moderating role of school facilitating conditions and attitudes towards ICT on teachers' ICT use and emphasis on developing students’ digital skills. Computers in Human Behavior. ISSN 0747-5632. 150. doi: 10.1016/j.chb.2023.107994. Full text in Research Archive
• Strello, Andrés; Strietholt, Rolf & Steinmann, Isa (2023). Mind the Gap… But Which Gap? The Distinctions Between Social Inequalities in Student Achievement. Social Indicators Research. ISSN 0303-8300. doi: 10.1007/s11205-023-03196-5.
• Marcq, Kseniia & Braeken, Johan (2023). Gender differences in item nonresponse in the PISA 2018 student questionnaire. Educational Assessment, Evaluation and Accountability. ISSN 1874-8597. doi: 10.1007/s11092-023-09412-7. Full text in Research Archive
• Zhang, Maoxin; Andersson, Björn & Greiff, Samuel (2023). Investigating Planning and Non-Targeted Exploration in PIAAC 2012: Validating Their Measures Based on Process Data and Investigating Their Relationships with Problem-Solving Competency. Journal of Intelligence. ISSN 2079-3200. 11(8). doi: 10.3390/jintelligence11080156. Full text in Research Archive
• Siddiq, Fazilat; Røkenes, Fredrik Mørk; Lund, Andreas & Scherer, Ronny (2023). New kid on the block? a conceptual systematic review of digital agency. Education and Information Technologies : Official Journal of the IFIP technical committee on Education. ISSN 1360-2357. doi: 10.1007/s10639-023-12038-3. Full text in Research Archive
• Marcin, Thimo; Lüthi, Ailin; Graf, Ronny R; Krummrey, Gert; Schauber, Stefan & Breakey, Neal [Show all 8 contributors for this article] (2023). Is language an issue? Accuracy of the German computerized diagnostic decision support system ISABEL and cross-validation with the English counterpart. Diagnosis. ISSN 2194-8011. doi: 10.1515/dx-2023-0047.
• Tondeur, Jo; Howard, Sarah K.; Scherer, Ronny & Siddiq, Fazilat (2023). Untangling the Great Online Transition: A network model of teachers’ experiences with online practices. Computers & Education. ISSN 0360-1315. 203. doi: 10.1016/j.compedu.2023.104866. Full text in Research Archive
• Zumbo, Bruno D.; Maddox, Bryan & Care, Naomi M. (2023). Process and Product in Computer-Based Assessments: Clearing the Ground for a Holistic Validity Framework. European Journal of Psychological Assessment. ISSN 1015-5759. 39(4). doi: 10.1027/1015-5759/a000748.
• Marcin, Thimo; Hautz, Stefanie C.; Singh, Hardeep; Zwaan, Laura; Schwappach, David & Krummey, Gert [Show all 16 contributors for this article] (2023). Effects of a computerised diagnostic decision support tool on diagnostic quality in emergency departments: Study protocol of the DDx-BRO multicentre cluster randomised cross-over trial. BMJ Open. ISSN 2044-6055. 13(3). doi: 10.1136/bmjopen-2023-072649.
• Zhang, Maoxin & Andersson, Björn (2023). Identifying Problem-Solving Solution Patterns Using Network Analysis of Operation Sequences and Response Times. Educational Assessment. ISSN 1062-7197. doi: 10.1080/10627197.2023.2222585. Full text in Research Archive Show summary

  Process data from educational assessments enhance the understanding of how students answer cognitive items. However, effectively making use of these data is challenging. We propose an approach to identify solution patterns from operation sequences and response times by generating networks from process data and defining network features that extract essential information from them. With these features, we group respondents to a problem-solving task from PISA 2012 using Gaussian mixture models. The results indicate the presence of two and four clusters for groups defined by failure and success on the task, respectively. We interpret the clusters as less-able, low-effort, adaptable, back-and-forth, deliberate, and trial-and-error clusters by considering the cluster-specific feature statistics. The proposed approach sheds light on students’ problem-solving mental processes, which can aid item development and facilitate individualized feedback to students. The method is applicable to many computer-based problems, but a limitation is that the feature definitions can be task-dependent.

• Veletic, Jelena; Price, Heather & Olsen, Rolf Vegar (2023). Teachers’ and principals’ perceptions of school climate: the role of principals’ leadership style in organizational quality. Educational Assessment, Evaluation and Accountability. ISSN 1874-8597. doi: 10.1007/s11092-023-09413-6. Full text in Research Archive
• Teig, Nani & Steinmann, Isa (2023). Leveraging large-scale assessments for effective and equitable school practices: the case of the nordic countries. Large-scale Assessments in Education. ISSN 2196-0739. 11(1). doi: 10.1186/s40536-023-00172-w. Full text in Research Archive
• Hübner, Nicolas; Fahrbach, Thorsten; Lachner, Andreas & Scherer, Ronny (2023). What predicts students’ future ICT literacy? Evidence from a large-scale study conducted in different stages of secondary school. Computers & Education. ISSN 0360-1315. 203. doi: 10.1016/j.compedu.2023.104847.
• Ji, Feng; Rabe-Hesketh, Sophia & Skrondal, Anders (2023). Diagnosing and Handling Common Violations of Missing at Random. Psychometrika. ISSN 0033-3123. 88, p. 1123–1143. doi: 10.1007/s11336-022-09896-0. Full text in Research Archive
• Chen, Chia-Wen & Chen-Wei, Liu (2023). Online Parameter Estimation for Student Evaluation of Teaching. Applied Psychological Measurement. ISSN 0146-6216. doi: 10.1177/01466216231165314.
• Kristensen, Jarl Kleppe; Andersson, Björn; Bratlie, Siri Steffensen & Torkildsen, Janne von Koss (2023). Dimensionality of Morphological Knowledge—Evidence from Norwegian Third Graders. Reading Research Quarterly. ISSN 0034-0553. doi: 10.1002/rrq.497. Full text in Research Archive
• Scherer, Ronny; Siddiq, Fazilat; Howard, Sarah K. & Tondeur, Jo (2023). Gender divides in teachers' readiness for online teaching and learning in higher education: Do women and men consider themselves equally prepared? Computers & Education. ISSN 0360-1315. 199, p. 1–16. doi: 10.1016/j.compedu.2023.104774. Full text in Research Archive
• Fütterer, Tim; Scherer, Ronny; Scheiter, Katharina; Stürmer, Kathleen & Lachner, Andreas (2023). Will, skills, or conscientiousness: What predicts teachers' intentions to participate in technology-related professional development? Computers & Education. ISSN 0360-1315. 198, p. 1–17. doi: 10.1016/j.compedu.2023.104756. Full text in Research Archive
• Andersson, Björn; Jin, Shaobo & Zhang, Maoxin (2023). Fast estimation of multiple group generalized linear latent variable models for categorical observed variables. Computational Statistics & Data Analysis. ISSN 0167-9473. 182. doi: 10.1016/j.csda.2023.107710. Full text in Research Archive
• Reis Costa, Denise & Chen, Chia-Wen (2023). Exploring the relationship between process data and contextual variables among Scandinavian students on PISA 2012 mathematics tasks. Large-scale Assessments in Education. ISSN 2196-0739. 11:5, p. 1–28. doi: 10.1186/s40536-023-00155-x. Full text in Research Archive
• Hukkelberg, Silje Sommer & Andersson, Björn (2023). Assessing social competence and antisocial behaviors in children: item response theory analysis of the home and community social behavior scales. BMC Psychology. ISSN 2050-7283. 11(1). doi: 10.1186/s40359-023-01045-1. Full text in Research Archive
• Campos, Diego Gonzalez; Cheung, M. W. & Scherer, Ronny (2023). A Primer on Synthesizing Individual Participant Data Obtained From Complex Sampling Surveys: A Two-Stage IPD Meta-Analysis Approach. Psychological methods. ISSN 1082-989X. doi: 10.1037/met0000539. Show summary

  The increasing availability of individual participant data (IPD) in the social sciences offers new possibilities to synthesize research evidence across primary studies. Two-stage IPD meta-analysis represents a framework that can utilize these possibilities. While most of the methodological research on two-stage IPD meta-analysis focused on its performance compared with other approaches, dealing with the complexities of the primary and meta-analytic data has received little attention, particularly when IPD are drawn from complex sampling surveys. Complex sampling surveys often feature clustering, stratification, and multistage sampling to obtain nationally or internationally representative data from a target population. Furthermore, IPD from these studies is likely to provide more than one effect size. To address these complexities, we propose a two-stage meta-analytic approach that generates model-based effect sizes in Stage 1 and synthesizes them in Stage 2. We present a sequence of steps, illustrate their implementation, and discuss the methodological decisions and options within. Given its flexibility to deal with the complex nature of the primary and meta-analytic data and its ability to combine multiple IPD sets or IPD with aggregated data, the proposed two-stage approach opens up new analytic possibilities for synthesizing knowledge from complex sampling surveys.

• Steinmann, Isa; Strello, Andrés & Strietholt, Rolf (2023). The effects of early between-school tracking on gender segregation and gender gaps in achievement: a differences-in-differences study. School Effectiveness and School Improvement. ISSN 0924-3453. doi: 10.1080/09243453.2023.2165510.
• Scherer, Ronny; Siddiq, Fazilat; Howard, Sarah K. & Tondeur, Jo (2023). The more experienced, the better prepared? New evidence on the relation between teachers’ experience and their readiness for online teaching and learning. Computers in Human Behavior. ISSN 0747-5632. 139. doi: 10.1016/j.chb.2022.107530. Full text in Research Archive
• Rutkowski, Leslie; Rutkowski, David & Valdivia, Svetina D. (2022). Multistage Test Design Considerations in International Large-Scale Assessments of Educational Achievement. In Nilsen, Trude; Stancel-Piątak, Agnes & Gustafsson, Jan-Eric (Ed.), International Handbook of Comparative Large-Scale Studies in Education. Springer. ISSN 978-3-030-38298-8. p. 749–767.
• Rabe-Hesketh, Sophia & Skrondal, Anders (2022). Ignoring Non-ignorable Missingness. Psychometrika. ISSN 0033-3123. p. 1–20. doi: 10.1007/s11336-022-09895-1. Full text in Research Archive
• Emslander, Valentin; Levy, Jessica; Scherer, Ronny & Fischbach, Antoine (2022). Value-added Scores Show Limited Stability over Time in Primary School. PLOS ONE. ISSN 1932-6203. 17(12). doi: 10.1371/journal.pone.0279255. Full text in Research Archive
• Heltne, Aleksander; Braeken, Johan; Hummelen, Benjamin; Germans Selvik, Sara; Christensen, Tore Buer & Paap, Muirne C. S. (2022). Do Flexible Administration Procedures Promote Individualized Clinical Assessments? An Explorative Analysis of How Clinicians Utilize the Funnel Structure of the SCID-5-AMPD Module I: LPFS. Journal of Personality Assessment. ISSN 0022-3891. doi: 10.1080/00223891.2022.2152344. Full text in Research Archive
• Stahnke, Rebekka & Blömeke, Sigrid (2022). Wie erfahrene und angehende Lehrkräfte Unterricht wahrnehmen. Journal für LehrerInnenbildung. ISSN 2629-4982. 22(4), p. 76–85. Full text in Research Archive
• Schwarz, Björn; Döhrmann, Martina & Blömeke, Sigrid (2022). Studien zur professionellen Kompetenz von Mathematiklehrkräften – Das TEDS-Forschungsprogramm im Überblick. In Buchholtz, Nils (Eds.), Initiationen mathematikdidaktischer Forschung. Springer Berlin/Heidelberg. ISSN 978-3-658-36765-7.
• Jentsch, Armin; Schlesinger, Lena & Blömeke, Sigrid (2022). Grundzüge einer Validierungsstrategie zur Erfassung von Unterrichtsqualität im Mathematikunterricht. In Buchholtz, Nils (Eds.), Initiationen mathematikdidaktischer Forschung. Springer Berlin/Heidelberg. ISSN 978-3-658-36765-7.
• Blömeke, Sigrid (2022). IEA’s Teacher Education and Development Study in Mathematics (TEDS-M): Framework and Findings from 17 Countries. In Nilsen, Trude; Stancel-Piątak, Agnes & Gustafsson, Jan-Eric (Ed.), International Handbook of Comparative Large-Scale Studies in Education. Springer. ISSN 978-3-030-38298-8. doi: 10.1007/978-3-030-88178-8_16.
• Reis Costa, Denise & Leoncio Netto, Waldir (2022). Process Data Analysis in ILSAs. In Nilsen, Trude; Stancel-Piątak, Agnes & Gustafsson, Jan-Eric (Ed.), International Handbook of Comparative Large-Scale Studies in Education. Springer. ISSN 978-3-030-38298-8. p. 927–952. doi: 10.1007/978-3-030-88178-8_60.
• Scherer, Ronny & Nilsen, Trude (2022). Accountability for the future? International testing and future directions. In Tierney, Robert J.; Rizvi, Fazal & Erkican, Kadriye (Ed.), International Encyclopedia of Education (4th Edition). Elsevier. ISSN 978-0-12-818629-9. p. 324–340. doi: 10.1016/B978-0-12-818630-5.09060-6.
• Luo, Hao; Andersson, Björn; Wong, Gloria & Lum, Terry (2022). Longitudinal measurement properties of the Montreal Cognitive Assessment. Journal of Clinical and Experimental Neuropsychology. ISSN 1380-3395. p. 1–13. doi: 10.1080/13803395.2022.2148634. Full text in Research Archive
• Ræder, Henrik Galligani; Andersson, Björn & Olsen, Rolf Vegar (2022). Numeracy across grades–vertically scaling the Norwegian national numeracy tests. Assessment in Education: Principles, Policy & Practice. ISSN 0969-594X. p. 1–21. doi: 10.1080/0969594X.2022.2147483. Full text in Research Archive
• Frans, Niek; Braeken, Johan; Veldkamp, Bernard P. & Paap, Muirne C. S. (2022). Empirical Priors in Polytomous Computerized Adaptive Tests: Risks and Rewards in Clinical Settings. Applied Psychological Measurement. ISSN 0146-6216. 47(1), p. 48–63. doi: 10.1177/01466216221124091. Full text in Research Archive
• Emslander, Valentin & Scherer, Ronny (2022). The Relation Between Executive Functions and Math Intelligence in Preschool Children: A Systematic Review and Meta-Analysis. Psychological bulletin. ISSN 0033-2909. 148(5-6), p. 337–369. doi: 10.1037/bul0000369.
• Blömeke, Sigrid; Nilsen, Trude; Olsen, Rolf Vegar & Gustafsson, Jan-Eric (2022). Conceptual and Methodological Accomplishments of ILSAs, Remaining Criticism and Limitations. In Nilsen, Trude; Stancel-Piątak, Agnes & Gustafsson, Jan-Eric (Ed.), International Handbook of Comparative Large-Scale Studies in Education. Springer. ISSN 978-3-030-38298-8. p. 603–656.
• Chen, Chia-Wen; Andersson, Björn & Zhu, Jinxin (2022). A Factor Mixture Model for Item Responses and Certainty of Response Indices to Identify Student Knowledge Profiles. Journal of Educational Measurement. ISSN 0022-0655. doi: 10.1111/jedm.12344. Full text in Research Archive
• Michel, Janet; Mettler, Annette; Stuber, Raphael; Müller, Martin; Ricklin, Meret E & Jent, Philipp [Show all 8 contributors for this article] (2022). Effects and utility of an online forward triage tool during the SARS-CoV-2 pandemic: a mixed method study and patient perspectives, Switzerland. BMJ Open. ISSN 2044-6055. 12(7). doi: 10.1136/bmjopen-2021-059765. Full text in Research Archive
• Jentsch, Armin; Hoferichter, Frances; Blömeke, Sigrid; König, Johannes & DR Kaiser, Gabriele Agnes Hildegard (2022). Investigating teachers' job satisfaction, stress and working environment: The roles of self-efficacy and school leadership. Psychology in the Schools. ISSN 0033-3085. 60(3), p. 679–690. doi: 10.1002/pits.22788. Full text in Research Archive Show summary

  This study investigates the relations between working environment and teachers' job satisfaction, perceived work-related stress, as well as work-related self-efficacy. The sample consisted of 226 mathematics teachers from German secondary schools. About 55% were female and they had been teaching for 13 years on average. We used self-reported measures to assess how teachers perceived their working environment (regarding autonomy, feedback, and social support by colleagues), administrative leadership and teachers' work-related self-efficacy, as well as job satisfaction and work-related stress. Structural equation modeling demonstrates that teachers' job satisfaction and stress were significantly associated with self-efficacy (moderate to large effects) and an administrative leadership at the corresponding schools (small to moderate effects). The effect of social support on teachers' job satisfaction and stress was fully mediated by teachers' self-efficacy. Our findings underscore the importance of self-efficacy and a positive working environment for teachers' job satisfaction and stress.

• Rutkowski, Leslie; Liaw, Yuan-Ling; Svetina, Dubravka & Rutkowski, David (2022). Multistage Testing in Heterogeneous Populations: Some Design and Implementation Considerations. Applied Psychological Measurement. ISSN 0146-6216. 46(6), p. 494–508. doi: 10.1177/01466216221108123.
• Mittal, Oleksandra; Scherer, Ronny & Nilsen, Trude (2022). Assessing the evidence for the comparability of socioeconomic status between students with and without immigrant background in Norway and Sweden. Large-scale Assessments in Education. ISSN 2196-0739. 10(1). doi: 10.1186/s40536-022-00132-w. Full text in Research Archive
• Scherer, Ronny & Campos, Diego Gonzalez (2022). Measuring those who have their minds set: An item-level meta-analysis of the implicit theories of intelligence scale in education. Educational Research Review. ISSN 1747-938X. 37. doi: 10.1016/j.edurev.2022.100479. Full text in Research Archive
• Lotz, Christin; Scherer, Ronny; Greiff, Samuel & Sparfeldt, J. R. (2022). g's little helpers – VOTAT and NOTAT mediate the relation between intelligence and complex problem solving. Intelligence. ISSN 0160-2896. 95. doi: 10.1016/j.intell.2022.101685.
• Blömeke, Sigrid; Jenßen, Lars & Eid, Michael (2022). The Role of Intelligence and Self-Concept for Teachers’ Competence. Journal of Intelligence. ISSN 2079-3200. 10(2). doi: 10.3390/jintelligence10020020. Full text in Research Archive
• Teig, Nani; Scherer, Ronny & Olsen, Rolf Vegar (2022). A systematic review of studies investigating science teaching and learning: over two decades of TIMSS and PISA. International Journal of Science Education. ISSN 0950-0693. 44(12), p. 2035–2058. doi: 10.1080/09500693.2022.2109075. Full text in Research Archive Show summary

  A great number of studies have investigated science teaching and learning (STL) using data from the Trends in International Mathematics and Science Study (TIMSS) and Programme for International Student Assessment (PISA). Nevertheless, there is little effort to synthesise these studies. Reviewing the status of research on STL, including the trends, approaches, and findings is crucial for identifying research gaps that require greater attention. Consequently, this review synthesises empirical studies investigating STL using TIMSS and PISA. First, we mapped their characteristics based on the aims, data, STL measures, and research approaches. Second, patterns of the findings were summarized by focusing on the (1) relationships between STL and student outcomes; (2) factors that explain STL variation; and (3) patterns of STL. Most studies examined STL related to inquiry activities and their relationships to student outcomes. Their findings were largely inconsistent and could be traced back to different ways STL was conceptualized and measured. This review calls for the studies examining TIMSS and PISA data to enhance the quality of research approaches and clarity in reporting them. It also encourages researchers to reflect upon the knowledge gained from harnessing these data to examine STL and discusses the challenges and opportunities that lie ahead.

• Marcq, Kseniia & Braeken, Johan (2022). The blind side: Exploring item variance in PISA 2018 cognitive domains. Assessment in Education: Principles, Policy & Practice. ISSN 0969-594X. 29(3), p. 332–360. doi: 10.1080/0969594X.2022.2097199. Full text in Research Archive
• Lopez-Pedersen, Anita; Mononen, Riikka; Aunio, Pirjo Annika; Scherer, Ronny & Melby-Lervåg, Monica (2022). Improving Numeracy Skills in First Graders with low performance in early numeracy: A Randomized Controlled Trial. Remedial and special education. ISSN 0741-9325. doi: 10.1177/07419325221102537. Full text in Research Archive
• Jenßen, Lars; Dunekacke, Simone; Eid, Michael; Szczesny, Markus; Pohle, Lara & Koinzer, Thomas [Show all 8 contributors for this article] (2022). From teacher education to practice: Development of early childhood teachers‘ knowledge and beliefs in mathematics. Teaching and Teacher Education : An International Journal of Research and Studies. ISSN 0742-051X. 114. doi: 10.1016/j.tate.2022.103699. Full text in Research Archive
• Blömeke, Sigrid; Jentsch, Armin; Ross, Natalie; Kaiser, Gabriele & König, Johannes (2022). Opening up the black box: Teacher competence, instructional quality, and students’ learning progress. Learning and Instruction. ISSN 0959-4752. 79, p. 1–11. doi: 10.1016/j.learninstruc.2022.101600. Full text in Research Archive
• Busch, Tobias; Brinchmann, Ellen Irén; Braeken, Johan & Wie, Ona Bø (2022). Receptive Vocabulary of Children With Bilateral Cochlear Implants From 3 to 16 Years of Age. Ear and Hearing. ISSN 0196-0202. 43(6), p. 1866–1880. doi: 10.1097/AUD.0000000000001220. Full text in Research Archive
• Jenßen, Lars; Roesken-Winter, Bettina & Blömeke, Sigrid (2022). Measuring Pre-service Primary Teachers’ Shame in Mathematics—a Comprehensive Validation Study. International Journal of Science and Mathematics Education. ISSN 1571-0068. doi: 10.1007/s10763-022-10253-x. Full text in Research Archive
• Van Laar, Saskia & Braeken, Johan (2022). Random Responders in the TIMSS 2015 Student Questionnaire: A Threat to Validity? Journal of Educational Measurement. ISSN 0022-0655. p. 1–32. doi: 10.1111/jedm.12317. Full text in Research Archive
• Van Laar, Saskia & Braeken, Johan (2022). Caught off Base: A Note on the Interpretation of Incremental Fit Indices. Structural Equation Modeling. ISSN 1070-5511. 29(6), p. 935–943. doi: 10.1080/10705511.2022.2050730. Full text in Research Archive

View all works in Cristin

• Liu, Chen-Wei; Andersson, Björn & Skrondal, Anders (2024). Erratum to: A Constrained Metropolis–Hastings Robbins–Monro Algorithm for Q Matrix Estimation in DINA Models (Psychometrika, (2020), 85, 2, (322-357), 10.1007/s11336-020-09707-4). Psychometrika. ISSN 0033-3123. doi: 10.1007/s11336-024-09974-5.
• Ulitzsch, Esther; He, Qiwei & Pohl, Steffi (2024). Innovations in Exploring Sequential Process Data. Zeitschrift für Psychologie. ISSN 2190-8370. 232(2), p. 71–73. doi: 10.1027/2151-2604/a000560.
• Leoncio Netto, Waldir; Matta, Tyler; Rutkowski, Leslie; Rutkowski, David & Liaw, Yuan-Ling (2023). lsasim: Functions to Facilitate the Simulation of Large Scale Assessment Data.
• Olsen, Rolf Vegar (2023). Hvilke mål for kvalitet trenger norsk skole?
• Melby-Lervåg, Monica; Amland, Tonje; Grande, German; Scherer, Ronny & Lervåg, Arne (2023). Does language underlie mathematical skill development? A systematic review and meta-analysis of concurrent and longitudinal predictors .
• Steinmann, Isa & Rutkowski, Leslie (2023). The Link between Gender Gaps in School Enrollment and School Achievement.
• Siddiq, Fazilat; Tondeur, Jo; Howard, Sarah K. & Scherer, Ronny (2023). New methods, new insights: Making sense of teachers' experiences with online education. Show summary

  The Covid-19 pandemic provided a unique opportunity to understand school teachers' perceptions of and experiences with online practices on a large scale. However, there is a need for a more detailed understanding of the multiple associations among teachers’ perceptions and experiences in change processes. Therefore, the main goal of the current study is to untangle the complexity of teachers' experiences and perceptions during this great online transition. Based on the data of 222 teachers' responses to open-ended questions, we used a combined method of thematic and network analysis. To illustrate this novel method, we will focus on the strongest associations between the categories “learning design for online teaching” and “Student learning”. The results indicated for instance a strong association between ‘Self-paced’, which was a dimension of ‘learning design’, and ‘Flexibility’, which was discussed in relation to ‘student experience’. During the presentation the authors will provide more examples of the associations between teachers’ online experiences and perceptions. They will also demonstrate how the network model together with the thematic analysis can be used as a 'roadmap' towards a more integrated approach of online and blended learning in schools and to guide future professional development efforts and school policies.

• Steinmann, Isa; Chen, Jianan & Braeken, Johan (2023). Challenging the design principle of mixed-worded questionnaire scales.
• Zitzmann, Steffen; Machts, Nils; Hübner, Nicolas; Schauber, Stefan; Möller, Jens & Lindner, Christoph (2023). The Yet Underestimated Importance of Communicating Findings from Educational Trials to Teachers, Schools, School Authorities, or Policy Makers (Comment on Brady et al. (2023)). Educational Psychology Review. ISSN 1040-726X. doi: 10.1007/s10648-023-09776-8.
• Gochyeev, Perman; Siddiq, Fazilat; Scherer, Ronny & WIlson, Mark (2022). Exploring collaborative problem-solving tasks for individual-level and team-level inferences using multilevel modeling. Show summary

  Collaborative problem solving (ColPS) is necessary to succeed in the 21st century and has therefore been introduced into compulsory K-12 education in many Western countries. ColPS is the “capacity of an individual to: recognize the perspective of other persons in a group; participate as a member of the group by contributing their knowledge, experience, and expertise constructively; recognize the need for contributions and how to manage them; identify structure and procedure involved in resolving a problem; and, as a member of the collaborative group, build and develop knowledge and understanding” (Griffin et al., 2012, p.7). Several aspects need consideration when designing ColPS tasks, including group dependence, team size, and activities to promote collaboration and problem solving (Siddiq & Scherer, 2017). Since students are often randomly assigned to teams, observations within groups are not independent. Hence, psychometric models need to account for the hierarchical structure of the data. Wilson et al. (2017) presented an approach for analyzing ColPS tasks using multilevel models, which are widely used in educational research. The authors focused on modeling the group effects on the overall test performances. We propose focusing on group effects on individual task performance instead. We aim to classify ColPS tasks into ones with and without ignorable group effects, implying lower and higher collaboration. We also propose calibrating these tasks using a two-dimensional Rasch model to further understand the two constructs these items represent. We analyse data collected from the Norwegian Digital literacy (LDN-ICT) instrument (Siddiq et al., 2017). In total, 39 tasks were administered to 144 ninth-grade students who worked in groups of three or four. Prior to modelling, it will be helpful to use descriptive statistics in an exploratory way to establish a framing context for modelling. Preliminary findings suggest that tasks showing significant group effects can be identified based on the estimated intraclass correlation (ICC)—a commonly used measure of group dependence. ICCs closer to zero indicate ignorable group effects, while higher ICCs imply substantial group effects. The ICCs ranged between 0.00 and 0.88, with an average ICC of 0.31. We also found a significant agreement between estimated group dependence and intended group dependence. Moreover, tasks requiring reflection, evaluation, and accessing information exhibited higher ICCs than others. This study contributes to the design and analysis of ColPS assessments and illustrates tools to assess if tasks function as they were intended.

• Ræder, Henrik Galligani (2022). Å tilpasse eksamen for streikerammede elever trenger ikke å være så vanskelig. Utdanningsnytt.no.
• Mittal, Oleksandra; Scherer, Ronny & Nilsen, Trude (2022). Assessing the Evidence for Comparability of Socioeconomic Status across Students with and without Immigrant Background in Norway and Sweden [Paper presentation]. Nordic Educational Research Association (NERA) Conference, Reykjavik, Iceland.
• Steinmann, Isa & Olsen, Rolf Vegar (2022). Measuring equality of school effectiveness with multi-level regression discontinuity.
• Fladmark, Silje Fossum; Christensen, Marit; Nielsen, Karina Marietta; Innstrand, Siw Tone & Haakstad, Haakon Eidem (2022). The Footprint – A Study of the Importance of Distinctive Work Characteristics for Employees of Different Age, Gender and Position in Academia. .
• Christensen, Marit; Nielsen, Karina Marietta; Innstrand, Siw Tone & Haakstad, Haakon Eidem (2022). The Footprint – A Study of the Importance of Distinctive Work Characteristics for Employees of Different Age, Gender and Position in Academia.
• Blömeke, Sigrid; Howie, Sarah J.; Kaplan, David & Scherer, Ronny (2022). Introduction to the Comparative Large-Scale Studies in Education: Structure and Overview of the Handbook. In Nilsen, Trude; Stancel-Piątak, Agnes & Gustafsson, Jan-Eric (Ed.), International Handbook of Comparative Large-Scale Studies in Education. Springer. ISSN 978-3-030-38298-8. doi: 10.1007/978-3-030-38298-8_1-1.
• Steinmann, Isa; Sanchez Ruiz, Jesus Daniel; Van Laar, Saskia & Braeken, Johan (2022). How do inconsistent responders to mixed-worded questionnaire scales affect inferences in international large scale assessments? .
• Raabe, Isabel J. & Steinmann, Isa (2022). Why do Girls get Better Grades in School? .
• Strello, Andrés; Strietholt, Rolf; Steinmann, Isa & Siepmann, Charlotte (2022). Combining Two Decades of Evidence to Study the Effects of Tracking on Inequality and Segregation.
• Lehre, Anne- Catherine W G & Braeken, Johan (2023). Using TIMSS data to explore Nordic struggles with fractions. UiO.
• Gunnulfsen, Ann Elisabeth; Jensen, Ruth; Hall, Jeffrey Brooks; Abrahamsen, Hedvig Neerland; Kleveland, Heidi Merete & Olsen, Rolf Vegar (2022). Underveis med nytt læreplanverk: Ledelse, styring og støtteressurser. Det utdanningsvitenskapelige fakultet. ISSN 978-82-93847-09-0. Full text in Research Archive

View all works in Cristin