How to measure quality of delivery: Focus on teaching practices that help students to develop proximal outcomes

Diego Catalán Molina, Tenelle Porter, Catherine Oberle, Misha Haghighat, Afiya Fredericks, Kristen Budd, Sylvia Roberts, Lisa Blackwell, and Kali H. Trzesniewski

PDF Version

How much students benefit from a school intervention depends on how well the intervention is delivered

When a new curriculum is introduced at a school, the quality of its implementation will vary across teachers. Does this matter? In this study, teachers varied widely in how well they implemented a 20-lesson social and emotional blended-learning curriculum. Teachers who delivered the program at higher quality, for example, encouraged student reflection and participation and provided feedback to students on how to improve skills. Teachers who delivered the program at higher quality had students with higher levels of motivation (growth mindset, effort beliefs, and learning goals) at the end of the program compared to teachers who delivered at lower quality.


How to read this chart
: Dots represent the average difference in student outcomes between students exposed to high and low quality of delivery. The difference is shown as how much students would, on average, improve over the 50th percentile (low quality of delivery) on each outcome if they were exposed to high quality of delivery. Dotted lines represent our level of uncertainty around our findings.   

How can these results inform your research and practice?

Researchers, curriculum developers, and education leaders have a role in ensuring that students are exposed to high-quality programs. Researchers can create tools to capture and learn about the quality of delivery of school programs; curriculum developers can create professional development resources to ensure that teachers are well prepared and supported to deliver a new program well; and education leaders can leverage resources to create conditions for teachers to implement new programs well.

Why did we do this study?

Few studies measure the quality of delivery of a curriculum and document its influence on intervention effects. There is little agreement on how to define and measure “quality of delivery.” This study calls attention to the importance of measuring quality of delivery for understanding why and for whom interventions work and offers recommendations for defining and measuring quality of delivery in school interventions.

What did we do in this study?

In this study 25 teachers were observed delivering a social and emotional intervention, BrainologyTM, to 1,171 sixth and seventh grade students as part of the intervention in a randomized controlled trial. The evaluation tested the efficacy of the intervention on students’ motivation. Teachers were observed between 6 and 23 occasions, resulting in over 400 classroom observations. Students’ motivation was captured through surveys before (baseline) and after (endline) the program was delivered.


Full Article Citation:
Catalan Molina, D., Porter, T., Oberle, C., Haghighat, M., Fredericks, A., Budd, K., Roberts, S., Blackwell, L., & Trzesniewski, K. H. (2022). How to measure quality of delivery: Focus on teaching practices that help students to develop proximal outcomes. Journal of Research on Educational Effectiveness. DOI: https://doi.org/10.1080/19345747.2022.2054481.

Share this post:

Comments on "How to measure quality of delivery: Focus on teaching practices that help students to develop proximal outcomes"

Comments 0-5 of 0

Please login to comment