Developing school-level instruments for better understanding effective numeracy instruction at scale [CIES 2023 Presentation]

While there has been substantial investment in early-grade reading in low- and middle-income country contexts (LMICs) in the last 15 years, and a concomitant increase in evidence around what works to improve reading outcomes, there has been much more limited investment in early-grades mathematics. As a result, the body of evidence on what works to improve mathematics teaching and learning in LMICs is more limited. This study has identified six government- and program-led interventions in LMICs that have evidence of impact on students’ numeracy outcomes and are working at scale, to understand how and why they are effective and consolidate that evidence for the international education community. In order to examine the target programs, the study team has developed a suite of instruments designed to examine the programs and to identify common elements that these successful numeracy may have in common. The goal in designing these instruments was to be able to examine a range of potential factors, based on the evidence that we have on mathematics teaching and learning from research in high-income country contexts, as well as the limited research evidence we have from LMICs. This suite of instruments includes: (1) a quantitative classroom observation instrument, based on multiple frameworks for high-quality math instruction, including work by The Danielson Group (2019), The University of Michigan’s High Leverage Teaching practices, and a cross-institutional working group of math education experts working in LMICs (co-author, 2019); (2) a student cognitive interview instrument intended to provide insight into students’ development of higher order, conceptual understanding of basic mathematics concepts; (3) a qualitative classroom observation instrument and accompanying lesson-based teacher interview; (4) a survey of teachers’ Mathematical Knowledge for Teaching, based on work by Deborah Ball (2011). Focusing primarily on the quantitative classroom observation and student cognitive interview instruments, this paper will present the theoretical foundations of the instrument and the processes for developing, piloting, and adapting the instruments for different country and program contexts. Preliminary findings and lessons learned from utilizing the tools for data collection across country contexts will also be shared. Given the need to expand the body of evidence around what works to improve math teaching and learning, these instruments represent potentially valuable resources for research in this area – and the authors look forward to discussing the potential for use and further development/adaptation.