While there has been substantial investment in early-grade reading in low- and middle-income country contexts (LMICs) in the last 15 years, and a concomitant increase in evidence around what works to improve reading outcomes, there has been much more limited investment in early-grades mathematics. As a result, the body of evidence on what works to improve mathematics teaching and learning in LMICs is more limited. This study has identified six government- and program-led interventions in LMICs that have evidence of impact on students’ numeracy outcomes and are working at scale, to understand how and why they are effective and consolidate that evidence for the international education community. In order to examine the target programs, the study team has developed a suite of instruments designed to examine the programs and to identify common elements that these successful numeracy may have in common. The goal in designing these instruments was to be able to examine a range of potential factors, based on the evidence that we have on mathematics teaching and learning from research in high-income country contexts, as well as the limited research evidence we have from LMICs. This suite of instruments includes: (1) a quantitative classroom observation instrument, based on multiple frameworks for high-quality math instruction, including work by The Danielson Group (2019), The University of Michigan’s High Leverage Teaching practices, and a cross-institutional working group of math education experts working in LMICs (co-author, 2019); (2) a student cognitive interview instrument intended to provide insight into students’ development of higher order, conceptual understanding of basic mathematics concepts; (3) a qualitative classroom observation instrument and accompanying lesson-based teacher interview; (4) a survey of teachers’ Mathematical Knowledge for Teaching, based on work by Deborah Ball (2011). Focusing primarily on the quantitative classroom observation and student cognitive interview instruments, this paper will present the theoretical foundations of the instrument and the processes for developing, piloting, and adapting the instruments for different country and program contexts. Preliminary findings and lessons learned from utilizing the tools for data collection across country contexts will also be shared. Given the need to expand the body of evidence around what works to improve math teaching and learning, these instruments represent potentially valuable resources for research in this area – and the authors look forward to discussing the potential for use and further development/adaptation.

To support the Ministry of Public Education (MoPE) in achieving its reform agenda, USAID initiated the Uzbekistan Education for Excellence Program (UEEP) to improve the quality of education and enable all students to be proficient in 21st century skills such as problem solving and critical thinking. UEEP is implemented by a Consortium of partners including RTI International, Florida State University, and Mississippi State University. The Program aims to achieve three overarching results: improved Uzbek language (UL) reading and mathematics outcomes in grades 1–4; enhanced information and communication technology (ICT) instruction for grades 1–11, and improved English as a foreign language (EFL) instruction in grades 1–11. In this presentation, we will describe the process of revising the student standards for Grades 1-4 in mathematics, led by Florida State University and UEEP math experts, with ongoing review and feedback from MoPE representatives. The revision of student standards was the first step of the country’s curricular reform, followed by the creation of TLMs and a corresponding teacher professional development package. This process consisted of several stages, beginning with comparing current MoPE standards with the GPF, student math standards from South Korea and the TIMSS framework for assessment. These three resources were chosen for specific purposes. The TIMSS framework reflected the Government of Uzbekistan’s priority to prepare children in Grade 4 to take TIMSS assessment and perform well. This framework only reflected Grade 4 learning, so it was used primarily to backwards map other standards in Grades 1-3 to ensure that all content on the TIMSS framework was adequately covered by the end of Grade 4. The GPF was the most detailed reference, and was used to map the current standards and identify gaps in the progression. However, there was concern from the Government of Uzbekistan that the GPF might be targeting skills that are too easy for children in Uzbekistan. Because of this, we also brought in the South Korea standards, which provided us with a reference from a country that is seen as a model in Uzbekistan, and we were able to use the South Korea standards to make decisions about certain skills. This process allowed us to ensure that the standards development process met national priorities (i.e. Presidential Decree on Improving Math Education, 2020), reflected best international practices in mathematics, created standards that were age appropriate and measurable, and was logically organized. We will discuss how we used a quantitative comparative analysis of Uzbekistan's existing standards for different grade levels with the GPF and South Korea standards to identify revisions needed to facilitate more alignment. The rigor of that exercise allowed us to revise and to create a set of standards that could be approved by MoPE. The rigorous student standards development process described in this presentation allowed us to create a set of standards that were approved by MoPE. These standards were the cornerstone of further reforms including creating a scope and sequence and Teacher/Learning Materials (TLMs) for Grades 1-4, currently being piloted with 10,000 teachers across Uzbekistan.

The Mathematics Knowledge for Teaching (MKT) is a short survey (23 items) that measures primary grade teacher knowledge by a) math domains and b) pedagogical and content knowledge. Math domain included Number Sense, Operations, Geometry, and Measurement. Pedagogical knowledge was measured by problems that measured teacher understanding of Developmental Progression, Scaffolding, and Content knowledge. In this presentation, we will discuss the process of developing the MKT survey, highlight exemplary results from the Kyrgyz Republic, and then discuss the various uses of this survey. The MKT survey build from previous work in measurement of teachers’ MKT in the United States and other countries (Ball et.al., 2008; Cole, 2012). Our goal was to create an instrument that focused on the early primary grade and was easily adaptable to multiple contexts. To do this, we created an initial instrument, conducted cognitive interviews with math and learning experts form several countries, and then conducted a pilot in the Kyrgyz Republic and Nepal. In Kyrgyz Republic, the MKT test was administered to 323 primary grade teachers in 30 pilot schools as a pre-post training survey as part of the USAID Okuu Keremet! The survey was administered online in two languages. Analysis of pre-post test showed that the survey was effective in detecting changes in teacher knowledge across all math domains and pedagogical and content knowledge areas. In Nepal, we conducted cognitive interviews with teachers, providing additional insights into how teachers were thinking about early math knowledge. Finally, we conclude with the different potential uses for this survey, such as diagnosing and measuring changes in teacher knowledge over time and using it as professional development tool to develop teacher knowledge. We will discuss implications for the use of this tool for the wider development audience.

The purpose of this panel presentation is to present the results of two uptake studies to understand how mathematics, Uzbek language arts, ICT, and EFL teachers in Uzbekistan are using and applying newly developed teaching and learning materials in the classroom.

The purpose of this paper is to present a methodology for understanding materials usage in primary classrooms in Sub-Saharan Africa that centers teachers’ actions and voices. The United Nation’s Sustainable Development Goal 4 focuses on improved primary education around the world. To meet this goal, many large, donor-funded interventions aim to improve education through provision of teacher’s guides and student textbooks. However, what many of these interventions lack is a systematic way to understand how and why teachers are making pedagogical decisions while using materials. There is a large body of work that seeks to understand how teachers make decisions as they teach, and the ways these decisions are influenced by their knowledge and beliefs. Drawing from this work, we describe a methodology and set of tools that uses observations and interviews to identify key decisions that teachers make in the classroom and why teachers made those decisions. We piloted and iteratively refined this methodology over the course of three studies and use examples from these studies to illustrate the methodology. By closely observing and listening to teachers, we gain insights that allow us to continually refine and improve materials to ultimately improve the quality of classroom instruction.

This report summarizes the findings of an effort to develop and validate tablet-based, self-administered assessments of English-language foundational literacy and numeracy in the early grades. The tools described in the report were developed at the request of Imagine Worldwide with the support of the Jacobs Foundation. RTI carried out field testing and a pilot study to assess the tools' internal consistency, test-retest reliability, and concurrent validity with respect to "traditional" EGRA and EGMA. RTI International developed the two assessments, known respectively as the Self-Administered Early Grade Reading Assessment (SA-EGRA) and the Self-Administered Early Grade Mathematics Assessment (SA-EGMA), with the support and at the direction of Imagine Worldwide. The assessments are deemed “self-administered,” because children complete the assessments independently in response to instructions and stimuli imbedded in the tablet-based software. However, adults typically supervise the organization and conduct of the assessment as well as the collection of individual data from the tablets for analysis. The tools have been developed under an open-source license. The code can be viewed and downloaded for reuse or modification at https://github.com/ICTatRTI/SE-tools/blob/main/README.md. Users of RTI's Tangerine software may request that the SA-EGRA and SA-EGMA tools be added to their Tangerine groups via https://www.tangerinecentral.org/contact

The purpose of this session is to articulate a research framework that centers teachers’ voices when trying to understand how teachers use curriculum materials in the classroom. Operating in the context of highly structured lesson plans, the approach identifies ways in which teachers exercise their professional discretion to modify the lesson and frames conversations to elaborate the motivations driving the teachers’ choices. The approach has been iteratively refined across three studies; taken together, the studies provide evidence for the value of listening to teachers and being responsive to their voices during implementation. The research framework uses the lens of modifications, or changes to the intended lesson plan implemented within one class period. Modifications can be large or small, additive or subtractive. For example, in a lesson with a section for independent student practice, a large modification might be skipping the practice section entirely. Or, during a lesson focused on blending of initial sounds using 3 example words, a small modification might be extending the exercise by adding extra words. Researchers observe the lesson, noting any modifications; after the lesson, the researchers select some of the modifications and ask the teachers why they made the choices they did. Analyses of teachers’ explanations highlight the importance of understanding why teachers make the choices they do. For example, a teacher who skipped the independent practice section because they don’t think their students are ready to do the skill on their own suggests that the teacher is exercising agency and using her knowledge of her students to inform her decision-making. Insights such as this one can guide decisions on projects. It may be that while the intention of the teacher was guided by knowledge of students, the end result is not desirable from the project’s point of view. Understanding why the teacher made this choice provides implementers with better and targeted ways to address choices that impact the overall goals of the project. In this presentation, we draw on data from an exploratory case study to understand use of new mathematics materials in Liberia, a more in-depth case conducted in Malawi on teacher use of reading materials, and finally, a systematic study examining how reading teachers use materials across four Sub-Saharan African countries. We use each case to highlight both an aspect of the research framework and instances of modifications to project implementation driven by teachers’ voices. By focusing on teacher voices, we disrupt the deficit notion that teachers are “resistant” to change, or do not “understand” new pedagogies. Instead, we aim to value teacher voices and integrate their insight into implementation programs. By doing this, we not only raise the likelihood of successful use of new materials and pedagogies, but we also develop more responsive pedagogy that better matches existing classroom cultures.

This brief describes the use of qualitative analysis in international education research.

Literacy instruction programs have arguably had limited success because they focus on the technical – but not the social – aspects of literacy instruction. Reform efforts in sub-Saharan Africa have regularly failed to shift pedagogy away from teacher-led whole-class direct instruction to activities that are more effective for learning. In part, the failure is due to a lack of recognition of the social nature of classrooms where teacher-child interactions are conditioned by cultural predispositions. New research from Tanzania identified such challenges to pedagogical reform and points to potential solutions. One approach focuses on the child - to develop their social and emotional competencies. Teachers in Mtwara, Tanzania - but not parents – think that confidence and curiosity are important for student learning and report that interactive teaching activities are less effective in rural areas where students lack these competencies. Evidence suggests that building students’ confidence to participate in class is achievable relatively quickly. A second approach is to adapt teaching activities. Teachers in Tanzania report reluctance to implement teaching activities that undermine the social goals of instruction, such as avoiding embarrassment and promoting a sense of fairness and togetherness in the classroom. Instruction would be more effective if activities are co-designed with teachers to achieve both the social goals and the cognitive/learning goals of teaching.

This document is intended for program and curriculum experts interested in implementing evidence-based early grade mathematics programs. It was developed by the authors of this document, who are mathematics teaching and learning experts with extensive experience adapting evidence-based practices in low and middle-income contexts. Our collective field and research experience, combined with the existing evidence base, led us to focus on four instructional strategies that are key to effective mathematics instruction: 1. Respecting developmental progressions 2. Using mathematical models to represent abstract notions 3. Encouraging children to explain and justify their thinking 4. Making explicit connections for children between formal and informal math While these four instructional strategies are very important, they are not the only instructional strategies that can result in improved learning outcomes. Effective early grade mathematics teachers draw from an extensive repertoire of evidence-based instructional strategies and strive to create a learning environment the supports that development of positive mathematical identities.