By Jonathan Stern, Wendi Ralaingita, Julianne Norman, and Yasmin Sitabkhan
Earlier this summer we provided initial insights into what made 
Numeracy at Scale’s six large-scale early grade math programs successful. We are excited to report that our analyses are now complete and that this post provides a sneak peek into the study results! The findings and recommendations are also summarized in our findings brief, covered in painstaking detail in our Final Report (here), and were discussed during an online event on October 24 (recording here).
A Brief Refresher
There are three research questions that have guided both phases of the Learning at Scale study:
- What classroom ingredients (such as teaching practices and classroom environment) lead to learning in programs that are effective at scale?
- What methods of training and support lead to teachers adopting effective classroom practices?
- What system-level support is required to deliver effective training and support to teachers and to promote effective classroom practices?
During this second phase, dubbed Numeracy at Scale, we identified six programs in low- and middle-income countries with evidence of improved math outcomes at scale (see figure at the end of this blog for a list of the six programs or read the program fact sheet for more details). Despite the varied approaches in program designs (e.g., nationwide program in every classroom, vs. provision of after-school remediation, vs. use of tablet-based instruction, etc.), we found that the programs tended to share a large number of common elements, as shown in the following figure.
These elements should be at the heart of any discussion around early grade math program design. It’s important to keep in mind, however, that while they provide a solid menu of factors to consider, their application can differ depending on the program. In other words, the selected successful programs show us that there are multiple pathways for achieving strong numeracy results at scale.
What are teachers in successful programs actually doing in the classroom?
Four themes emerged from this first research question, highlighting the variety of approaches used to increase student engagement, adapt instruction to be more responsive to students’ varying needs, and ultimately improve learning.
Theme 1: Teachers use multiple representations and models to support learning.
Teachers in all programs used multiple approaches and representations to scaffold the learning of math content for their students. Often, teachers used different representations of the same concept to help students understand mathematical concepts and draw linkages between them.
For instance, as shown in the example to the right, a teacher in the TAFITA program in Madagascar used concrete materials to introduce the concept of addition, then linked this to a pictorial representation, and finally linked it to abstract symbols. Importantly, we found that in all of the programs, students had the opportunity to use the concrete materials themselves during independent and group work, rather than simply observing the teacher doing so.
Similarly, interviews with students from the RAMP program in Jordan revealed that when solving simple addition and subtraction problems, students were successful using manipulatives, drawings, and representations.
Theme 2: Instructional approaches include a specific focus on both conceptual understanding and procedural fluency.
All programs sought to promote students’ conceptual understanding of material rather than focusing on memorizing algorithms. Programs varied in the way in which they did this, with common strategies including using open-ended questions, helping students find the right answer when they are incorrect, encouraging the use of multiple strategies for solving problems, and having children share and discuss their mathematical ideas. In GKA in India, teachers made frequent connections between math concepts and children’s real-world experiences, which supports the development of conceptual understanding. For example, teachers asked children to draw shapes and then discuss where they might see those shapes in the real world. This also made learning more meaningful and engaging…leading us to Theme 3.
Theme 3: Various approaches are used to ensure active student engagement throughout lessons.
All Numeracy at Scale programs included a focus on active student engagement and participation during math lessons. Whether it was teachers interacting with students through various questioning techniques in RAMP, students engaging in math games in TAFITA, tablet-based instruction in Nanhi Kali, or independent student work under ESMATE, teachers used a diversity of strategies to keep students engaged and learning.
Importantly, all programs ensured that a substantial amount of independent and/or group work time was spent on problem solving and active learning (Nanhi Kali achieved this through tablets, while the other five programs drew on multiple strategies, including the use of manipulatives, playing math games, measuring or cutting out and manipulating shapes, etc., as shown to the left).
Theme 4: Teachers use assessment-informed instruction approaches to address differentiated needs.
Assessment was used to inform instruction across Numeracy at Scale programs. This meant that teachers were aware of their students’ varying learning levels, monitored their students’ progress, and adjusted their instruction to adapt to students’ needs and abilities. Although assessment-informed instruction was present in most programs, it played out differently in each. For example, TAFITA’s after-school remediation program used periodic formal assessments to group and teach students according to their learning level, whereas Nanhi Kali used the Mindspark app to instruct students based on assessment results. In this app, students’ performance on a question influences the next question posed, providing guided practice at the student’s individual level. In all programs, teachers monitored students during independent and group work and helped those who were struggling.
How can teachers be supported to adopt new classroom practices?
Teachers in all programs reported on the importance of strong support mechanisms (e.g., training, coaching, teacher meetings, materials), but there was variation in terms of which types of support teachers found most effective. For example, teachers from the GKA and Nanhi Kali programs felt that the program materials were most essential, while teachers from R-Maths, RAMP, TAFITA, and ESMATE all found training and teacher meetings to be the most useful. Coaching emerged as a valuable support mechanism for the half of the programs (see below).
Theme 1: Teacher supports focus explicitly on math content and improving instruction.
While the design of teacher support models varied greatly across programs, all prioritized supporting teachers in teaching math more effectively. This was particularly true for teacher trainings and coaching support visits. In lieu of a historically prevalent focus on classroom management and general classroom practices, trainings focused specifically on introducing teachers to new math content while providing them with a variety of methods for improving students’ conceptual understanding of mathematics. Teachers in four of the six programs reported that math instruction and lesson plan development were the most useful training content that they received.
Theme 2: Trainings emphasize modeling and practice over lecturing, providing teachers with opportunities to practice and discuss.
Teachers across all six programs reported that trainings incorporated more modeling, demonstration, and hands-on practice than lecturing. More than half of the teachers in GKA, RAMP, R-Maths, and TAFITA cited modeling and demonstration as the most useful training methods. RAMP teachers focused on instructional practice during their teacher meetings, and teachers reported that their discussion with other teachers during these meetings was the most helpful aspect. RAMP teachers also reported that trainings did a particularly good job at modeling and demonstration and allowed the teachers to learn new math content and practice new skills. Other program trainings employed a similar approach based on demonstration and teacher practice, with teachers demonstrating new instructional strategies to one another.
Theme 3: Teacher and student materials provide explicit guidance for instruction.
Teachers in all programs received teacher materials and tended to report that these materials were better organized, easier to follow, and included better activities and examples than previous materials (see figure to right). Teachers in ESMATE and R-Maths emphasized the usefulness of step-by-step instructions, while GKA teachers spoke to the importance of material alignment with the curriculum. Teacher’s guides in several programs included instructions for teachers that provided guidance on how to model concepts, how to lead practice sessions with students, and how to reinforce skills. ESMATE’s teacher guides provided detailed instructions on the problem-solving approach woven throughout each lesson, while the student workbook mirrored this approach.
Theme 4: Ongoing support emphasizes feedback, problem solving, and learning new content over inspection and evaluation.
While there was variation in the ongoing supports that were provided to teachers across programs (e.g., teacher meetings, coaching, mentoring, etc.), there was consistency in the fact that teachers had various opportunities to receive feedback, solve problems, and learn new content. Further, large proportions of teachers reported that the individuals who provided professional development were more supportive and friendlier than those from previous programs. Lastly, in Madagascar, coaches in TAFITA schools were less likely than those in non-TAFITA schools to categorize their role as an evaluator or inspector. This finding was similar across other programs.
How can programs successfully impact teaching and learning at scale?
We hypothesized at the outset of this study that one component of these programs’ success would be that key system actors are informed about the program, that the respective education systems communicate expectations for districts, schools, teachers, and students, and that system actors play substantive roles in implementation. As it turns out, we were right.
Theme 1: Programs actively collaborate with key stakeholders.
In Numeracy at Scale programs, stakeholder collaboration occurred at all levels of the system, both formally and informally. For example, the Western Cape Education Department in South Africa used existing policy statements to solicit and coordinate input from other stakeholders for the R-Maths program. In this way, ideas from outside of the department were considered and included, but shaped to fit within the government framework. Each country brief from the Numeracy at Scale study includes specific examples of how multiple levels of the education system worked together to promote uptake and scale-up of the program.
Theme 2: Investments are made in resources to improve the quality of classroom instruction.
While the six programs had varied funding sources, they shared one aspect in common: programs prioritized investment in resources to bolster the quality of instruction. Every program in this study invested heavily in developing the capacity of teachers and other education professionals, including “middle managers” (who coach and supervise teachers). Rather than improving support for one group of system actors, the programs ensured that the entire system received support. Additionally, while teaching and learning materials constitute common investments for education programs, the programs studied here made these investments with an important consideration in mind: they ensured that professional development was closely aligned with the goals of improved instruction and learning outcomes and that the teaching and learning materials were evidence based, of high quality, and given to students and teachers in adequate quantities.
Theme 3: Programs emphasize continuous monitoring and use of data for system improvement.
We found that the six programs tended to focus their monitoring efforts on two areas: students’ learning outcomes and teachers’ pedagogical practices. Importantly, this was done through government systems when possible. For example, decentralized government officials—such as subject advisors in South Africa, District Institute for Education and Training officials in India, and senior teachers in Jordan—monitored teachers’ practices. Additionally, GKA and RAMP used an electronic system for classroom observation, with the data readily available via online dashboards.
Theme 4: Programs focus on systematically embedding and institutionalizing best practices, with an eye toward sustainability.
Finally, whether directly implementing through government systems or engaging in policy advocacy, programs focused on institutionalization as a strategy for sustainability and scale-up. All programs were successful in enacting policy changes or integrating best practices into the government system. This approach was aptly summarized by a high-level official in Jordan:
“CPD [continuous professional development] policies have changed in recent years, and all of RAMP’s best practices are included in these policies. All of RAMP’s procedures are part of the official CPD programs now—tied to teacher promotion. All of these programs are based on teaching standards developed by RAMP.”
ESMATE and R-Maths took institutionalization a step further by officially transitioning from implementer-led programs to system-embedded government initiatives.
So, where do we go from here?
Perhaps the clearest takeaway from this work is that there is no single prescribed way to improve math instruction. Instead, it appears that there are several pathways that lead to success. These include asking students higher-order questions, making real-life connections, using concrete materials, and providing clear and accurate explanations of content. Additionally, it is valuable to involve students in modeling and explanation, and it is essential to provide students with time to practice (with a variety of approaches, including the use of concrete materials). No one program necessarily did all of these things as expected, however. This shows that it is more about selecting a number of key elements and doing them well, as opposed to trying to incorporate everything into a single program.
Further, ensuring that classrooms have high-quality teaching and learning materials will make teachers’ lives easier, but it is equally important to provide teachers with sufficient time to practice their new skills and approaches outside of the classroom. There should also be a focus on building teachers’ mathematical knowledge for teaching, which can be reinforced by continuous and complementary teacher supports (such as training, coaching, teacher meetings, etc.).
At the system level, it is essential to use data and evidence to inform decision-making and to make programmatic adjustments as needed. Additionally, designing for scale and implementing through government systems (with an eye toward institutionalization and sustainability) are necessary to achieve improved learning outcomes at scale. But this task can’t just be left up to external programs—governments must strategically invest in resources at various levels of the education system in order to sustainably improve instruction and learning outcomes.
This study is the first of its kind, examining effective, large-scale numeracy programs in low- and middle-income countries. While we have learned a lot over the past few years, there is still much to explore. We’ve identified a range useful numeracy program elements but don’t yet have evidence on how to prioritize or select among them. We support the hypothesis that mathematical knowledge for teaching is important but don’t have strong evidence for how best to improve it for teachers with limited math experience in relatively low-resource settings. This study also only scratched the surface of the role that communities can play in supporting continuous math learning for children.
Ultimately, we’re excited to see this level of attention being paid to early grade mathematics and hope that this study is just the beginning.
Special thanks to Bidemi Carrol, Matthew Jukes, Rachel Jordan, Kellie Betts, Peggy Dubeck, Joe DeStefano, and Jessica Mejia for their technical contributions to the Numeracy at Scale study
