Learning at Scale Interim Report

The Learning at Scale study was designed to identify existing early grade reading programs with demonstrated impact on basic skills at scale and to conduct in-depth investigations of these programs to determine what makes them successful. After an extensive search, eight programs (spanning seven countries) were selected for inclusion in the study. Research on these programs has been conducted in order to answer the three overarching research questions, focused on understanding the components of instructional practices, instructional supports, and system supports that lead to effective instruction. Learning at Scale data collection activities for some of these programs were delayed due to COVID-19. However, with demand for information about how to implement effective interventions at large scale at an all-time high, we believe that the timely sharing of findings from Learning at Scale is essential. Accordingly, this interim report provides preliminary findings from our study to date, highlighting key high-level findings across all eight programs, as well as quantitative and qualitative findings from primary research for select programs. The Learning at Scale study is led by RTI International, as part of the Center for Global Development (CGD) education research consortium, funded by the Bill and Melinda Gates Foundation.

Improving Education for Deaf Students in Kenya

In Kenya, USAID supports the training of deaf education teachers like Martha Mwangu to improve learning outcomes of children with disabilities in grade 1, 2, and 3 through the Tusome Literacy Program. Tusome ('Let's read' in Kiswahili) is implemented in all public primary schools.

Uganda/LARA: EGR Action Research Tools

LARA developed action research tools to measure the level of fidelity of implementation of the EGR methods and generate lessons learned to inform adaptations in EGR programming. The action research tools gather both historical and real-time data at the school. They include the EGR core methodologies action research tool; the remedial instruction action research tool and the intensive coaching action research tool (subdivided into two tools i.e. head teacher coaching event log and school based community of practice event log). The EGR core methodologies action research tool assesses the teacher’s perception of the Teacher Guide usability, level of macro pacing, implementation of lesson plan elements, implementation of core EGR methodologies and tracking of instructional adaptations by the teacher. The remedial instruction research tool tracks the teacher’s perception to remedial instruction, the implementation of group-based instruction as well as in-class assessment. The intensive coaching action research tool tracks teacher’s perception of intensive coaching and keeps a log of head teacher instructional coaching events in addition to school-based community of practice activities. The action research tools are designed to be deployed electronically in order to seamlessly incorporate extra data quality standards and innovations like the Stalling’s classroom observation snapshot (Stallings and Kaskowitz, 1974 ). The project also developed the action research process flow guidelines to guide data collection activities.

Do’s and Don’ts of Improving Teaching Through Instructional Support: Findings from a Multi-country study of coaching and communities of practice

RTI’s multi-country study examines a range of instructional support packages that include a variety of designs for coaching and communities of practice across all of RTI’s existing programs in order to determine what modes of coaching support are most effective. This includes coaches at both the school level and external to the school, as well as government officers compared with project hired coaches. This research is able to compare the relative effectiveness of coaching approaches with educational support provided by communities of practice meetings held at the school or the school cluster level. These findings have provided existing and future programs with best practices for how to more effectively implement coaching support structures at scale.This presentation was delivered by RTI at CIES (April 2020) as part of a panel focused on how literacy improvement efforts work at scale, and what types of coaching designs and behaviors impact teacher pedagogy more effectively than others.

Is It Possible to Improve Learning at Scale? Reflections on the Process of Identifying Large-Scale Successful Education Interventions

Improving learning outcomes at scale is hard. That may seem obvious, but only recently have policymakers and donors become aware of just how dire—and broad—the learning crisis is. Most of their efforts to improve learning have been pilot programs, and although in some cases it has been possible to improve outcomes at this small scale, it is an entirely different challenge at scale, which can involve thousands of schools—the level at which change must happen to fix the crisis.

Guiding Teachers Rather than Scripting Them

It is difficult to imagine an effective teacher who does not have mastery of the content nor command of the pedagogical skills needed to teach literacy. Yet many teachers, particularly in low- and middle-income countries, lack these very skills. In those same countries, students exit school without the essential capabilities they need. How can we help teachers to teach more effectively?

Mathematics from the Beginning: Evaluating the Tayari Preprimary Program’s Impact on Early Mathematics Skills

Given the dearth of research on early numeracy interventions in low- and middle-income countries, this paper presents the instructional methodology and impact results of the Tayari program. Tayari is a preprimary intervention in Kenya (2014–2019) that prepares children aged four and five for entry into primary school by providing materials for students, training for teachers, and continuous in-classroom support. The Tayari methodology was built on the Kenyan government’s preprimary syllabus to produce instruction that was developmentally sequenced, linked to out-of-school experiences, and supportive of children’s number sense. Tayari was evaluated using a randomized controlled trial (RCT) and collection of longitudinal data from 2,957 children in treatment and control schools at three time points. Pupil assessment items were drawn from a growing body of research on preprimary numeracy in developing contexts, plus instruments and techniques from the Measuring Early Learning and Quality Outcomes (MELQO) program (UNESCO, UNICEF, Brookings Institution, & World Bank Group, 2017). The impact evaluation of the longitudinal RCT results showed statistically significant effects in the numeracy tasks of producing sets, identifying numbers, and naming shapes, while revealing no initial effects in the areas of oral and mental addition. We present recommendations for Tayari’s improvement in terms of mathematics instruction, as well as preprimary policy implications for Kenya and similar contexts.

Measuring Executive Function Skills in Young Children in Kenya

Interest inmeasuring executive function skills in young children in lowand middle-income country contexts has been stymied by the lack of assessments that are both easy to deploy and scalable. This study reports on an initial effort to develop a tablet-based battery of executive function tasks, which were designed and extensively studied in the United States, for use in Kenya. Participants were 193 children, aged 3–6 years old, who attended early childhood development and education centers. The rates of individual task completion were high (65–100%), and 85% of children completed three or more tasks. Assessors indicated that 90% of all task administrations were of acceptable quality. An executive function composite score was approximately normally distributed, despite higher-than-expected floor and ceiling effects on inhibitory control tasks. Children’s simple reaction time (β = –0.20, p = .004), attention-related behaviors during testing (β = 0.24, p = .0005), and age (β = –0.24, p = .0009) were all uniquely related to performance on the executive function composite. Results are discussed as they inform efforts to develop valid and reliable measures of executive function skills among young children in developing country contexts.

Effectiveness of Teachers’ Guides in the Global South: Scripting, Learning Outcomes, and Classroom Utilization

This report presents the results of RTI International Education’s study on teachers' guides across 13 countries and 19 projects. Using quantitative and qualitative methods, we examine how teachers’ guides across the projects differ and find substantial variation in the design and structure of the documents. We develop a scripting index so that the scripting levels of the guides can be compared across projects. The impact results of the programs that use teachers’ guides show significant impacts on learning outcomes, associated with approximately an additional half year of learning, showing that structured teachers’ guides contribute to improved learning outcomes. During observations, we find that teachers make a variety of changes in their classroom instruction from how the guides are written, showing that the utilization of structured teachers’ guides do not create robotic teachers unable to use their own professional skills to teach children. Unfortunately, many changes that teachers make reduce the amount of group work and interactivity that was described in the guides, suggesting that programs should encourage teachers to more heavily utilize the instructional routines designed in the guide. The report includes a set of research-based guidelines that material developers can use to develop teachers’ guides that will support effective instructional practices and help improve learning outcomes. The key takeaway from the report is that structured teachers' guides improve learning outcomes, but that overly scripted teachers' guides are somewhat less effective than simplified teachers' guides that give specific guidance to the teacher but are not written word for word for each lesson in the guide.

Persistence of pre-primary programs in Kenya: Evaluating how Tayari's program impacts change over time and into primary school- CIES 2018 Presentation

CIES 2018 Presentation, given by Ben Piper. While there has been recent evidence pointing to the efficacy and cost-effectiveness of pre-primary interventions, there remains little known about how those interventions might work at medium or national scale, nor whether the theoretical impacts of the programs increase, decrease or remain steady over time, nor whether the gains persist into primary school. The endline results from two years of Tayari’s intervention provide longitudinal data on nearly 2500 children who have been evaluated across a range of early learning skills (literacy, numeracy, socio-emotional) at three data points, including their first year of Grade 1. The results presented will explain how Tayari program impacts change over these three data points and determine whether Tayari’s impact persists into Grade 1.