Improving learning outcomes at scale is hard. That may seem obvious, but only recently have policymakers and donors become aware of just how dire—and broad—the learning crisis is. Most of their efforts to improve learning have been pilot programs, and although in some cases it has been possible to improve outcomes at this small scale, it is an entirely different challenge at scale, which can involve thousands of schools—the level at which change must happen to fix the crisis.

Programs to promote social and emotional learning (SEL) risk making assumptions about the global relevance of core competences. Because scholarship is lacking about SEL in many parts of the world, new approaches are needed to contextualize the goals of SEL programs in a realistic time frame. Previous work in anthropology and developmental psychology can help us predict which competences are likely to be valued, given the sociodemographic characteristics of a society. In rural environments where subsistence agriculture is common, for example, communities are likely to value social responsibility, respect, and obedience. Attention should look beyond the needs of the here and now, however, to speculate what competences today’s children will require in the future. Looking at the current variation of competences within a society—for example, the values that teachers, but not parents, place on confidence and curiosity—can help identify immediate pathways for developing new competences. In all of these considerations, the goals of an SEL program must be negotiated with the communities themselves in order to ensure relevance, effectiveness, and acceptance. The hope is that such considerations can help prevent global homogenization of SEL programs, instead ensuring that they genuinely meet the needs of the communities they aim to serve.

Several forces are at play in determining whether pedagogical approaches are optimally adapted to the culture of children’s behaviour and of teacher-child interaction in the classroom. Teachers’ expectations for children’s behaviour may differ from the way in which children are raised at home (Jukes et al, 2018). Teaching activities may be designed by experts from outside the beneficiary education system or who may be removed from the culture of rural schools. The evidence for the effectiveness of recommended teaching practices may be based on children and teachers from a different culture. The current study sought to understand the role of cultural factors in the teaching of early grade reading in Tanzania. The aim was to investigate teacher’s pedagogical choices and the implicit theory of teaching and learning that underpinned these choices. We also aimed to understand teachers’ perceptions of students’ social and emotional skills and how this influenced their pedagogical choices. The current study took place in three regions of Tanzania – Zanzibar, Mtwara and Iringa – in the context of the USAID supported Tusome Pamoja (“let’s read together”) project. Researchers observed one lesson from each of 36 teachers and recorded key teaching activities. A subsequent qualitative interview with the teachers examined the decisions they made during the lesson and how their perception of students’ competencies influenced their decisions. Two themes emerged from the results. First, teachers said that children who did not participate in classroom activities lacked confidence and curiosity, particularly in rural areas. These two qualities - confidence and curiosity - were identified in previous work as traits valued more by teachers than parents in the context of learning. Second, teachers said the independent activities were not effective because pupils always cooperate with others in life. There was a strong preference for activities that involved the whole class and against students learning by doing, independent of the teacher. Schools can respond by conducting more activities that build curiosity and confidence. Children becoming more confident and curious in rural Africa represents a cultural change. But one that emerges from the community - not imposed upon it. Instructional design should consider the strong cultural preference for group-orientation and work with - rather than against - this preference where possible.

There is growing demand for policy based on rigorous evidence. Many consider the strongest evidence to come from studies that identify causality with high internal validity - such as RCTs - and systematic reviews of these studies. If policies are based strictly on such rigorous evidence there is a risk of bias towards simple, discrete, measurable interventions and away from complex interventions. Rigorous evidence is also better suited to some questions than others. Evaluations may provide stronger conclusions about impact than about the mechanisms, implementation, context, generalisability and scaling of interventions. For these reasons, policy-making does – and should – consider issues for which there is no conclusive evidence. However, there is little guidance as to how and when such inconclusive evidence can be used. We present a framework for considering inconclusive evidence applied to examples from evidence-based education in low- and middle-income countries. The framework involves a systematic consideration of the estimated costs, benefits and potential harm of a policy, along with the uncertainty in those estimates. This analysis is conducted using standard decision theory and an examination of the utility of policies. We argue that it is rational to pursue a policy with uncertain outcomes if there is a reasonable probability of large positive utility (compared to the cost of the intervention) and a low probability of negative utility. The decision to act under uncertainty is influenced by a number of other considerations including: the potential to improve the evidence base, the urgency of the decision and the analysis of alternative options. The framework also calls for systematic analysis of uncertainty associated with all components of a policy decision. For example, some interventions may have robust evidence of impact but considerable uncertainty associated with the generalisability of the evidence to a new context, or with the scalability of the intervention. We discuss our approach to measuring and reducing uncertainty in policy decisions and its implications for evaluation and research. The overall aim of this work is to make evidence-based decision-making more effective and applicable to a wider range of problems.

The Ministry of Education and Higher Education (MOEHE) conducted the first EGRA assessment in the West Bank in March 2014 among a nationally representative sample of Grade 2 students, followed by a benchmarking exercise in September 2014. In 2018, the Early Grade Reading (EGR) project conducted a project baseline using an adapted EGRA 1 and MELQO 2 3 . Following this assessment, the MOEHE expressed interest in revising the 2014 provisional Grade 2 benchmarks and developing Grade 1 benchmarks. EGR conducted a technical benchmarking workshop in November 2018.

CIES 2018 Presentation, given by Matthew Jukes. There is an increased demand for assessments of social and emotional competencies of young children in low- and middle-income countries. These competencies are increasingly seen as important for children’s development and for their education. In the context of preschool and primary education, such assessments have a number of uses. They are used to evaluate the impact of programs on children’s social and emotional learning. They can also be used to monitoring individual children’s progress in such programs and to tailor interventions to their needs. We developed a tool to assess aspects of SEL that are important for children’s education in Tanzania. Using this work as a case study, we describe the challenges inherent in developing such a tool. The tool was developed as part of the USAID Tusome Pamoja preschool program in Mtwara, Southern Tanzania.

CIES 2018 Presentation, given by Matthew Jukes. Randomized controlled trials (RCTs) have been used increasingly in an effort to reduce uncertainty and improve rigour of impact estimates. However, RCTs do not provide certainty in other important aspects of the evaluation, such as its external validity and in understanding which components of a complex program are essential for its effectiveness. This paper reviews two additional approaches which are currently underutilized which may help researchers and policy makers make better use of uncertain evidence.

Article published in the International Journal of Education and Development, Volume 50. From Journal abstract: "Initial access to school is nearly universal in Kenya, but many children who enroll drop out before completing primary school. In this mixed-methods study, we use quantitative data from a randomized control trial involving 2666 upper primary-grade students, as well as qualitative data from interviews with 41 schoolchildren, dropouts, and parents, to examine dropout. Poorer baseline performance on literacy and numeracy assessments predicted a higher risk of dropout. Interviews revealed that children are the primary decision-makers rather than parents. Together, these findings suggest that school quality interventions may be an effective means of reducing primary school dropout in this region."

Article published in Journal of Research on Educational Effectiveness. Published abstract: We evaluated a program to improve literacy instruction on the Kenyan coast using training workshops, semiscripted lesson plans, and weekly text-message support for teachers to understand its impact on students’ literacy outcomes and on the classroom practices leading to those outcomes. The evaluation ran from the beginning of Grade 1 to the end of Grade 2 in 51 government primary schools chosen at random, with 50 schools acting as controls. The intervention had an impact on classroom practices with effect sizes from 0.57 to 1.15. There was more instruction with written text and more focus on letters and sounds. There was a positive impact on three of four primary measures of children’s literacy after two years, with effect sizes up to 0.64, and school dropout reduced from 5.3% to 2.1%. This approach to literacy instruction is sustainable, and affordable and a similar approach has subsequently been adopted nationally in Kenya.

Article published in the International Journal of Educational Development. Published abstract: "What should be considered when developing a literacy intervention that asks teachers to implement new instructional methods? How can this be achieved with minimal support within existing policy? We argue that two broad sets of considerations must be made in designing such an intervention. First, the intervention must be effective by bridging the gap between current teacher practice and the scientific literature on effective instruction. This broad consideration is detailed with 10 design recommenda- tions. Second, the intervention must be amenable to being scaled-up and mainstreamed as part of government policy. This involves being (i) simple and replicable; (ii) well received by teachers; and (iii) cost effective. The paper describes how these factors were considered in the design of a literacy intervention in government primary schools in coastal Kenya. It also includes reactions from teachers about the intervention and their change in knowledge."