This presentation was delivered to a team of researchers who participated in a "Data Deep Dive" convened by the XPRIZE Foundation after the announcement of the Global Learning XPRIZE award.

The Early Grade Reading Barometer is an interactive dashboard that lets you: - SEE an overview of how well students are reading, the percentage of struggling readers, and how comprehension is associated with reading fluency. - VIEW information about student performance on EGRA subtasks. See how outcomes vary by key student and school characteristics. - EXPLORE how different EGRA subtasks are related to one another. View graphic displays of these relationships. - CHOOSE a target oral reading fluency benchmark, and see information on how many students are meeting the targeted benchmark now and how the percentage of students meeting the benchmark value could change over time. - SEE how countries compare with each other with respect to student outcomes on EGRA assessments and progress towards meeting UN SDG Goal 4. - REVIEW the impact of interventions aimed at improving the fundamental reading skills of students. - COMPARE how the distributions for selected EGRA subtasks have changed or remained the same over time.

The Government of Tanzania is in the process of implementing a policy of one year of free and compulsory pre-primary education for all children. The policy pronouncement was made within the context of limited resources for education and unclear implementation guidelines. Nonetheless, the declaration of fee-free and compulsory pre-primary led to an immediate increase in enrolment of 46% in the year after the policy was established. While access has improved, quality has not. A recent study of school readiness of children starting Standard 1 in Mainland Tanzania (78% of whom completed pre-school) found no difference between those who attended pre-primary school and those who did not (RTI International, 2017). The study also found weaknesses in the quality of early learning environments, such as high pupil-teacher ratios, poor pedagogy, and lack of high-quality teaching and learning materials. The key challenge facing the Government of Tanzania is how to develop an early learning system that can produce learning, not just broaden access, and to do so in a manner that is sustainable. Earlier experiences with the drive for universal primary education have shown that it is possible (and perhaps easier) to improve access to school without producing any learning. The current study uses a scale and sustainability framework proposed by Crouch and DeStefano (2017) to examine the extent to which the pre-primary (early learning) education system in mainland Tanzania is set up to provide quality pre-primary at scale and sustainably. They identified a core set of functions that education systems should be able to perform to produce learning: (1) set and communicate learning expectations; (2) monitor against expectations; and (3) provide minimum inputs to all schools, and targeted support to struggling schools and classrooms. We interviewed various actors at the national, district and school levels including government officials from the ministries responsible for education and local government; head teachers; teachers; parents of pre-primary aged children; and lecturers and administrators from teacher training colleges. In addition, we reviewed policies, plans, and strategies related to education and development. We found that the early learning system is still developing its capacity to perform the identified core functions. There are some learning goals outlined in a new curriculum, but most of the goals expressed in plans and policies relate to access, not learning, and are not known throughout the system. Monitoring against learning is weak as officials responsible for monitoring schools are unable to do so on a regular basis, and even when they do go, their observation protocols are not linked to learning or curricula expectations. Accountability mechanisms are weak as there is not enough data within the system to track performance against learning expectations. Finally, the Government of Tanzania struggles with providing basic instructional inputs – teachers, teaching and learning materials, continued professional development – to the pre-primary school system. This study and its findings are important because they point to priority areas for system reform for the Government of Tanzania and other countries facing the challenge of producing learning at scale sustainably.

Several forces are at play in determining whether pedagogical approaches are optimally adapted to the culture of children’s behaviour and of teacher-child interaction in the classroom. Teachers’ expectations for children’s behaviour may differ from the way in which children are raised at home (Jukes et al, 2018). Teaching activities may be designed by experts from outside the beneficiary education system or who may be removed from the culture of rural schools. The evidence for the effectiveness of recommended teaching practices may be based on children and teachers from a different culture. The current study sought to understand the role of cultural factors in the teaching of early grade reading in Tanzania. The aim was to investigate teacher’s pedagogical choices and the implicit theory of teaching and learning that underpinned these choices. We also aimed to understand teachers’ perceptions of students’ social and emotional skills and how this influenced their pedagogical choices. The current study took place in three regions of Tanzania – Zanzibar, Mtwara and Iringa – in the context of the USAID supported Tusome Pamoja (“let’s read together”) project. Researchers observed one lesson from each of 36 teachers and recorded key teaching activities. A subsequent qualitative interview with the teachers examined the decisions they made during the lesson and how their perception of students’ competencies influenced their decisions. Two themes emerged from the results. First, teachers said that children who did not participate in classroom activities lacked confidence and curiosity, particularly in rural areas. These two qualities - confidence and curiosity - were identified in previous work as traits valued more by teachers than parents in the context of learning. Second, teachers said the independent activities were not effective because pupils always cooperate with others in life. There was a strong preference for activities that involved the whole class and against students learning by doing, independent of the teacher. Schools can respond by conducting more activities that build curiosity and confidence. Children becoming more confident and curious in rural Africa represents a cultural change. But one that emerges from the community - not imposed upon it. Instructional design should consider the strong cultural preference for group-orientation and work with - rather than against - this preference where possible.

There is growing demand for policy based on rigorous evidence. Many consider the strongest evidence to come from studies that identify causality with high internal validity - such as RCTs - and systematic reviews of these studies. If policies are based strictly on such rigorous evidence there is a risk of bias towards simple, discrete, measurable interventions and away from complex interventions. Rigorous evidence is also better suited to some questions than others. Evaluations may provide stronger conclusions about impact than about the mechanisms, implementation, context, generalisability and scaling of interventions. For these reasons, policy-making does – and should – consider issues for which there is no conclusive evidence. However, there is little guidance as to how and when such inconclusive evidence can be used. We present a framework for considering inconclusive evidence applied to examples from evidence-based education in low- and middle-income countries. The framework involves a systematic consideration of the estimated costs, benefits and potential harm of a policy, along with the uncertainty in those estimates. This analysis is conducted using standard decision theory and an examination of the utility of policies. We argue that it is rational to pursue a policy with uncertain outcomes if there is a reasonable probability of large positive utility (compared to the cost of the intervention) and a low probability of negative utility. The decision to act under uncertainty is influenced by a number of other considerations including: the potential to improve the evidence base, the urgency of the decision and the analysis of alternative options. The framework also calls for systematic analysis of uncertainty associated with all components of a policy decision. For example, some interventions may have robust evidence of impact but considerable uncertainty associated with the generalisability of the evidence to a new context, or with the scalability of the intervention. We discuss our approach to measuring and reducing uncertainty in policy decisions and its implications for evaluation and research. The overall aim of this work is to make evidence-based decision-making more effective and applicable to a wider range of problems.

CIES 2018 Presentation, given by Tara Weatherholt. A current challenge in the equitable access to quality pre-primary education for young children is the current lack of relevant and reliable data on pre-primary education to inform improvement of the education sector in developing country contexts. In 2015, the global Measuring Early Learning and Quality Outcomes (MELQO) initiative was formed by UNICEF, UNESCO, the World Bank, and the Brookings Institution to focus on facilitating feasible, accurate, and useful measurement of young pupils’ development at the start of primary school as well as the quality of pre-primary learning environments, particularly in low- and middle-income countries. The first nationally representative school readiness study utilizing the MELQO suite of instruments was recently completed in Tanzania. This presentation will report on the quality of pre-primary education environments in both Mainland Tanzania and Zanzibar, as found through the national MELQO study conducted in early 2017, and how this may inform sector improvement.

CIES 2018 Presentation, given by Matthew Jukes. There is an increased demand for assessments of social and emotional competencies of young children in low- and middle-income countries. These competencies are increasingly seen as important for children’s development and for their education. In the context of preschool and primary education, such assessments have a number of uses. They are used to evaluate the impact of programs on children’s social and emotional learning. They can also be used to monitoring individual children’s progress in such programs and to tailor interventions to their needs. We developed a tool to assess aspects of SEL that are important for children’s education in Tanzania. Using this work as a case study, we describe the challenges inherent in developing such a tool. The tool was developed as part of the USAID Tusome Pamoja preschool program in Mtwara, Southern Tanzania.

Final, validated instruments used in the 2017 Tanzania National EGRA. The EGRA and SSME were made possible with funding from USAID. The EGMA component was made possible by funding from Global Affairs Canada. The Life Skills component was funded by UNICEF Tanzania.

Dissemination of the results from the 2015/2016 round of the Early Grade Reading Assessment (EGRA), the Early Grade Mathematics Assessment (EGMA), the Snapshot of School Management Effectiveness (SSME), and Life Skills occurred at four forums. First, a National Dissemination Workshop for education stakeholders, government representatives, and development partners occurred in mid-June 2016 in Dar es Salaam, Tanzania. This workshop was followed by three locally led, Regional Dissemination Workshops held in late October 2016 in three central regions and attracted practitioners and education representatives from surrounding districts and regions. Third, a Directors’ Meeting was held in mid-November 2016 in Dodoma, Tanzania. The Directors’ Meeting attracted high-level representatives from relevant branches of the Ministry of Education, Science, and Technology (MoEST) and key stakeholders to discuss the findings and their implications for education policy and planning for Tanzania. The final event was a two-hour presentation to the Ministry of Education in Zanzibar to present findings specific to the Zanzibar context. The four events are further discussed in the remainder of this report.

Findings report from the Tanzania National EGRA, EGMA, SSME, and Life Skills.