Global

This report presents the results of RTI International Education’s study on teachers' guides across 13 countries and 19 projects. Using quantitative and qualitative methods, we examine how teachers’ guides across the projects differ and find substantial variation in the design and structure of the documents. We develop a scripting index so that the scripting levels of the guides can be compared across projects. The impact results of the programs that use teachers’ guides show significant impacts on learning outcomes, associated with approximately an additional half year of learning, showing that structured teachers’ guides contribute to improved learning outcomes. During observations, we find that teachers make a variety of changes in their classroom instruction from how the guides are written, showing that the utilization of structured teachers’ guides do not create robotic teachers unable to use their own professional skills to teach children. Unfortunately, many changes that teachers make reduce the amount of group work and interactivity that was described in the guides, suggesting that programs should encourage teachers to more heavily utilize the instructional routines designed in the guide. The report includes a set of research-based guidelines that material developers can use to develop teachers’ guides that will support effective instructional practices and help improve learning outcomes. The key takeaway from the report is that structured teachers' guides improve learning outcomes, but that overly scripted teachers' guides are somewhat less effective than simplified teachers' guides that give specific guidance to the teacher but are not written word for word for each lesson in the guide.

This paper highlights patterns in school enrollment indicators that affect the efficiency and effectiveness of education systems in a set of low-income countries: those that have expanded access quickly in the last decade or two, but have not yet absorbed that expansion efficiently. Although the patterns in these indicators are observable in the first few years of schooling, they could constitute a cause of low learning outcomes at the end of primary school. The data show strong empirical relationships between an early primary enrollment bulge, low levels of pre-primary participation, and poor performance on early grade cognitive skills. This work does not attribute causal precedence to these patterns but instead argues that the indicators are reflections of each other, constituting a ‘‘knot’’ of issues undermining the foundations of the affected education systems. The article presents some of the cost implications and suggests that many countries are already paying for pre-primary education without realizing it.

CIES 2018 Presentation, given by Jessica Mejia. In order to understand how literacy programs differ with respect to the amount of scripting in teachers’ guides, we studied teachers’ guide documents from 19 projects in 13 countries. We present the quantitative differences in the level of scripting in these projects. In addition, we selected four countries that had substantial differences in the level and type of scripting in their teachers’ guides. We observed classrooms in those contexts to determine what type of modifications teachers made to the lesson plan as written in the teachers’ guides. We also interviewed the teachers and asked them to describe the reasons for their modifications to the teachers’ guides.

CIES 2018 Presentation, given by Matthew Jukes. Randomized controlled trials (RCTs) have been used increasingly in an effort to reduce uncertainty and improve rigour of impact estimates. However, RCTs do not provide certainty in other important aspects of the evaluation, such as its external validity and in understanding which components of a complex program are essential for its effectiveness. This paper reviews two additional approaches which are currently underutilized which may help researchers and policy makers make better use of uncertain evidence.

CIES 2018 Presentation, given by Amber Gove and Helen Jang. The Early Grade Reading Barometer offers a wealth of actionable Early Grade Reading Assessment (EGRA) data to help to change the lives of young children. With 40 datasets from 19 places, the Barometer supports data transparency. The Barometer is designed for USAID education officers, policy makers, education staff in host countries, implementation partners, education researchers, and practitioners. It is designed primarily for users who may have limited knowledge about early grade reading, and no or little sophisticated statistical knowledge. This presentation was an interactive demonstration of the Barometer, highlighting the vast amount of data available to the public and how the information can be used to improve learning outcomes.

CIES 2018 presentation, given by Luis Crouch on behalf of Hank Healey. The EGR Sustainability Framework emanates from a) a general understanding of evidence-based EGR programs, b) the notion of a “learning coherent” education system (Pritchett, 2015), c) the notion of a “bare bones” or “core functions” system (Crouch and Destefano, 2015); and effective curriculum implementation. To this end, our proposed EGR sustainability framework maintains that one must a) map the existing education system’s institutional, systemic, cultural, and attitudinal capacity to function as an effective learning coherent core functions curriculum implementation system, b) identify the various gaps and barriers that prevent the system from working in this manner, c) develop a plan to address those gaps and barriers, and d) help the MOE to implement that plan.

CIES 2018 Presentation, given by Amber Gove. With few exceptions (Aboud & Hossain, 2011; Gertler et al., 2013), much of the published research on the impact of preprimary participation stems from high-income countries (Campbell et al., 2014; Heckman, Moon, Pinto, Savelyev, & Yavitz, 2010). Few studies have examined preschool and later learning across multiple countries (Raikes, Devercelli, & Kutaka, 2015). This paper seeks to fill this gap by examining the relationship between preprimary participation and early primary reading outcomes, as measured by Early Grade Reading Assessment (EGRA) (Dubeck & Gove, 2015; RTI International, 2016). Drawing on a unique set of data using student-level learning assessments from sixteen low and middle income countries, we use preprimary participation to explain primary reading outcomes.

CIES 2018 Presentation, given by Luis Crouch. Given the prominence of learning and quality in the SDGs, much discussion has gone into how to measure in a manner that is reasonably comparable. Some have argued (or feared) that this would necessitate a single, dominant, global assessment. Aside from the political or ethical acceptability of this kind of imposition, one has to wonder how meaningful this could be, psychometrically or pedagogically. The paper will argue that unless one were to increase the cost of assessment tremendously, or make children sit through lengthy assessments or until highly adaptive computerized assessments can be used, a single or a few dominant assessments are an unlikely approach. Instead, a variety of assessments is a more likely solution. These might have better psychometric “resolution” for poorer countries with greater cognitive inequality than the OECD countries, might be closer to the children’s actual levels, and might be psychometrically more reliable if done properly. The ability to make the results comparable or equitable in some sense need not be lost, however, if some sort of universal learning scale is created, so that countries can peg themselves, with reasonable rigorous, to that scale. A global neutral arbiter can “sponsor” the scale and compile country-based reports using it.

This presentation was prepared for UNESCO Mobile Learning Week, 2018. It describes an RTI International internal research program to learn whether mobile gaming can be used to assess soft skills important for employability. The presentation was created by Lee Nordstrum and delivered by Sarah Pouezevara.

While the SDGs now officially call for global reporting on learning outcomes, many institutions and scholars had noticed, at least since the mid-2000s, that many children were not learning much, and were starting to respond by, as a first step, advocating and developing assessments that, they felt, was perhaps more appropriate to learning at the bottom of the pyramid, or at the left end of the cognitive distribution. The GMR sounded a clarion with their estimate that there are some 250 million children in the world hardly learning. This paper addresses an issue that can be put simply but is extremely hard to answer: “Is the array of assessments emerging helping researchers, policy-makers, and implementers get a more accurate sense of how much or how little the poorest children in the world know, and is it helpful in remediating the situation?” The paper specifically is not addressed at the question of global reporting—although it does touch upon the issue. The problem of interest here is what is most useful for countries to generate movement along the bottom of the pyramid.