In 2014, the Nepal Ministry of Education, Science, and Technology (MOEST) developed a National Early Grade Reading Program (NEGRP), aimed at improving the reading achievement of early-grade students throughout Nepal. USAID has supported the government’s efforts to improve early-grade reading under NEGRP through both Government-to-Government funding and through the Early-Grade Reading Program (EGRP) I & II from 2015-2022. The NEGRP focused on evidence-based pillars for foundational literacy: Curriculum and materials, Teacher training and ongoing support, Community Mobilization, Assessment, and Systems strengthening. This paper will focus on the teacher ongoing support pillar of NEGRP and describe how processes of Monitoring, Evaluation, Research, Learning and Adaptation informed the evolution of the Ministry’s Teacher Professional Support System. EGRP & EGRP II were implemented in the context of Nepal’s transition to federalism, which began following the ratification of its constitution in 2015. This resulted in a situation where roles and responsibilities within the education system were in flux and authority for decision-making and funding shifted to the local level. Thus, implementation decisions had to recognize and adjust to a context in-flux. From the outset of NEGRP, the model for ongoing teacher support was based on evidence in the field on what works, with a heavy reliance on external coaches who could provide feedback and support to teachers on a regular basis. Challenges in the Nepal context, including the diversity of contexts within Nepal, as well as the changes taking place due to federalism, made it apparent that adjustments would be needed. As a result, EGRP supported the MOEST to undertake the first implementation research study, which supplemented Monitoring and Evaluation data to identify the strengths and weaknesses of the teacher support model, as well as to identify possible alternatives. A second iteration of the government’s model for Teacher Professional Support was developed based on the findings from this learning and joint reflection by stakeholders, government officials, and project staff – together with the continuing evolution around teacher learning and support in the field more broadly. A second round of monitoring, learning, and operational research enabled EGRP and MOEST to further hone the model – as well as to provide needed further adjustments to align with the newly federalized system as it evolved. Based on those findings and joint reflection, GoN developed a Teacher Professional Support that incorporates a flexible menu of options that enables local authorities to determine the combination of approaches that will be most suitable for their local context and to address local challenges. This paper will briefly present the iterations to the NEGRP ongoing teacher support model, explore the MERLA approaches that were used in this adaptive process and discuss lessons learned. The authors would then seek to engage the audience in discussion around the broader implications and pose the question: how can program design build in opportunities for this type of iterative learning from the outset?

This presentation on development of an assessment system under Senegal's new bilingual policy includes an explanation of the bilingual policy in Senegal and plans for development of a national assessment system including use of the ANLAS methodology (Analysis of National Learning Assessment Systems)

The Kingdom of Jordan’s Human Resource Development (HRD) Strategy mandates the universal provision of kindergarten (KG2) by 2025. The HRD Strategy emphasizes using partnerships between the Ministry of Education (MoE) and other governmental and nongovernmental actors to assure expanded provision of kindergarten services. The strategy further emphasizes the need to improve the quality of kindergarten services, while reaching every child. To fully understand how the MoE can increase access, improve quality, and assure equitable provision of KG1 services, more accurate information is needed, especially about other, as yet unrecognized service providers who may be offering KG or KG-similar services to five-year-old children. The MoE needs a more complete picture of the current provision of KG2 to determine the best strategy to achieve the HRD goal that every child in Jordan receives a high-quality kindergarten experience.

This presentation was given at the Comparative and International Education Society (CIES) annual conference in Washington, DC on February 21, 2023. The presentation describes a systematic research method used in USAID's Early Grade Reading Program II (EGRP II) in Nepal to analyze the alignment between the early grade reading curriculum, teacher instruction, and student assessment. The file includes both the presentation slides and the explanatory notes.

Despite the growing interest in supporting learning through play across many low and middle-income countries, measures of how contexts can support learning through play are lacking. As part of the Playful Learning Across the Years (PLAY) project, the concept of “self-sustaining engagement” was identified as central to learning through play. That is, learning through play is effective because children are deeply engaged in their learning and are self-motivated to learn. The PLAY toolkit was designed to measure how settings – particularly adult-child interactions in those settings – support children’s self-sustaining engagement in learning. The toolkit was developed for use in multiple age-groups across different settings. For the 0-2 age-group, the toolkit assessed support for children’s engagement in the home, largely through interactions between the caregiver and child. These interactions were assessed through observations and through an interview with the caregiver. In the 3-5 age-group, tools were developed to measure support for engagement in the home and the classroom. Tools for the 6-12 age-group were focused only on the classroom. The classroom-based tools had several components. Teacher-child interactions were assessed through observations, a teacher survey and – for the 6-12 age-group only – a student survey. There was also a classroom inventory to assess physical aspects of the classroom – such as materials on the walls – which might support self-sustaining engagement in learning. The toolkit was developed in three phases – the Build phase used qualitative data to understand local concepts of self-sustaining engagement. The Adapt phase used cognitive interviewing and small-scale (approx. 25 schools, centers or homes) quantitative data to refine the tools. In the Test Phase we used large-scale (approx. 150 schools, centres or homes) quantitative data to assess the psychometric properties of the tool. This presentation focuses on these psychometric analyses. Data were collected for the 6-12 age group from Kenya, Ghana and Colombia; for the 3-5 age group from Colombia, Jordan and Ghana; and from Colombia only for the 0-2 age group. Results indicate how the concept of “support for self-sustaining engagement” can be divided into constituent sub-scales and how the different methods of assessment – observation, teacher report and student report – relate to one another. We will discuss plans to develop a final toolkit, based on these analyses, which can help strengthen the evidence base on learning through play.

The lack of information about children’s oral language skills limits our understanding of why some children do not respond to literacy instruction. Even though native language oral language skills are not strong predictors of native language decoding (Durgunoglu et al., 1993; Lesaux et al., 2006), oral language skills have been shown to have a small role in non-native word reading for non-native speakers (Geva, 2006; Quiroga et al., 2002). Yet, understanding that threshold of language skills is not understood. Cross-linguistic studies show that some literacy skills transfer between languages (Abu-Rabia & Siegel, 2002; Bialystok, McBride-Chang, & Luk, 2005; Cisero & Royer, 1995; Comeau, Cormier, Grandmaison, & Lacroix, 1999; Denton et al., 2000; Durgunoglu, 2002; Durgunoglu et al., 1993; Genesee & Geva, 2006; Gottardo, Yan, Siegel, & Wade-Woolley, 2001; Koda, 2007; Wang et al., 2006). This includes letter knowledge, print concepts, and language skills (phonological awareness and vocabulary). The transfer of these skills is considered a resource (Genesee, Geva, Dressler, & Kamil, 2006) that assists reading in the additional languages. Children learning to read in a non-native language bring their first language (i.e., mother tongue) to the instructional setting. Yet, its use will depend on the teacher’s use translanguaging between the language of instruction and children’s home language (s). The presence of two or more languages contributes to children having domains of knowledge in specific languages. For example, domains of knowledge children learn at school such as shapes, might only be known in the language of instruction. Relatedly, domains of knowledge they learn at home from family interactions, such as cooking, might only be known in the mother-tongue. And domains of knowledge that children learn on the playground, are likely to be learned in a lingua franca, or a common language to the area. Even though mixing languages is common, most language assessments do not capture this knowledge. Even in samples with multi-lingual students, for reasons of reliability and consistency, most language assessments assess children in just one language and describe results for that language. The results are used to help to explain results on reading assessments. But measuring language skills in just one language overlooks the concepts that a multilingual child may have in other languages and describe them from a deficit approach opposed to the asset of being multi-lingual. To address this problem, we developed a tool, the Multi-Language Assessment (MLA), to measure children’s expressive language across multiple languages to understand the skills they have to support their learning. The tool is intended to be reliable, valid, and child friendly. It is our hypothesis that children’s expressive language scores across multiple languages can help to explain their success or struggles in the early years of formal schooling. We conceptualized and developed the Multi-Language Assessment (MLA) to capture children’s language skills across multiple languages in a 7-minute interaction with a trained assessor. The MLA measures expressive language of 36 concepts shown in 36 images that children would be exposed to through family and community interactions, conversations, media, books, or school. The items included in the assessment yield variable distribution; they are not intended to be items that would yield ceiling effects. A child’s utterance is coded to one of nine categories of varying weights. Furthermore, the items are intended to have levels of a familiarity. For example, for an image of a coconut tree, some children might call it that, while other children describe it by its domain, a tree, not identifying the specific type. Both utterances would earn a child points but of varying weight. Research Problem 1. Many children in low- and middle-income contexts do not learn to read in lower primary efficiently. 2. Some people hypothesize that the reason for children’s poor performance is language related. 3. When children’s language skills are assessed it is usually in one language and describes their abilities as deficits as opposed to considering their assets of being multi-lingual. 4. Assessing language requires time and young children’s attention spans are short, reducing data quality if the assessment is too long. 5. The MLA was created to understand expressive language use across multiple languages and to be brief Study: The paper presents results from a recent longitudinal study that collected child level results at two time points in government school in rural Kenya. It includes the aforementioned multi-language assessment and measures of reading achievement (e.g., letter knowledge and spelling) at Time 1 when all children were in kindergarten and again at Time 2 when they had advanced to either a higher Kindergarten or to Grade 1. An existing measure of expressive language was used to explore concurrent validity of the MLA. The Time 1 sample (n=215) was large enough to examine the technical properties of the tool and the Time 2 sample (n=200) had only 7% attrition so there was sufficient power to describe individual changes. The features of the language assessment suggest that it is reliable and sensitive. The following analysis been conducted: 1) Sample demographics; 2) Distributions by subtasks; 3) Measures of association between subtasks; 4) Item analysis The following research questions are addressed: 1. How does expressive language use evolve for children who use three languages as they progress from kindergarten into first grade? For example, do they shift from using the home language for some items at Time 1 to a language of instruction at Time 2? 2. How does children’s overall expressive language knowledge (as measured in three languages) contribute to their reading achievement (letter sound knowledge and spelling) as measured in two languages over time?

This brief presents findings on what worked to improve learning outcomes at scale under 8 successful early grade literacy programs, with a focus on findings from the Tusome program in Kenya.1 These findings were generated as part of the Learning at Scale study, conducted by RTI International with the Center for Global Development and funded by the Bill and Melinda Gates Foundation.

Most of the research that informs our understanding of children’s social-emotional learning (SEL) comes from Western, Educated, Industrialized, Rich, Democratic (WEIRD) societies where behavior is guided by a view of the self as autonomous, acting on individual preferences. In the subsistence agricultural communities (home to more than a quarter of the world’s population), obligations and communal goals override personal preferences and individuals see themselves as part of a social hierarchy. These contrasting models of the self have profound implications for SEL. Many studies underestimate these implications because they use assessment tools developed in WEIRD settings to understand SEL in lower- and middle-income countries. The aim of our study was to build an SEL assessment from the ground up, based on community definitions of valued competencies in southern Tanzania. In Study 1, Qualitative data from parents and teachers indicated that dimensions of social responsibility, such as obedience and respect, were valued highly. Teachers valued curiosity and self-direction more than parents, as competencies required for success in school. Quantitative assessments in Study 2 found that individuals more exposed to sociodemographic variables associated with WEIRD settings (urban residence and higher parental education and SES) were more curious, less obedient and had poorer emotional regulation. Overall findings suggest that the conceptualization of social-emotional competencies may differ between and within societies; commonly held assumptions of universality are not supported. Based on the findings of this study we propose a systematic approach to cultural adaptation of assessments. The approach does not rely solely on local participants to vet and adapt items but is instead guided by a rigorous cultural analysis. Such an analysis, we argue, requires us to put aside assumptions about behavioral development and to consider culture as a system with an origin and function. Such an approach has the potential to identify domains of SEL that are absent from commonly used frameworks and to uncover other domains that are conceptualised differently across contexts. In so doing, we can create SEL assessments and SEL programs that are genuinely relevant to the needs of participants.

“Agent based modeling: A method for understanding individual, social, and environmental influences on learning and equality in the classroom.” Learning science in the past decade has provided considerable evidence that learning is at once emotional, social, and cognitive (MH Immordino-Yang, et al 2018). As we seek to improve social, emotional and learning outcomes in schools around the world, it is important to better understand how individuals adapt to and influence each other and their environments as they connect and interact daily in and outside the school and classroom. Furthermore, it is important to develop a better sense of how local interactions shape education. How do individual interactions shape the patterns of learning outcomes in a school or at a larger scale such as a district? How do they shape the nature of the learning environment and in turn, how do differential learning environments shape the patterns of interactions and relationships in a school? This information holds enormous potential to inform international education programming that may hold promise for improved uptake of innovations and education outcomes. How interactions locally shape education – in schools, administrative offices, or higher education institutions – is not well understood or studied extensively. Agent-based modeling (ABM) is a technique that can be applied to better understand the link between local dynamics of individuals in a school or community and certain aggregate characteristics of a school or emerging school changes. ABMs are based on the application of algorithms or simple rules representing the lower-level interactions of individuals (or system components) that give rise to higher level system structures or changes, providing a tool to understand bottom-up influences on education outcomes. (See M Macy & R Willer, 2002; M Jacobson, et al, 2017). In this presentation we present an agent-based model to show the potential impact of teacher feedback on student participation in the classroom and the relative impact of students who are more or less vulnerable (e.g., have lower/higher ability levels and are from more/less marginalized backgrounds). The model was informed by student data from primary schools in Uganda and Tanzania. The model demonstrates that over time, when met with repeated experiences of negative feedback, more and more students will quit participating entirely and some will dropout, especially children who are more vulnerable. On the other hand, when teachers are increasingly positive, more and more students participate more actively, even among the most vulnerable children. Thus, the nature of teachers’ responses to students when they answer questions in class can powerfully impact student participation and shape equality in participation. To extend, this would seem to impact student learning. The objectives of this presentation are: 1. To introduce the agent-based modeling method. 2. To present an application of ABM in international education 3. To demonstrate the utility of ABM in research, policy dialogue, and programming.

Social network analysis methods for international development: Applications in higher education development. Social Network Analysis (SNA) is a promising yet underutilized tool in the international development field. SNA entails collecting and analyzing data to characterize and visualize social networks, where nodes represent network members and edges connecting nodes represent relationships or exchanges among them. SNA can help both researchers and practitioners understand the social, political, and economic relational dynamics at the heart of international development programming. It can inform program design, monitoring, and evaluation to answer questions related to where people get information; with whom goods and services are exchanged; who people value, trust, or respect; who has power and influence and who is excluded; and how these dynamics change over time. This brief advances the case for use of SNA in international development, outlines general approaches, and discusses two recently conducted case studies in higher education development that illustrate its potential. It concludes with recommendations for how to increase SNA use in international development. Key research recommendations advanced by this paper include: • Incorporate SNA into monitoring, evaluation, and learning processes. SNA can be conducted at various points of a project to inform program design, adaptive management, learning, and evaluation by considering network structure and network changes over time. • Demystify the use of SNA. Increased use of SNA tools and clear presentation in widely read publications are needed to bring the analytic approach into the mainstream of international development. • Build capacity to conduct SNA. The capacity to conduct and interpret SNA is lacking across actors in international development. Efforts by some organizations to build capacity in the community are well noted and should be built upon. • Build understanding of relationships between social networks and development outcomes. SNA will be useful only to the extent it helps users understand the relationship between networks and development outcomes that matter. • Establish norms for data collection and identity protection. Data about individuals and their interactions with others are inherently sensitive data. As a part of standard research ethics protocols, SNA practitioners must make carefully considered decisions about how or if to anonymize data when reporting it.