This case study examines a training targeting 18,000 head teachers and subject heads over seven days using IVR accompanied by electronic materials shared via WhatsApp conducted by the USAID-funded and RTI-implemented Malawi Early Grade Reading Improvement Activity (MERIT). While the training was conducted in June of 2020, this case study includes findings from a follow up study one conducted year later, to see what training content and skills participants retained, and the training elements that were most effective in their delivery. The case study applies a process-based and learning-oriented approach, drawing from the MEL-Tech Framework (Paper 1), to understand technology-supported remote teacher training introduced in response to COVID-19.

Due to increasing rates of COVID19 infection in Cambodia, primary schools were closed from March 2021 through the end of the school year. In Kampong Thom, Kampot, and Kep, the USAID-funded and RTI-implemented All Children Reading-Cambodia project rolled out comprehensive activities to support learning at home. While these interventions were implemented successfully, students had still missed nearly an entire year of explicit reading instruction. When schools reopened, they, and their teachers, would need intensive and targeted support to help make up for lost learning and to sustain gains under the National Early Grades Reading Program, Komar Rien Komar Cheh. To help inform this support, All Children Reading conducted a “temperature check” study with the aim of understanding teachers’ current status, well-being, concerns, capacity gaps, and plans for reading instruction when schools reopen; as well as measurement of students’ levels of reading proficiency, vis a vis where they are expected to be reading according to the curriculum. This study sought to better understand teacher’s perceptions around returning to school after prolonged closures, and how they plan to support students, in order to strategically target support that is both in line with best practice and responsive to where teachers are. A purposeful sample of 100 schools was drawn from two provinces, Kampong Thom (50 schools) and Kampot (50 schools). Schools were randomly sampled, proportional to each district. Data will be presented from 200 phone-based interviews conducted with grade 1 and 2 teachers, 100 in-person school director interviews, and 1,600 levelled student assessments conducted with grade 1 (800) and grade 2 (800) students. Findings are presented on the following: •Teachers’ levels of confidence returning to school and implementing the national Komar Rien Komar Cheh reading package, and specific areas where they still struggle •Teachers’ concerns and priorities for students’ learning in the coming year •Teachers’ plans to help students catch-up on missed content and skills, vis a vis their approach to curriculum coverage •Students’ reading levels in grade 1 and grade 2 at the end of a protracted school closure for most of the year, relative to where they should be per the curriculum

The All Children Reading–Asia (ACR–Asia) Return to Learning Cambodia Case Study was designed to research the Cambodia Ministry of Education, Youth, and Sport’s (MoEYS’s) response to the disruption in learning created by the COVID-19 pandemic during different phases of school closure and reopening in the 2019–2020 and 2021 school years. The study used qualitative interview methods to study the education system’s response, from the central ministry level down, to how school directors and teachers implemented expectations communicated to them. Forty-four participants were interviewed, including central MoEYS personnel, school directors, teachers, and caregivers. Participants were asked about the phases of MoEYS response to the pandemic and what procedures, implementations, and subsequent outcomes occurred The presenter will present findings that positive changes were implemented in the participating schools. Many stakeholders within the education system and its stakeholders act and react to an education crisis in ways that would not be typical of the system functions and individual behaviors experienced in a regular school year. Fundamentally, the education system and its actors showed that during COVID-19, they could carry out a set of core functions (RTI International, 2018). These core systems functions were to set and communicate expectations, monitor progress, provide targeted support to facilitate students’ access to continued education during school closures, and make stakeholders accountable for meeting the expectations.

“Agent based modeling: A method for understanding individual, social, and environmental influences on learning and equality in the classroom.” Learning science in the past decade has provided considerable evidence that learning is at once emotional, social, and cognitive (MH Immordino-Yang, et al 2018). As we seek to improve social, emotional and learning outcomes in schools around the world, it is important to better understand how individuals adapt to and influence each other and their environments as they connect and interact daily in and outside the school and classroom. Furthermore, it is important to develop a better sense of how local interactions shape education. How do individual interactions shape the patterns of learning outcomes in a school or at a larger scale such as a district? How do they shape the nature of the learning environment and in turn, how do differential learning environments shape the patterns of interactions and relationships in a school? This information holds enormous potential to inform international education programming that may hold promise for improved uptake of innovations and education outcomes. How interactions locally shape education – in schools, administrative offices, or higher education institutions – is not well understood or studied extensively. Agent-based modeling (ABM) is a technique that can be applied to better understand the link between local dynamics of individuals in a school or community and certain aggregate characteristics of a school or emerging school changes. ABMs are based on the application of algorithms or simple rules representing the lower-level interactions of individuals (or system components) that give rise to higher level system structures or changes, providing a tool to understand bottom-up influences on education outcomes. (See M Macy & R Willer, 2002; M Jacobson, et al, 2017). In this presentation we present an agent-based model to show the potential impact of teacher feedback on student participation in the classroom and the relative impact of students who are more or less vulnerable (e.g., have lower/higher ability levels and are from more/less marginalized backgrounds). The model was informed by student data from primary schools in Uganda and Tanzania. The model demonstrates that over time, when met with repeated experiences of negative feedback, more and more students will quit participating entirely and some will dropout, especially children who are more vulnerable. On the other hand, when teachers are increasingly positive, more and more students participate more actively, even among the most vulnerable children. Thus, the nature of teachers’ responses to students when they answer questions in class can powerfully impact student participation and shape equality in participation. To extend, this would seem to impact student learning. The objectives of this presentation are: 1. To introduce the agent-based modeling method. 2. To present an application of ABM in international education 3. To demonstrate the utility of ABM in research, policy dialogue, and programming.

Social network analysis methods for international development: Applications in higher education development. Social Network Analysis (SNA) is a promising yet underutilized tool in the international development field. SNA entails collecting and analyzing data to characterize and visualize social networks, where nodes represent network members and edges connecting nodes represent relationships or exchanges among them. SNA can help both researchers and practitioners understand the social, political, and economic relational dynamics at the heart of international development programming. It can inform program design, monitoring, and evaluation to answer questions related to where people get information; with whom goods and services are exchanged; who people value, trust, or respect; who has power and influence and who is excluded; and how these dynamics change over time. This brief advances the case for use of SNA in international development, outlines general approaches, and discusses two recently conducted case studies in higher education development that illustrate its potential. It concludes with recommendations for how to increase SNA use in international development. Key research recommendations advanced by this paper include: • Incorporate SNA into monitoring, evaluation, and learning processes. SNA can be conducted at various points of a project to inform program design, adaptive management, learning, and evaluation by considering network structure and network changes over time. • Demystify the use of SNA. Increased use of SNA tools and clear presentation in widely read publications are needed to bring the analytic approach into the mainstream of international development. • Build capacity to conduct SNA. The capacity to conduct and interpret SNA is lacking across actors in international development. Efforts by some organizations to build capacity in the community are well noted and should be built upon. • Build understanding of relationships between social networks and development outcomes. SNA will be useful only to the extent it helps users understand the relationship between networks and development outcomes that matter. • Establish norms for data collection and identity protection. Data about individuals and their interactions with others are inherently sensitive data. As a part of standard research ethics protocols, SNA practitioners must make carefully considered decisions about how or if to anonymize data when reporting it.

Title: Right-fit EdTech: Leveraging Loquat and other (machine) learning to support foundational learning in LMICs Abstract The COVID-19 pandemic has severely disrupted global education systems: according to the Global Education Monitoring Team, over 1.6 billion students have lost significant instructional time, with many yet to return to school. But even before the pandemic struck, the state of education around the world was so poor that the World Bank was decrying a “learning crisis.” To address the crisis, the Bank called for both a renewed emphasis on what teachers actually do in a classroom and the strategic deployment of technology to improve teaching and learning. During the pandemic, remote-first instruction became a feature of basic, secondary, and tertiary education systems in many high-income countries. [European Commission, Rickles et al. 2020, Hodges et al. 2020, Gillis & Krull 2020] Coverage was not universal, as wealth and income inequality within high-income countries remains a persistent challenge. [Romm 2020, Blaskó & Schnepf 2020] Nonetheless, the market for educational technology firms boomed during the pandemic, with investment in the sector expected to exceed $80 billion by the end of the decade. [Gillespie 2021] The growth in remote schooling, while temporary, coincides with a long-standing and ongoing interest in so-called “artificial intelligence” or “machine learning” technologies and their applications in business contexts. [Sestino & de Mauro 2021, Benaich & Hogarth 2018, 2019, 2020, 2021] An increasing number of so-called “EdTech” (education technology) companies are riding this wave to attain “unicorn” status (a market valuation in excess of $1 billion), often touting their use of cutting-edge machine learning techniques to support their products or deliver their services. In many lower- and middle-income country (LMIC) contexts, transitions to remote learning were uneven, with significant variance in access, quality, and coverage. [World Bank 2020, UNICEF 2020] Indeed, in many LMICs even the most basic of modern technologies can be unavailable: as of 2015, up to 90 million children in Sub-Saharan Africa were studying in classrooms that lacked electricity. What is the relevance to these education systems of analytical techniques developed in contexts where computing devices are plentiful? This presentation will answer that question in two ways. First, by introducing Loquat, a novel application from RTI International that leverages automatic speech detection and the increasing ubiquity of low-end smartphones to give teachers accessible, affordable instructional coaching. Using machine learning, Loquat detects and classifies verbal interactions between teachers and students. Automated analyses translate these data into simple but powerful visualizations that, combined with guided reflection, provide tailored, actionable feedback teachers can use to understand their talk management and facilitate its improvement. Second, the presentation will review recent advances in machine learning and so-called “artificial intelligence”, detail their potential relevance to lower- and middle-income country education systems, and discuss how programs that aim to improve foundational learning outcomes could begin to leverage these powerful new tools.

Description of collaborative process used ensure coherence and alignment of curricular content of preservice teacher preparation and in-service teacher professional development.

Digital transformation: an opportunity for long-lasting improvement of quality of education service delivery in Kyrgyz Republic, Tajikistan, and Uzbekistan Over the past decade, Uzbekistan, Tajikistan, and Kyrgyz Republic have been consistently demonstrating their commitment to quality education, evidenced by the amount of GDP allocated to education. These education systems aim to produce world-class graduates capable of success both globally and at home. Various donors have stepped in to support these efforts in a coordinated way via direct funding of initiatives that enhance instructional skills, increase accountability for excellence, and improve formal and non-formal learning environments. In Uzbekistan, critical thinking, problem solving, and practical skills are at the forefront of the Ministry of Public Education (MPE)’s agenda. In the Kyrgyz Republic, performing well on the upcoming PISA survey in 2025 is a focal point for the Ministry of Education and Science (MOES) in recent years. And, in Tajikistan, COVID-19 provided an important opportunity to rethink the delivery of in-service teacher training from in-person to a blended learning format, resulting in impressive early grade learning gains. However, the COVID-19 crisis brought many education systems to a grinding halt, and these three countries were no exception. Nevertheless, the speed and efficiency with which their education systems adapted has been remarkable. It speaks to their resilience, rooted in the fact that these countries enjoy strong institutional, financial, and policy legacies that are strongly rooted and capable of weathering storms. Undoubtedly, the strongest driving force behind this change is the belief shared at all levels of their education systems – especially at the level of parents and teachers – that education is important. It comes as no surprise that when faced with the challenge of school closure due to a pandemic, their various systemic actors collaborated and adapted their approaches to continue delivering teacher professional development (TPD). In the Kyrgyz Republic, after less than a year following the outbreak of the pandemic, the majority of teachers reported accessing online learning resources, and 85% of teachers did so using smartphones. The USAID-funded Okuu Keremet! project had planned to deliver teacher training in person, but swiftly shifted to development of blended learning modules for reading and mathematics. It designed an online training platform that in a short period of time was accessed by 18,000 registered teachers and librarians. The online learning platform integrates Google Analytics that can allow the Ministry of Education and Science (MOES) to not only monitor progress towards certification of individual teachers, but to also detect, in real time, the teachers and schools that are lagging behind . In Uzbekistan, the Ministry of Public Education (MPE) has made significant investment in continuous professional development, and as part of its growth strategy, Avlonyi has created a Learning Management System (LMS) for teacher accreditation based on OECD teacher professional development standards. To aid Avlonyi in this significant undertaking, the USAID-funded Uzbekistan Education for Excellence Program (UEEP) project developed a blended TPD model comprised of synchronous and asynchronous in-person and online assessment-based learning events (with the focus to improve Information and Communication Technology (ICT) and English as a Foreign Language (EFL) instruction for grades 5–11 and 1-11, respectively). In addition to gaining skills through this opportunity, teachers will achieve career advancement through the accreditation of their efforts by Avlonyi. In Tajikistan, the impact evaluation data shows strong performance as a result of the USAID Read with Me project implemented in collaboration with the Ministry of Education. TPD in Tajikistan applies digital development principles, a multi-user coaching and training platform, which provides content and accountability for self-regulated learning, coaching, peer learning and performance monitoring, including uploading assignments and assessments, aligned with the National Teacher Competency Framework. The EGRA end line results clearly showed the benefit of effective teaching techniques correlating to oral reading fluency gains of more than thirteen correct words per minute. This panel brings together researchers and practitioners documenting the transformational change of Teacher Professional Development in the three Central Asian Republics – Kyrgyz Republic, Uzbekistan and Tajikistan. Panelists will discuss details about the effectiveness and utility of online learning platforms vis-à-vis TPD and student learning gains; the skills required to make this shift to digital, self-regulated learning; and lessons learned for getting these new platforms mainstreamed into educational systems.

The CRLA was initially conceived as a 5-minute start-up reading assessment designed to help teachers quickly determine the reading profiles of their G1-G3 learners, and develop appropriate reading instructional strategies. The main goal is to identify children who need additional support in reading. The CRLA was developed in response to the extended break that learners experienced over the summer of 2020 due to the pandemic. After 32 weeks of no class, teachers needed a rapid, easy to use tool, to determine what level of readiness and support the returning children needed. The beginning of school year tasks focused on letter sounds, isolated words and sentence in reading in MT (G1), Filipino (G1-G2) and English (G1-G3). In the Philippines, learners are expected to transition from their mother tongue (MT) to Filipino and English (L2 and L3 respectively) by grade 4. However, the earliest standardized assessment used in the Philippines is for Filipino starting in G3 and English starting in G4. Prior to the CRLA, there were no existing standardized tools or systems used to assess in the MT. It was also a tool that could be administered remotely (online or even over the phone) which proved critical for schooling in the new normal where face to face classes were not permitted. Thus, the CRLA met the needs of teachers who were missing such a tool in their early grade assessment toolbox. The CRLA was initially piloted in November 2020 with select schools. The experience of the pilot led DepEd to scale up the CRLA to all schools in the ABC+ Target Regions. The feedback from the schools and teachers who utilized the CRLA was encouraging. Many schools used the results of the CRLA to develop remedial and focused reading programs in the context of the new normal and identify the struggling readers to provide additional support. Based on the success of the beginning of school year RLA, the Department of Education requested ABC+ to help develop middle and end of year assessments that would align to their curricular competencies. Keeping to the same criteria (rapid and easy to administer and analyze), the team incorporated listening comprehension, oral reading fluency and reading comprehension tasks for end of year administration. At the request of DepEd, ABC+ is currently in the process of expanding the number of MT languages supported by the CRLA and incorporating it into their operations manual for mother tongue-based multi-lingual education.

Efforts to promote playful pedagogies in low- and middle-income countries face many challenges. Among these challenges are the norms of teacher-child interaction that have a strong cultural basis. These norms include children’s reluctance to speak in front of adults, teachers’ desire to avoid embarrassing children and to promote a sense of fairness and togetherness in the classroom (Jukes et al, 2021a). In this paper I suggest an approach for adapting pedagogy to cultural context. The approach involves designing teaching activities to achieve two goals. Building on the work of others (O’Sullivan, 2004; Schweistfurth, 2013), the first is the cognitive learning goal of the activity. The second is the culturally appropriate social goal of the activity. This framing helps to design instructional approaches that promote learning and can be readily taken up by teachers. This approach is particularly effective when teaching activities are co-constructed with teachers (Randolph et al, 2020). In this paper I will discuss 3 studies of teacher perceptions and classroom behaviours in Tanzania that led to the development of this approach (Jukes et al, 2021b). I will also present one example of a teacher co-creation workshop in Tanzania based on the suggested approach to contextualising teaching activities. I will also illustrate the application to the contextualisation of playful pedagogies, based on qualitative research in Kenya and Ghana.