This report summarizes the findings of an effort to develop and validate tablet-based, self-administered assessments of Chichewa-language foundational literacy and numeracy in the early grades in Malawi. RTI International developed the two assessments, known respectively as the Self-Administered Early Grade Reading Assessment (SA-EGRA) and the Self-Administered Early Grade Mathematics Assessment (SA-EGMA), with the support and at the direction of Imagine Worldwide. The assessments are deemed “self-administered,” because children complete the assessments independently in response to instructions and stimuli embedded in the tablet-based software. However, adults typically supervise the organization and conduct of the assessment as well as the collection of individual data from the tablets for analysis.

The purpose of this paper is to present a methodology for understanding materials usage in primary classrooms in Sub-Saharan Africa that centers teachers’ actions and voices. The United Nation’s Sustainable Development Goal 4 focuses on improved primary education around the world. To meet this goal, many large, donor-funded interventions aim to improve education through provision of teacher’s guides and student textbooks. However, what many of these interventions lack is a systematic way to understand how and why teachers are making pedagogical decisions while using materials. There is a large body of work that seeks to understand how teachers make decisions as they teach, and the ways these decisions are influenced by their knowledge and beliefs. Drawing from this work, we describe a methodology and set of tools that uses observations and interviews to identify key decisions that teachers make in the classroom and why teachers made those decisions. We piloted and iteratively refined this methodology over the course of three studies and use examples from these studies to illustrate the methodology. By closely observing and listening to teachers, we gain insights that allow us to continually refine and improve materials to ultimately improve the quality of classroom instruction.

Case studies of RTI's work on strengthening institutional capacity in Nepal, Jordan, Malawi, and Uganda focusing on three core functions: (1) setting and communicating expectations; (2) monitoring against expectations; (3) providing targeted support to struggling schools.

Existing research on the uptake of technologies for adult learning in the global South is often focused on the use of technology to reinforce in-person learning activities and too often involves an oversimplified “with or without” comparison (Gaible and Burns 2005, Slade et al. 2018). This MEL Framework for Technology-Supported Remote Training (MEL-Tech Framework) features a more nuanced perspective by introducing questions and indicators that look at whether the technology-supported training was designed based on a solid theory of learning; whether the technology was piloted; whether there was time allocated to fix bugs and improve functionality and user design; how much time was spent using the technology; and whether in-built features of the technology provided user feedback and metrics for evaluation. The framework presents minimum standards for the evaluation of technology-supported remote training, which, in turn, facilitates the development of an actionable evidence base for replication and scale-up. Rather than “just another theoretical framework” developed from a purely academic angle, or a framework stemming from a one-off training effort, this framework is based on guiding questions and proposed indicators that have been carefully investigated, tested, and used in five RTI monitoring and research efforts across the global South: Kyrgyz Republic, Liberia, Malawi, the Philippines, and Uganda (Pouezevara et al. 2021). Furthermore, the framework has been reviewed for clarity, practicality, and relevance by RTI experts on teacher professional development, policy systems and governance, MEL, and information and communications technology, and by several RTI project teams across Africa and Asia. RTI drew on several conceptual frameworks and theories of adult learning in the design of this framework. First, the underpinning theory of change for teacher learning was informed by the theory of planned behavior (Ajzen 1991), Guskey’s (2002) perspective on teacher change, and Clarke and Hollingsworth’s (2002) interconnected model of professional growth. Second, Kirkpatrick’s (2021) model for training evaluation helped determine many of the categories and domains of evaluation. However, this framework not only has guiding questions and indicators helpful for evaluating one-off training events focusing on participants’ reactions, learning, behavior, and results (as is the focus in Kirkpatrick’s model) but also includes guiding questions and indicators reflective of a “fit for purpose” investigation stage, a user needs assessment and testing stage, and long-term sustainability. Furthermore, this framework’s guiding questions and indicators consider participants’ attitudes and self-efficacy (based on the research underpinning the theory of planned behavior), as well as aspects of participants’ post-training, ongoing application and experimentation, and feedback (Clarke and Hollingsworth; Darling-Hammond et al. 2017; Guskey). Lastly, the framework integrates instructional design considerations regarding content, interaction, and participant feedback that are uniquely afforded by technology.

This case study examines a training targeting 18,000 head teachers and subject heads over seven days using IVR accompanied by electronic materials shared via WhatsApp conducted by the USAID-funded and RTI-implemented Malawi Early Grade Reading Improvement Activity (MERIT). While the training was conducted in June of 2020, this case study includes findings from a follow up study one conducted year later, to see what training content and skills participants retained, and the training elements that were most effective in their delivery. The case study applies a process-based and learning-oriented approach, drawing from the MEL-Tech Framework (Paper 1), to understand technology-supported remote teacher training introduced in response to COVID-19.

This report presents case studies of five remote training activities conducted by USAID-funded and RTI-implemented programs: the Advancing Basic Education Project (ABC+) in the Philippines, the Malawi Early Grade Reading Improvement Activity (MERIT), Read Liberia, Okuu Keremet! in the Kyrgyz Republic, and the Uganda Literacy Achievement and Retention Activity (LARA). The case studies seek to apply a more process-based and learning-oriented approach, drawing from the MEL-Tech Framework, to understand technology-supported remote teacher training introduced in response to COVID-19.

This MEL Framework for Technology-Supported Remote Training seeks to help education program implementers, governments, and program evaluators more effectively design, implement, and learn lessons from remote training activities. It places special consideration on the unique characteristics of technology-supported interventions in the global South. The framework presents minimum standards for the evaluation of technology-supported remote training, which, in turn, facilitates the development of an actionable evidence base for replication and scale-up. Rather than “just another theoretical framework” developed from a purely academic angle, or a framework stemming from a one-off training effort, this framework is based on guiding questions and proposed indicators that have been carefully investigated, tested, and used in five RTI monitoring and research efforts across the global South: Kyrgyz Republic, Liberia, Malawi, the Philippines, and Uganda (Pouezevara et al. 2021). Furthermore, the framework has been reviewed for clarity, practicality, and relevance by several RTI project teams across Africa and Asia.

The purpose of this session is to articulate a research framework that centers teachers’ voices when trying to understand how teachers use curriculum materials in the classroom. Operating in the context of highly structured lesson plans, the approach identifies ways in which teachers exercise their professional discretion to modify the lesson and frames conversations to elaborate the motivations driving the teachers’ choices. The approach has been iteratively refined across three studies; taken together, the studies provide evidence for the value of listening to teachers and being responsive to their voices during implementation. The research framework uses the lens of modifications, or changes to the intended lesson plan implemented within one class period. Modifications can be large or small, additive or subtractive. For example, in a lesson with a section for independent student practice, a large modification might be skipping the practice section entirely. Or, during a lesson focused on blending of initial sounds using 3 example words, a small modification might be extending the exercise by adding extra words. Researchers observe the lesson, noting any modifications; after the lesson, the researchers select some of the modifications and ask the teachers why they made the choices they did. Analyses of teachers’ explanations highlight the importance of understanding why teachers make the choices they do. For example, a teacher who skipped the independent practice section because they don’t think their students are ready to do the skill on their own suggests that the teacher is exercising agency and using her knowledge of her students to inform her decision-making. Insights such as this one can guide decisions on projects. It may be that while the intention of the teacher was guided by knowledge of students, the end result is not desirable from the project’s point of view. Understanding why the teacher made this choice provides implementers with better and targeted ways to address choices that impact the overall goals of the project. In this presentation, we draw on data from an exploratory case study to understand use of new mathematics materials in Liberia, a more in-depth case conducted in Malawi on teacher use of reading materials, and finally, a systematic study examining how reading teachers use materials across four Sub-Saharan African countries. We use each case to highlight both an aspect of the research framework and instances of modifications to project implementation driven by teachers’ voices. By focusing on teacher voices, we disrupt the deficit notion that teachers are “resistant” to change, or do not “understand” new pedagogies. Instead, we aim to value teacher voices and integrate their insight into implementation programs. By doing this, we not only raise the likelihood of successful use of new materials and pedagogies, but we also develop more responsive pedagogy that better matches existing classroom cultures.

Chapter 5 of the book Cultivating Dynamic Educators: Case Studies in Teacher Behavior Change in Africa and Asia. This chapter critically reviews the design, implementation, and evaluation of an attempt to study an exploratory short message service (SMS)–based intervention conducted under the auspices of the United States Agency for International Development’s (USAID’s) Malawi Early Grade Reading Activity (EGRA).1 The overall EGRA program, which was implemented from July 2013 to October 2016 in 1,614 schools across 11 educational districts, was designed to support Malawi’s Ministry of Education, Science, and Technology (MOEST) to improve reading outcomes in both Chichewa. and English languages among children in grades 1–3. The

This report presents key findings of the 2018 national assessment of early grade reading skills in Chichewa, a core language for reading and language development in Malawi. This early grade reading assessment (EGRA) for Standards 1 and 3 was conducted in June 2018 in a nationally representative sample of 86 Ministry of Education, Science and Technology (MoEST) primary schools. The 1,720 tested students—860 Standard 1 and 860 Standard 3 learners, half boys and half girls in each standard from each school—were randomly selected during visits to each school. The results are nationally representative of Standard 1 and 3 learners of MoEST primary schools in Malawi.