Social network analysis methods for international development: Applications in higher education development. Social Network Analysis (SNA) is a promising yet underutilized tool in the international development field. SNA entails collecting and analyzing data to characterize and visualize social networks, where nodes represent network members and edges connecting nodes represent relationships or exchanges among them. SNA can help both researchers and practitioners understand the social, political, and economic relational dynamics at the heart of international development programming. It can inform program design, monitoring, and evaluation to answer questions related to where people get information; with whom goods and services are exchanged; who people value, trust, or respect; who has power and influence and who is excluded; and how these dynamics change over time. This brief advances the case for use of SNA in international development, outlines general approaches, and discusses two recently conducted case studies in higher education development that illustrate its potential. It concludes with recommendations for how to increase SNA use in international development. Key research recommendations advanced by this paper include: • Incorporate SNA into monitoring, evaluation, and learning processes. SNA can be conducted at various points of a project to inform program design, adaptive management, learning, and evaluation by considering network structure and network changes over time. • Demystify the use of SNA. Increased use of SNA tools and clear presentation in widely read publications are needed to bring the analytic approach into the mainstream of international development. • Build capacity to conduct SNA. The capacity to conduct and interpret SNA is lacking across actors in international development. Efforts by some organizations to build capacity in the community are well noted and should be built upon. • Build understanding of relationships between social networks and development outcomes. SNA will be useful only to the extent it helps users understand the relationship between networks and development outcomes that matter. • Establish norms for data collection and identity protection. Data about individuals and their interactions with others are inherently sensitive data. As a part of standard research ethics protocols, SNA practitioners must make carefully considered decisions about how or if to anonymize data when reporting it.

Title: Right-fit EdTech: Leveraging Loquat and other (machine) learning to support foundational learning in LMICs Abstract The COVID-19 pandemic has severely disrupted global education systems: according to the Global Education Monitoring Team, over 1.6 billion students have lost significant instructional time, with many yet to return to school. But even before the pandemic struck, the state of education around the world was so poor that the World Bank was decrying a “learning crisis.” To address the crisis, the Bank called for both a renewed emphasis on what teachers actually do in a classroom and the strategic deployment of technology to improve teaching and learning. During the pandemic, remote-first instruction became a feature of basic, secondary, and tertiary education systems in many high-income countries. [European Commission, Rickles et al. 2020, Hodges et al. 2020, Gillis & Krull 2020] Coverage was not universal, as wealth and income inequality within high-income countries remains a persistent challenge. [Romm 2020, Blaskó & Schnepf 2020] Nonetheless, the market for educational technology firms boomed during the pandemic, with investment in the sector expected to exceed $80 billion by the end of the decade. [Gillespie 2021] The growth in remote schooling, while temporary, coincides with a long-standing and ongoing interest in so-called “artificial intelligence” or “machine learning” technologies and their applications in business contexts. [Sestino & de Mauro 2021, Benaich & Hogarth 2018, 2019, 2020, 2021] An increasing number of so-called “EdTech” (education technology) companies are riding this wave to attain “unicorn” status (a market valuation in excess of $1 billion), often touting their use of cutting-edge machine learning techniques to support their products or deliver their services. In many lower- and middle-income country (LMIC) contexts, transitions to remote learning were uneven, with significant variance in access, quality, and coverage. [World Bank 2020, UNICEF 2020] Indeed, in many LMICs even the most basic of modern technologies can be unavailable: as of 2015, up to 90 million children in Sub-Saharan Africa were studying in classrooms that lacked electricity. What is the relevance to these education systems of analytical techniques developed in contexts where computing devices are plentiful? This presentation will answer that question in two ways. First, by introducing Loquat, a novel application from RTI International that leverages automatic speech detection and the increasing ubiquity of low-end smartphones to give teachers accessible, affordable instructional coaching. Using machine learning, Loquat detects and classifies verbal interactions between teachers and students. Automated analyses translate these data into simple but powerful visualizations that, combined with guided reflection, provide tailored, actionable feedback teachers can use to understand their talk management and facilitate its improvement. Second, the presentation will review recent advances in machine learning and so-called “artificial intelligence”, detail their potential relevance to lower- and middle-income country education systems, and discuss how programs that aim to improve foundational learning outcomes could begin to leverage these powerful new tools.

CIES2021 Presentation on the use of Tangerine:Coach to provide support to teachers around the world.

This presentation was held by Carmen Strigel during the second webinar of the Basic Education Coalition EdTech working group on April 27, 2020. The presentation is about using Tangerine for student self-study and self-assessment as well as family outreach. The presentation also introduces a new tool developed by RTI on considering access, user engagement, and content in making the right technology choices for your audience.

Presentation for the 2019 ICERI conference (Seville, Spain.)

The acquisition and use of so-called “soft skills”, including problem solving, resilience, and self-regulation have been associated with better performance at school and in the workplace [1], [2]. Problem-solving is defined as the ability to acquire or use prior knowledge in order to solve new problems. Strengthening this skill is a concern to educators and employers as the 21st century labor market is increasingly unpredictable and requires skills that go beyond mastering and executing familiar processes. Students need to identify and solve problems that they have never encountered before, formulate a solution plan specific to that problem, and execute the plan. Thus far, the body of research that has measured these relationships relies on traditional self-reporting measurement questionnaires. This methodology is prone to bias since youth may respond in a way they know is desirable, rather than the way they actually behave [3]. Stealth assessment attempts to gather more authentic measurement of skills by asking children to demonstrate them in a structured environment where data collection is unobtrusive [4]. Digital games can be used for stealth assessment, with data on decisions and strategies collected in the application during game play. Since 2017, RTI has been developing games that target a range of soft skills by simulating real-world tasks in a virtual environment. The game designed to measure problem-solving skills gathers metrics on task completion, time management, accepting instruction, problem identification, solution identification, and self-regulation. This paper describes the multi-year process of development and testing of this game, the results obtained from pilots in the Philippines and Morocco, and the implications for strengthening problem-solving skills among youth worldwide. Cite this paper: Pouezevara, S., Powers, S. Strigel, C., McKnight, K. (2019). Assessing soft skills in youth through digital games. ICERI2019 Proceedings. 12th Annual International Conference on Education, Research and Innovation (ICERI), Seville, Spain. p. 3057-3066. https://doi.org/10.21125/iceri.2019.0774

The traditional face-to-face (FTF) survey method cannot accommodate the privacy needed to mitigate the effect of social-desirability bias, particularly with the most sensitive topics such as corporal punishment and sexual violence, nor does it provide a means to elicit authentic responses. Indeed, an assessor asking a respondent questions about their experiences of violence will contribute to the stress of taking such a survey. However, audio computer-assisted self-interview (ACASI) does hold promise in addressing this issue with survey administration. This brief provides an overview of ACASI, discusses a 2019 large-scale study that compared the ACASI and FTF administration methods, and provides data for discussion regarding ACASI’s viability as a more effective method of survey administration when collecting data on experiences of SRGBV.

This report describes the results of a program designed to expand use of Tangerine:Tutor (now known as Tangerine:Coach), a model scaled successfully in Kenya, to Ugandan coaches. The first aim of the program was to improve the quality of interactions between Coordinating Centre Tutors (CCTs, or “coaches” for convenience) and teachers, as well as the quantity of those interactions (increase the frequency of school visits). During this pilot effort, which lasted approximately 18 months, we studied the added value of a digital case management tool and job aids to improve coaching activities in two Early Grade Reading (EGR) programs in Uganda. The iterative, user-centered design and monitoring focused on changes in the quality and quantity of coach and teacher interactions.The second aim was to improve the quality of communication between CCTs and Teacher Training Colleges (TTCs), between CCTs and districts and between local stakeholders and institutions (i.e., schools, TTCs and district offices) and the Ministry of Education and Sports (MoES). This aim was to be accomplished through a Web-accessible dashboard based on the digital tools that quickly communicated school support coverage, as well as teacher and learner attendance. The pilot effort successfully introduced the case management tool and job aids, built a dashboard to communicate progress and trained users across four regions of Uganda.

This presentation was delivered to a team of researchers who participated in a "Data Deep Dive" convened by the XPRIZE Foundation after the announcement of the Global Learning XPRIZE award.

Executive functions are a cognitive skill set that underlie our goal-directed, planning, and problem solving behavior, and include the components of working memory, inhibitory control, and cognitive flexibility. EF skills undergo the majority of development during the pre-primary years of a child’s life and have been shown to contribute to academic success. However, most of our knowledge about children’s EF skills have been based on research with children living in high-income countries. This presentation reports on findings from the administration of an EF assessment with children from a West African country. Students entering Primary 1 grade for the first time from Kindergarten class were sampled. All students were administered four pre-literacy tasks and a set of questions measuring socio-economic status. Half of the sample also received EF touch games, including two training modules, two tasks measuring inhibitory control and 1 task measuring working memory. The presentation will focus on the findings of the use of EF Touch with this sample of children from a West African country. First, a brief description of the process of adapting and revising the tools for use in Liberia is reported. Second, a descriptive analysis is presented in order to describe the feasibility of using EF Touch with young children in this context. Third, children’s performance on the three tasks is summarized and correlations among the scores on the three tasks is reported and discussed. Fourth, a model exploring the unique contributions of simple reaction time and demographic characteristics is presented. Finally, the overall contribution to the field of early childhood assessment and executive function measurement in LMICs is discussed.