Evaluation

Presentation delivered at CIES 2017 (Atlanta). CIES Panel title: Measuring regional progress toward early grade reading and mathematics targets in a national application of LQAS as part of the RAMP Initiative in Jordan. RTI International (RTI) and its partners are implementing the United States Agency for International Development (USAID) and United Kingdom’s Agency for International Development (UKAID) funded Early Grade Reading and Mathematics Initiative (RAMP). RAMP is a nationwide initiative of the Jordanian Ministry of Education (MoE) designed to improve the reading and mathematics skills of students in kindergarten 2 through grade 3 (K2–G3). RAMP, which will be carried out over five years, expects to deliver improved reading and mathematics instruction to all public school students in Jordan in grades K2–G3—about 400,000 students. Central to the RAMP approach is developing reflective practice at all levels of the education system. At the classroom level, teachers use screening and diagnostic tools developed by the initiative for reading and mathematics to gain a better sense of the developmental level(s) of the students in their class and in response, to use research based pedagogical approaches and materials to respond to these needs. At the national level the MoE uses biannual national surveys that include the Early Grade Reading Assessment (EGRA) and Early Grade Mathematics Assessment (EGMA) to measure progress toward national benchmarks. RAMP is exploring the efficiency of the Lot Quality Assurance Sampling (LQAS) methodology in providing regional structures (Field Directorates) with regular information on the progress by schools and Field Directorates toward achieving the national benchmarks for early grade reading and mathematics. In addition, the LQAS based assessments will also contribute to the development of school reports. This panel will discuss the first national implementation of the LQAS based assessment conducted in May 2016 (Brombacher, 2016). The first presentation will provide background to the RAMP initiative, the centrality of reflective practice at all levels of the system to improving performance in early grade reading and mathematics, and the role that the LQAS based assessments will play in providing regional structures with the information that they need to monitor and support progress toward the national benchmarks. The second presentation will describe the development of the instruments used in the LQAS assessment activity, the implementation of the first national LQAS based assessment, the findings of the study, and a range of important lessons learned. The third and final presentation will describe the response of the MoE to the national implementation. The presentation will also deal with the anticipated institutionalization of the approach into the roles and responsibilities of the ministry supervisors. The panel contributes to the conference theme, “Problematizing (In)Equality: The Promise of Comparative and International Education”, as it examines the implementation of an efficient and cost-effective mechanism for national and regional education structures to identify the schools and regions that are in greatest need of support – allowing the structures to deploy limited resources in a cost effective way.

Presentation delivered at CIES 2017 (Atlanta). The Tayari intervention’s randomized controlled trial design is structured to allow for causal inference of Tayari’s impact on school readiness. Previous research has shown how similar ECD programs have affected learning outcomes, but the literature remains silent on how individual children’s skills transition over time. The Tayari longitudinal research design allows us to estimate growth trajectories of individual children. This is particularly salient as the literature lacks models for how literacy and numeracy skills interact with the executive functioning and socioemotional skills that Tayari investigates interact with each other over time. The Tayari longitudinal study follows more than 3100 learners across three rounds of data collection and a wider range of sills than is available in the Tayari impact evaluation. Given how little is known about how children’s core ECD skills grow, the Tayari longitudinal intervention estimates first how the skills grow in the normal control condition, and then how the Tayari program affects growth rates and relationships between learning elements. Finally, the Tayari longitudinal study will continue to develop an understanding of how children transition skills between the two levels of pre-primary in Kenya to the primary education sector where the Tusome literacy program is being implemented at national scale.

Workshop delivered at CIES 2017 (Atlanta). The workshop will begin with introducing participants to accessible online educational survey data from EGRA and EGMA. Participants will be introduced to two different early grade reading surveys and the data sets associated with them. They will be given the opportunity to review the original research questions, sample methodology, and instruments in order to understand the context of the survey. Participants will then explore the data to better acquaint themselves with the data’s: structure, content, abilities, and limitations.

Presentation delivered at CIES 2017 (Atlanta). A systematic way to use baseline evaluation studies to help define and identify disadvantaged schools in an intervention program.

Presentation delivered at CIES 2017 (Atlanta).

Ce rapport est axé sur les leçons pratiques tirées de la mise en oeuvre de ces évaluations qui pourraient s’avérer utiles pour d’autres évaluations en RDC et ailleurs.

L’objectif de l’étude est d’avoir une évaluation de référence des compétences en lecture et en mathématiques des élèves de 3e et 5e années d’école primaire en République démocratique du Congo (RDC). Plus concrètement, cette étude offre à l’Agence américaine pour le développement international (USAID) et le Gouvernement de la République démocratique du Congo (GRDC) des données portant sur les capacités des enfants et 3e année d’école primaire à lire dans les trois langues nationales de la RDC (notamment le lingala, le kiswahili ou le tshiluba) et sur les compétences de compréhension et de production orales en français des mêmes élèves. D’autres éléments de l’étude offrent des aperçus des compétences en mathématiques d’enfants en 3e année dans un des districts du Katanga. La présente étude offre aussi un certain contexte et des données portant sur les variables principales en rapport avec les élèves, les enseignants, les directeurs des écoles et les écoles, variables supposées être de bons prédicteurs de la performance des élèves de primaire en lecture et mathématiques. Ces informations diagnostiques aideront les décisionnaires à prendre des décisions éclairées par les données, décisions sur les stratégies correctives qui pourront améliorer la qualité des résultats de l’enseignement et de l’apprentissage des élèves dans les premières années de l’école primaire.

This report presents key findings of the 2nd national assessment of Grade 3 reading skills in primary schools of the Ministry of Education (MOE) in Egypt. This mid-line assessment reports early progress and student achievements after three years’ implementation of the MOE Early Grade Reading Program nationwide, albeit only one year at Grade 3 level. The MOE program, initiated in Grade 1 during the 2011/12 school year, added enhanced reading instruction to Grade 2 in 2012/13 and then to Grade 3 in 2013/14. This 2nd national assessment measures the improvement in Arabic reading proficiency of this first cohort of MOE students to receive successive years of improved reading instruction in Grades 1-3. The findings of this mid-line assessment provide timely results and empirical measures of the initial progress and reading outcomes of the MOE’s Reading Program since the national Early Grade Reading Assessment (EGRA) baseline for Grade 3 in 2013.

This analysis report for the reading portion of the Grade 2 NAS presents the results for pupil performance on the EGRA in seven local languages, each of which also had a small set of subtasks in English. Additional information about schools, teachers, head teachers, and pupils’ socioeconomic background that was collected alongside the EGRA provides context for many of the results. For grade 2, a nationally representative sample of pupils was selected to take locally developed versions of the EGRA and the Early Grade Mathematics Assessment (EGMA), which together comprised the Grade 2 National Assessment Survey (NAS). Outcomes on the EGMA portion of the survey, sponsored by the British Department for International Development (DFID), are documented in a separate findings report.