Tusome External Evaluation Midline Report

Management Systems International (MSI) led the Tusome baseline study using multiple data collection methods, including an early grade reading assessment (EGRA); surveys of pupils, teachers, head teachers, curriculum support officers (CSO) and households; and classroom observation. The EGRA assessment tool was developed during the baseline and includes eight subtasks in English and six subtasks in Kiswahili. The midline included developing additional data collection tools, revising the baseline surveys, recruiting and training supervisors and enumerators, administering the tool and surveys in the same sample schools as the baseline, ensuring quality control, establishing the reliability of the assessment tool and analyzing the data. For the midline, the evaluation team assessed pupils from the same 204 schools sampled for the baseline. Through discussions with USAID, MOE and RTI, the evaluation team created the sampling frameworks and set up the design for a national sample in 2015. Using a three-stage cluster sampling procedure from a sampling frame of 22,154 formal public schools and 1,000 non-formal (or Alternative Provision of Basic Education and Training – APBET) schools, the evaluation team drew a clustered, random sample, resulting in a target of 4,896 total pupils comprising 2,448 boys and 2,448 girls divided equally between Class 1 and Class 2. The evaluation team reached the following conclusions: • The Tusome approach is having a strong, positive influence on reading outcomes, with relationships between project implementation and reading outcomes. • Reading outcomes for Class 1 and 2 pupils greatly improved during the one-year period between the baseline and midline evaluations. While impressive gains have been made, continuing with the Tusome approach will be critical to sustaining or improving on those gains. The Tusome project has achieved a high level of national implementation of the activities at each level of the education system. Given that project activities such as CSO observations, in-service training and access to materials are associated with higher ORF scores, the high level of implementation across all schools appears to be a key part of its success. The effect sizes seen during the PRIMR pilot have been at least sustained, and in most cases strengthened, in the national scale-up of Tusome. • The evaluation methodology and implementation resulted in valid, reliable data for the midline evaluation, including the changes from baseline to midline.