Tablet programs are extremely popular among beneficiaries. And we get it. When a grant opportunity came up to provide tablets and training to Coordinating Center Tutors (CCTs)[1], the USAID/Uganda School Health and Reading Program jumped. We had been working with CCTs since 2013 and had a strong process in place involving literacy-focused classroom observation tools and a system of joint support supervision designed to build capacity of the CCTs while supporting teachers at the schools. Yet there were still challenges getting the CCTs to the schools and ensuring that the support they provided was meaningful and specific enough for teachers to act upon. We needed to try something new.
Process
In 2017, we introduced technology-enhanced teacher coaching using Tangerine:Coach with the dual aim of increasing the quantity and quality of CCT school visits. Several features were expected to have an effect on quality and quantity of coaching visits:
- app-generated instructional feedback based on observation tool inputs would improve the consistency and specificity of the coach feedback to teachers;
- case management tools would make it easier for CCTs to see which schools had been visited and which had not;
- a web-based dashboard available to district- and central-level officials would introduce accountability mechanisms that would encourage CCTs to keep to their schedule of visits. [2]
The idea can be illustrated as follows:
The tablet program was rolled out to all 154 SHRP CCTs in February, 2018 after a period of piloting and iterative redesign with user feedback in 2017. The tool we created used Yes/No questions for observable behaviors, making it easy to use (e.g., Did the teacher guide learners to read silently when the methodology called for them to do so?). We worked with ICT staff based at the colleges to provide technical support and troubleshooting for the tablets, communicating over a WhatsApp group so that ICT staff could answer each others questions and learn from each other. We worked with the Ministry of Education and Sports (MoES) and local government to design a quick-reference dashboard with the information they most wanted. Lastly, we incorporated our research questions on the frequency and quality of CCT visits in ongoing, routine monitoring to provide fast, actionable data at no extra cost.
Findings
What did we find? Feel good stories? For sure. The average age of the CCTs was over 50 years and many had never used a touch screen before. One told us that they knew students in neighboring Rwanda had this kind of technology, but they never dreamed it for themselves. They also reported feeling more professional and more respected when they used the tablets. Their enthusiasm alone could have been enough to call the program a success.
But what about the quantity and quality of school visits?
Comparing baseline data collected in Term 1, 2017 (February to April, 2017) to Term 1, 2019[3] we found virtually no difference in the percent of schools visited by a CCT in that 3 month period (the quantity)--41% of schools had been visited by a CCT before and after the introduction of technology. We knew from the onset that the tablets alone could not actually get the CCTs to the schools (they needed other often unavailable means to do this), but we hoped that they would provide some additional incentive to work within the system. Moreover, the dashboard, which appeared to be a key accountability mechanism contributing to increased school visits in Kenya only became operational at the very end of the program.
However, the quality of visits increased! Again, comparing Term 1, 2017 to Term 1, 2019 we found that CCTs were more likely to leave written feedback for teachers at the schools and this feedback focused more specifically on teaching and learning and reading. At baseline in 2017, 22% of CCTs left written feedback for the teachers in the “blue book”; in 2019, this figure had doubled to 44%.[4] The percent of general feedback (not directly related to teaching and learning) decreased from 41% at baseline to 17%. Reading specific guidance increased from 16% to 30%.
Feel good? Yes. Effective? Partially. Promising? Definitely.
This piece was co-written by Rachel Jordan.
The full impact study report is available here.
[1] A CCT is employed by the local Primary Teacher College. Part of their role is to provide support to schools and teachers in their catchment area through regular schools visits.
[2] The model was similar to that of USAID/Tusome in Kenya. For more information see here and here.
[3] Source of data: SHRP Field Assistant monitoring data from school visit review of records (visitor’s book and blue books).
[4] Blue book: Is a hardcover blue notebook provided to every teacher for recording reading related supported activities. It is used in lieu of paper feedback forms since reproduction is cost prohibitive. When a head teacher or CCT provides support to the teacher, this support (along with recommended actions) should be recorded in the blue book.
