The National Rollout of Coaching with Tangerine in Kenya

This article was co-written by Scott Kipp, Sarah Pouezevara and Benjamin Piper. 

What do you picture when you think of technology improving education? 

If you read that question and started with an image of a child in front a screen, you’re not alone – that’s what most of us associate with ‘education technology’. It’s hard to avoid wondering what potential there is when a young and infinitely curious mind has the power of modern technology at their fingertips. But what if the child can’t read? Or what if the content on their screen just doesn’t make any sense? Teachers will always be an important mediator between technology and education. But what if their teacher also needs help with basic skills and pedagogy; who can they turn to? In Kenya’s more than 24,000 primary schools, the teachers receive support from an instructional coach, known formally as the Curriculum Support Officer. The coach has a tablet, and the tablet has a Tangerine® tool on it, designed just for them. 

Coaches across Kenya have been making regular visits to schools with their Tangerine tablets for more than 3 years now--and it’s working. It’s one of the only national-scale, technology-enhanced coaching programs in the world, and the impact of the education system it supports is clear. The Tangerine tools Kenyan coaches use aren’t magic and weren’t delivered from a lab in some far-away country; the software is effective because it has been developed with Kenyan educators over several years, beginning in 2012 with PRIMR, the USAID-supported Primary Math and Reading Initiative. Following on the evidence from PRIMR, the Kenyan Ministry of Education and RTI continued to improve coaching delivery under the National Tablets Programme (supported by USAID and DfID) and it is this work that carries on today under the USAID-supported Tusome activity. 

In a newly published article in the International Journal of Education and Development using Information and Communication Technologies, Piper et al. give us a close look under the hood of how Tangerine:Tutor has evolved in Kenya, and how the data collected as a coaching by-product is used for accountability and decision-making within the Kenyan Ministry of Education. 

What does Tangerine:Tutor do for coaches in Kenya?

In short, it helps them do their job of supporting teachers. When the PRIMR project began working with the Kenyan Ministry of Education in 2012, the teams could see that changing instructional practice required coaches who could master the new methodologies and materials being promoted so that they could support teachers to implement them effectively. This, of course, required being present in the classrooms regularly. Up until then, there had been more barriers than incentives to do so.

Starting small, the team developed a classroom observation checklist in Tangerine to help coaches see if teachers were deliverring lessons as expected and then give them the right pointers if they weren’t. The tablet coaching tools also included curriculum-aligned formative reading assessments the coaches adminster to a few random students so that teachers and the Ministry get a snapshot of learning progress on each visit. If needed, the coach can pull up a digitized version of the day’s lesson plan on the tablet to see how well the teacher’s following suggestions for reading and math instruction. As Tangerine is ‘offline-first’ software, all of this can be done offline by coaches, who upload the data from their school visits when Internet is available. The data included school name and a GPS location, making it easy to track which schools were being visited and where gaps remained.

Good coaching can be done without technology - why go digital?

Early on, the team tested the Tangerine:Tutor coaching approach in an experimental study to see if this model was cost effective compared with other technology used to support learning and teaching. When factoring cost into impact, Tangerine:Tutor came out ahead of the other technology interventions tested (e-readers and tablets for teachers), but its impact wasn’t statistically significant over the base coaching program as implemented without tablets. So, why bother? What can technology add?

That’s a key question to answer before expanding any technology program, and the authors in the IJEDICT article lay out the details for us, informed by a survey given to coaches, Ministry stakeholders and partners in Kenya:

• Reliability and Consistency: coaches reported that Tangerine helps to focus their classroom observation more efficiently, keeping their practice more consistent. Its prompts help them stay alert and engaged during the observation, have engaging feedback sessions, and better demonstrate appropriate teaching methods to a teacher, while also tracking teacher and student progress across time. Overall and most importantly, the coaches felt that the software improved their effectiveness as teacher supporters.

• Accountability: Tangerine data from coaching visits feeds into a national, interactive dashboard. As a first step, the Tangerine server checks the data’s validity. Once accepted as valid, the data provides county and national-level Ministry staff the means to answer essential questions about education service delivery:  Are coaching visits taking place where and as often as they should be? How are students and teachers progressing against learning targets? Are there gaps in achievement that need increased support and attention, or any particular schools or zones that are falling behind?

Though in theory all of this is achievable without technology, the details of the Kenyan example show that doing so at a national scale would require vast, intricate and costly efforts to verify and share consistent, reliable data for all the disparate groups of stakeholders involved. As the IJEDICT article discusses, the online Tangerine dashboard is used not only to inform higher-level decision-making, it’s also viewed by coach supervisors at the sub-county level, serving as a means for those directly overseeing coaches to trace progress and provide timely feedback. The digital advantage Tangerine brings to the coaching equation, as the authors note, serves to professionalize the role of coaches in Kenya--providing a tool that serves both to help coaches monitor their own impact while also being held to account by the public Ministry they serve. 

Impact

An external program evaluation (2017) showed that within one year, the Tusome reading programme doubled or tripled outcomes in literacy because of improved coaching. Moreover, the Tusome program had an effect size of .7 standard deviations in one year, which was larger than the impact experienced during the preceding USAID/PRIMR literacy program (Piper and Zuilkowski, 2015), the PRIMR mathematics program (Piper et al, 2016) or DFID/PRIMR pilot programs (Piper et al, 2018).

Behind the national impact, coaching was changing. The survey RTI conducted found that though 84% of County Directors had given coaches positive feedback based on their Tangerine data, and serious consequences were reported by 14% of coaches, indicating that in some counties, leadership had used the data generated by the tablets to push back on poorly performing coaches. These coach-reported findings are echoed by the responses given by County Directors, who reported on their survey that data provided in the dashboards affected supervisory visits (66%), methods to address student performance (31%), and decisions to reassign or fire coaches (36%). About half of Country Directors reported giving negative or positive feedback based on dashboard results.

What’s emerged out of the evolution of Tangerine and coaching in Kenya is that when well-trained and well-equipped coaches do their jobs regularly and consistently, clear and reliable results are achieved. The survey data show that coaches and their supervisors within the Ministry have developed a shared understanding of what their respective responsibilities: coaches know how the Ministry will monitor their work and they have the tools in their hands to get their job done, a simple recipe for giving the coaches a clear path to success. The Ministry of Education’s enthusiasm for the tablet model as an effective way to create accountability is demonstrated by their decisions to expand the tablet model to other civil servant roles in the education sector.

Improvement & Iteration

The research discussed above is just one step in what’s been Tangerine’s process of continuous improvement using human-centered design approaches. Since national rollout of Tangerine began in 2015 in Kenya, the software has been improved and updated every term based on user feedback and requests to include additional features. More recently, with support from partners, the team has used Design Sprints with Kenyan coaches and Tusome staff to chart out how Tangerine could continue to improve and better serve coaches. In these sprints, coaches and Ministry staff work closely with RTI and engineers to decide on priority features. We then quickly begin prototyping, re-designing and iterating through rapid user tests. As a result of these sprints, several goals are now driving Tangerine redevelopment:

• To improve the case management features that allow coaches to manage their classroom visit schedule, ensure equal coverage, and better monitor trends by school and in their zone. 

• To include additional opportunities to capture audio and video clips during the lesson observation to use during discussion with teachers, and links to related resources for improving mastery. 

• To improve the usability of the existing dashboard and to make more data available for the project and the MOE.

Across all of this work, our goal is to make sure that what Tangerine delivers for coaches and Ministry partners in Kenya can be shared elsewhere easily--something we are now starting to test as Tangerine:Tutor branches out into Uganda, Malawi, Sierra Leone and Jordan this year. 

Lessons learned

Over the 5-year collaborative journey around Tangerine’s development and use in Kenya, we’ve learned a fair amount about what proved valuable for success: 

• We have learned that it is essential that the tool work. This is obvious, but it needs to work for a typical education officer in an offline, accessible location and with limited guidance. To be ready for national scale, it must work consistently across the range of settings that might occur in a large and diverse country.

• We find that consistent and rapid access to reliable data, in settings that have previously had little available, can be transformative. The Tusome program’s consistently available data has revolutionized Kenyan education officers’ view of what is possible, and having each instructional coach be aware that their data will be consistently reviewed has changed performance and activity.

• We also know that the incentive system matters quite a bit. RTI’s work in the region has tried several mechanisms for providing incentives to coaches. The solution utilized in Kenya is low-cost and actually affordable by the Kenyan government within the rates already provided by government, but done in a consistent and reliable way that coaches see as being closely related to their daily classroom support.

• This study shows that it is possible to have light touch ICT systems work at large scale, but that we should not discount the time spent from 2013 to 2015 in piloting various approaches and responding to user preferences and learning outcomes data. The program was successful in no small part because we had the opportunity to test many options, and only then was the larger decision made to settle on scaling up the option that was cheapest, most impactful, and most tightly related to the behavior change that was essential to improve reading outcomes.

To keep up with the latest on Tangerine and try it out for yourself please visit: www.tangerinecentral.org