Much donor effort has gone into developing evidence-based early grade reading (EGR) programs. Now donors are striving to scale them up. RTI has been exploring these issues with support from USAID since 2012 and recently published this review of scale up efforts in seven countries. In our analysis, we found that as these scale-up efforts proceed, many are encountering implementation fidelity problems: the EGR programs are not being implemented as well as they were during the pilot that proved that specific blends of various inputs produced a set of significant learning outcomes. Implementation fidelity problems are manifest when, for example, teachers are not trained with the same intensity and/or coaches do not coach as often as when these occurred during the piloted evidence-based EGR program. This implementation fidelity problem arises largely because as scale-up occurs, most efforts must rely more and more on government actors, implementers over whom these efforts have little control. When government actors do not implement in the manner set by the evidence-based program, it can be attributed largely to 3 factors: 1) they may not know how to do what they are being asked to do, 2) they may not be able to do it, or 3) they may not choose to do it.
When government actors do not implement in the manner set by the evidence-based program, it can be attributed largely to 3 factors: 1) they may not know how to do what they are being asked to do, 2) they may not be able to do it, or 3) they may not choose to do it.
The first factor can be addressed through better and/or more training: more tightly coupled training-of-trainer cascades, or follow-up training, or through more coaching. We realize that by offering follow-up training, or more coaching, implementation fidelity is not being realized, but in these instances, one is doing more than what was done during the pilot, so this does not present itself as an implementation fidelity problem. The second factor requires a mapping exercise that identifies the institutional, systemic, and organizational gaps and barriers that prevent government staff from doing what is required of the scale-up effort. Once identified, measures must then be taken to fill those gaps and/or remove the barriers. For example, an EGR program may require teachers to be coached 4 times every semester. However, the coaches’ job description may force them to do a lot more than this, leaving them insufficient time to coach the required amount. For government coaches to do the required amount of coaching, measures must be taken to address this barrier. The third factor requires working with the government to put in place institutional forces that serve to motivate staff to do the work. These may include career ladders premised in part on performance tied to the work, incentives, and accountability mechanisms.
As EGR programs go to scale, attention must also focus on sustainability: enabling the government to sustain the EGR program once taken to scale. But what exactly does this entail? The government must be able to do much of what an effort did to develop and implement the evidence-based EGR program and take it to scale. We offer the following capabilities as an example:
-
Design EGR curriculum materials that are aligned with a best practice EGR curriculum learning standards and benchmarks (standards/benchmarks that the project might have helped the government put in place),
-
Ensure that these materials get to all the end users in time for use during the school year
-
Develop and implement teacher training programs/materials that are aligned with the curriculum learning standards and curriculum material
-
Design and implement classroom observation tools that support teachers using the curriculum materials; tools that help teachers to do what they have been trained to do and to help them to achieve the learning standards put forth in the curriculum
-
Design and implement a coaching system (training coaches/supervisors, organizing a set number of coaching visits for every school/teacher, organizing how classroom observations are conducted) that provides ample quality pedagogical support to teachers
-
Gather, analyze, and use classroom observation data to ensure that teachers/schools in need get targeted support
-
Gather and analyze classroom observation data to identify needs across several teachers, and use that data to develop teacher training programs/materials that address those needs; then train those teachers
-
Design and use student assessment instruments that examine how well the curriculum learning standards are being met
-
Use this student assessment data to provide targeted support to schools and to reflect on how effectively the scale-up effort is performing; to the extent that it is not performing as well as it should, take measures to rectify the problem
-
Mobilize communities and parents to support EGR
-
Apply “force” throughout the project to ensure that project actors do what they are supposed to do
-
Perform A-K in a coherent and coordinated manner
-
Pay for much of this work to be done
When one reflects on all of this, one can say that an EGR intervention and scale-up effort model an effective Core Functions (Crouch and DeStefano, 2015) curriculum implementation system. Granted, the effort may work through some government systems, but unless measures are taken to ensure that the government can do what the project is doing—operate a CoreFunctions curriculum implementation system, sustainability will not likely happen.
Unless measures are taken to ensure that the government can do what the project is doing—operate a Core Functions curriculum implementation system, sustainability will not likely happen.
So, if the government is to sustain an EGR program, it pretty much must be able to do A-M above. When faced with the task of systems reform one can be overwhelmed by the task; after all, an education system is a seemingly endless array of nodes and linkages. To simplify the task, especially as it relates to high-quality teaching and learning, one can take the Core Functions approach. The core functions of a viable teaching and learning system are:
-
Establishing and communicating learning and learning-related performance expectations or standards
-
Supporting people to meet those expectations/standards
-
Gathering/using data to determine the degree to which those expectations/standards are being met
-
Holding people accountable for supporting people to meet those expectations/standards and for meeting those expectations/standards
The expectations/standards should largely be about learning: what children need to learn/do to be able to read with comprehension. Expectations can also be about the performance of people within the system to help those learning standards be met (i.e., a 10% increase in the number students who meet learning standards). These standards are to be communicated throughout the system such that all actors know what is expected of them. Moreover, these curriculum standards should also be communicated through curriculum materials, teacher training programs and materials, coaching, coaching tools, student assessment instruments, and M&E systems. Put another way, these things must all be aligned with the curriculum learning standards. When they are so aligned, and when all actors in this Core Functions EGR curriculum implementation system are working to help children meet the learning standards put forth in the curriculum, the system is said to be learning coherent (Pritchett, 2016).
To transform a government’s existing and likely somewhat dysfunctional curriculum implementation system into a Core Functions curriculum implementation system, we suggest a highly participative set of exercises around the following:
-
generate widespread demand among government decision makers and actors to sustain EGR
-
review what the scale-up project is doing (generate a list like A through M above) and ask government staff and stakeholders to identify the elements of that list they believe are critical for them to sustain EGR once taken to scale
-
given this list of critical elements, support government actors to assess their current institutional, systemic, organizational, and behavioral/attitudinal capacity to do each one
-
to the extent that they cannot, support the government to understand why: do government staff not know how to carry out certain functions; are government staff unable to perform certain functions (are there structural gaps/barriers that prevent them from doing what needs to be done); are there not enough institutional forces in place to ensure that people carry out certain functions (note that the factors that give rise to the implementation fidelity problem are also relevant for sustainability)
-
support the government to address these shortcomings
Metaphorically speaking, one could say that the best practice curriculum materials, teacher training programs, coaching templates/tool, data systems, and student assessment instruments that an EGR program develops are finally crafted square pegs, and that the donor-sponsored scale-up effort creates the “square holed system” within which these square pegs can be utilized as intended. If the exiting and oftentimes dysfunctional government curriculum implementation system can be likened to a set of corresponding round holes, then sustainability is about a) transforming that round-holed system into a square-holed system, and b) institutionalizing those square pegs.
Browse other RTI resources on scale and sustainability here.
Read other pieces by Hank Healey here.
