Extensive evaluation of a four-year DFID funded WASH programme reveals lessons for future large-scale programmes.
Delivering WASH at scale – lessons from the WASH Results Programme. Photo by Kelly Lacey from pexel.com
The WASH Results Programme was a large-scale programme undertaken as part of the UK Government pledge to ensure that 60 million people would gain access to water, sanitation and hygiene (WASH) services by December 2015 as a result of UK investments.
The Department for International Development (DFID) funded WASH Results Programme worked in 11 countries, and aimed to provide 1.2 million people with access to clean drinking water sources, 4.3 million people with access to an improved sanitation facility, and reach 9.3 million people with hygiene promotion activities. By most measures the programme was very successful in achieving its stated objectives. The vast majority of output-level targets were achieved by all suppliers, and in many areas with significant overachievement. The suppliers also overwhelmingly achieved the outcome-level targets, with significant overachievement in several areas and modest underachievement in relatively few others.
The WASH Results Programme had two defining characteristics: the fact that it was a Payment-by-Results (PbR) programme, and its size and structure. The design and effects of the PbR modality are explored in earlier blog posts.
This blog post, based on a paper by Richard Carter and Jeremy Colin, draws on the mid-term and end-point evaluations of the WASH Results Programme to reflect on a number of aspects of working at scale in the WASH sector. In particular it explores:
- How the WASH Results Programme and other WASH sector actors have conceptualised scale.
- The outcomes and impacts set out in the WASH Results Programme design, and the extent of their achievement.
- What requirements were placed on suppliers, in the pursuit of programme outcomes and impacts at scale.
- How inclusive and sustainable services and behaviour changes at scale can best be pursued in future WASH programmes.
This blog post is intended for governments, international agencies, NGOs and development funding partners which wish to bring about beneficial, inclusive and sustainable changes in water, sanitation and hygiene at scale. The evaluation that this post is based on is available from the DFID Research Portal as a Synthesis Report, Executive Summary and Annexes.
Aspects of working at scale in WASH sector
The wider literature on scale, within the WASH (eg Taylor, 2013[i]), health (eg Mangham and Hanson, 2010[ii]; Gargani and McLean, 2017[iii]), education (eg Coburn, 2003[iv]; Burns, 2014[v]) and food security (Frake and Messina, 2018[vi]) sectors, understandably focuses more on “scaling-up” than on scale per se. The two ideas are linked, however, in the question “assuming we know how to deliver services at small scale, how can those services be extended or accelerated to reach large numbers?” Cynthia Coburn provides a helpful four-dimensional understanding of scale.
| Understanding “scale” (Coburn, 2003) |
| Coburn (writing about the education sector) highlights four necessary dimensions of scale:
1. Depth. The nature and quality of services, “change that goes beyond surface structures or procedures to alter … beliefs … norms … and principles (as enacted in the curriculum)”.
2. Sustainability. “The distribution and adoption of an innovation are only significant if its use can be sustained … there is ample evidence that sustainability may be the central challenge of bringing reforms to scale”.
3. Spread. “ … the importance of taking into consideration what is spread … but also … influencing district policies, procedures and professional development”.
4. Shift in reform ownership. “ … ownership over the reform must shift so that it is no longer an external reform, controlled by a reformer, but rather becomes an internal reform …”. |
The significance of ‘scale’ in WASH programming has different implications in each of the sub-sectors. For example water services are more likely to be financially sustainable if the customer base is large and diverse, so that wealthier market segments (enjoying high levels of service) can subsidise less wealthy segments with more basic services. In regard to sanitation, beneficial health impacts are only likely to emerge as high levels of actual use are realised – reducing open defecation and other forms of unsafe disposal of human excreta to the minimum. Hygiene practices too need to reach high levels of compliance in order to provide a basis for health impacts.
Linked to all of the individual sub-sectors is the significance of the programme’s overall scale with regards to the geographical and administrative boundaries in which it is operating. Administrative boundaries are particularly significant with regards to achieving sustainability and capacity building aims, and programme’s that align themselves with the administrative levels that have responsibility for maintaining services enhance the prospects for sustainability. In the rural context this often entails taking a ‘district-wide approach’ to delivering services, and in the urban context focusing on city-wide service delivery. This view of scale is also significant with regards to equity as by aiming to reach everyone with services in a given area, by definition, implies a focus on equitable service delivery.
Scale in the WASH Results Programme
Scale was defined in terms of beneficiary numbers
In the WASH Results Programme, DFID called for suppliers which could deliver results at scale. No explicit definition of “scale” was provided, although the programme aimed to reach 1.2 million people with access to clean drinking water sources, 4.3 million people with access to an improved sanitation facility, and 9.3 million people with hygiene promotion activities. “Scale” was effectively defined therefore in terms of beneficiary numbers.
In the WASH Results Programme, the suppliers did not formally define scale, but they pursued it in different ways, some working across multiple countries and districts, partially filling the gaps left by other government and non-government agencies; and others working in more focused locations (e.g., in certain cities or geographical areas).
The WASH Results Programme aimed to achieve scale by bringing services to millions of people. Alternative interpretations of scale in WASH programming, such as focusing on improving the effectiveness and reach of government interventions; establishing national funds which could be accessed by local governments; or advancing sector reforms and fiscal decentralisation were not pursued.
The programme was phased: an initial focus on outputs was followed by an outcomes phase
The WASH Results Programme consisted of two ‘phases’, and this had an important influence on suppliers’ programming. The output phase ran from April 2014 until December 2015 (the MDG deadline). During this phase, services were brought to the target populations and approximately 80% of the budget was disbursed. In the outcome phase (January 2016 to early 2018) the reach and sustainability of the services provided in the first phase were to be consolidated. The main argument in favour of the WASH Results Programme’s phasing was the fact that suppliers would have a real opportunity, post-implementation, to focus entirely on equity and sustainability issues. One of the three suppliers in the WASH Results Programme effectively ignored the phasing, due to having a more comprehensive framework for addressing sustainability. Though for the other two suppliers the pressure in the output phase to deliver results resulted in comparatively less attention being paid to equity and sustainability issues earlier in the programme.
The issue of phasing is significant for future programmes that seek results at the output and outcome level and as it is possible that a similar arrangement could occur in future as the SDG deadline approaches or in response to the WASH challenges posed by the COVID-19 pandemic. In particular, that the pressure to deliver output-level results should not take precedence over sustainability or equity considerations.
Delivering at scale – key outcomes and impacts sought under the WASH Results Programme
Inclusion. The WASH Results Programme did not set inclusion-related targets; though many suppliers nonetheless intended that all members of target communities should benefit from their interventions. In view of the public good (and public health) aspects of WASH specifically, this was important and necessary, even beyond the human rights issues involved. Although DFID did not explicitly require suppliers to undertake “total sanitation” interventions (i.e. those in which entire communities access and use safe sanitation), this was in fact the strategy pursued by most suppliers, in line with their usual practices. Local-level “scale” (i.e. near-complete coverage of services in targeted communities and areas) is essential in WASH programmes of any size.
Sustainability. The WASH Results Programme intended to deliver sustainable services and hygiene practices. DFID required suppliers to address the various aspects (functionality of systems, institutional aspects, financial and environmental matters) of sustainability, and it drew attention of suppliers to the need to engage both with communities and with local and national governments in the pursuit of sustainability.
Impacts. DFID’s intended impact was that “poor people have improved health (as a consequence of using safe and sustainable WASH services)”. The indicators of that improved health were to be reduced incidence of diarrhoea and reduced child mortality. Health impacts were not measured, either by suppliers or by the programme evaluation team.
As a statement of impact, this was simultaneously both highly ambitious and very limited. It was ambitious in the sense that the direct impact of WASH programming on diarrhoeal disease is frequently less than hoped for; indeed rigorous studies taking place at the time of the WASH Results Programme were finding such impacts to be elusive. It was also limited however, in the sense that potential health impacts extend well beyond the reduction in diarrhoea (to include a number of neglected tropical diseases – or NTDs – and possible reductions in stunting), and the wider potential impacts extend well beyond health, as attested by DFID’s own Evidence Paper on WASH Impact (Cairncross et al, 2013). With the exception of interventions which were scaled to be truly community-wide or area-wide (i.e. in which all or nearly all inhabitants of communities or administrative areas practise safe sanitation and good hygiene), the impacts on diarrhoeal disease as a result of the WASH Results Programme were likely to have been modest.
In future WASH programmes, a more explicit focus on community-wide use and practice, and on sustainability of services into the indefinite future would bring about a greater likelihood of such impacts being achieved – although current research evidence suggests that even these measures may not necessarily lead to major health impacts.
In addition to the requirements set out in DFID’s Business Case and Invitation to Tender, a number of other requirements were placed on suppliers, especially in regard to sustainability, in-country capacity, and disaggregated monitoring of results.
However, a number of potential requirements were not explicitly made; this was in part due to the programme having a rather rudimentary theory of change, in which key assumptions were neither made explicit nor tested. Had they been, then there is reason to believe that the WASH Results Programme’s outcomes and impact may have been even greater.
Delivering WASH results at scale: recommendations for future programmes
DFID’s emphasis on outcomes (inclusion and sustainability), and its requirement for robust and disaggregated monitoring, are important for WASH programmes at any scale.
A number of additional recommendations can be made for future programmes:
- Make explicit that all interventions should be undertaken community-wide and not simply in pursuit of numbers of beneficiaries. In the event, the suppliers (all reputable INGOs) carried out their programming in this way, but this cannot be assumed.
- Where possible, work to achieve full coverage across administrative units such as districts. This would further enhance the likelihood of health impacts, as well as facilitating further scaling-up within the country.
- Sanitation and hygiene interventions are likely to be more effective where water services are already adequate. In cases where these services are not adequate; improvements in water services are necessary too.
- Carry out hygiene promotion programmes of sufficient reach and duration to effect significant behaviour change. In the WASH Results Programme, suppliers were expected to increase hand-washing by 15% compared to baseline through their hygiene promotion efforts. The level of this target would appear limited in ambition and certainly insufficient to bring about significant health impacts. Furthermore, the assumption made in parts of the WASH Results Programme that Community Led Total Sanitation triggering included sufficient hygiene promotion, is unsafe.
- Use the influence of large-scale budgets to put pressure on national governments to implement fiscal decentralisation more fully. The inability of local governments to provide effective post-implementation support to community-level services is due in no small measure to inadequate budgets.
Defining and delivering scale in WASH programming requires attention not only to the numbers of people served, but to the quality and sustainability of services; providing services which are used by very high proportions of the populations of served communities; and achieving a real transfer of ownership from externally-delivered programming to government-ownership of scaling-up processes.
A general lesson from the WASH Results Programme is that outcomes around sustainability, equity and inclusion need to be designed in and pursued from the outset. It is too late to address matters of equity and sustainability if they have been neglected in the pursuit of scale defined solely or mainly in terms of numbers of people reached.
Delivering WASH outcomes at scale is challenging. While it is relatively easy to provide many people with access to services, the more difficult task of ensuring changed use and behaviour takes different skills, more time, and sustained engagement. The transfer of ownership of that engagement with communities, from external suppliers to local government – as envisaged in a full conceptualisation of scale – is particularly difficult when those local governments lack capacity, and crucially, resources.
Given the present uncertainties in predicting health impacts consequent upon WASH interventions, those funding WASH programmes should exercise caution in articulating such intended impacts. However, an important prerequisite for such impacts is community-wide and area-wide use of safe sanitation and of good hygiene (especially handwashing) practices. An explicit and thorough Theory of Change can help in the specification of future programme designs, and of full sets of requirements for implementing partners.
This blog post is adapted from an original paper by Richard Carter and Jeremy Colin, both members of the OPM evaluation team.
The evaluation on which this post is based is available from the DFID Research Portal and is summarised here: WASH Results Programme Evaluation: findings and recommendations for the WASH sector
[i] Taylor B (2013) Effectiveness, scale and sustainability in WASH programmes – a review. Springfield Working Paper Series (2), The Springfield Centre, Durham. Available at: http://www.springfieldcentre.com/wp-content/uploads/2013/10/2013-10-Effectiveness-Scale-and-Sustainability-in-WASH.pdf
[ii] Mangham L J and Hanson K (2010) Scaling up in international health: what are the key issues? Health Policy and Planning, Vol 25 pp85-96. Available at: https://www.ncbi.nlm.nih.gov/pubmed/20071454
[iii] Gargani J and McLean R (2017), Scaling science. Stanford Social Innovation Review, Fall 2017. Available at: https://ssir.org/articles/entry/scaling_science
[iv] Coburn C E (2003) Rethinking scale: moving beyond numbers to deep and lasting change. Educational Researcher, Vol 32, No. 6, pp 3-12. Available at: https://www.sesp.northwestern.edu/docs/publications/139042460457c9a8422623f.pdf
[v] Burns M (2014) the myths of scaling-up. Centre for Education Innovations. Blog available at: https://educationinnovations.org/blog/myths-scaling
[vi] Frake A N and Messina J P (2018) Toward a common ontology of scaling-up in development. Sustainability Vol 10, pp 835-845. Available at: https://www.mdpi.com/2071-1050/10/3/835
Like this:
Like Loading...
For those who were unable to attend the live webinar, it can be viewed 
The evaluation found that PbR appeared to be a viable option for funding WASH programmes but with some important design considerations that we explore in this blog post.
The shape of implementing consortia, the countries in which programmes were implemented, and the targets that were set were all influenced by the PbR modality. In this post we explore how the PbR modality of the WASH Results Programme shaped the behaviour of the organisations who were implementing the programme, known as “suppliers”.
Payment by Results is a form of Results Based Financing increasingly used by aid donors but opinions remain divided over its merits and limitations. Based on the evaluation of a large-scale water, sanitation and hygiene programme, this series of blogs explores one example of how this much-discussed funding mechanism has worked in practice.
