Using Payment by Results to deliver WASH services at scale: lessons from the WASH Results Programme

Introducing our series of blog posts sharing lessons from implementing Payment by Results (PbR) to deliver WASH services at scale. 

Payment by Results is a form of Results Based Financing increasingly used by aid donors but opinions remain divided over its merits and limitations. Based on the evaluation of a large-scale water, sanitation and hygiene programme, this series of blogs explores one example of how this much-discussed funding mechanism has worked in practice.

The WASH Results Programme was one of the first large-scale applications of PbR in the WASH sector and within the Department for International Development (DFID). As such, the programme provides a rich source of learning about how PbR works at scale and how to use it in WASH in the future.

Blog posts in the series explore:

• What PbR was intended to achieve and key features of the programme, including what DFID aimed to incentivise through using PbR in the WASH Results Programme, the perceived advantages and disadvantages of the PbR modality (and if they materialised).
• How PbR shaped the implementation of the WASH Results Programme and other non-PbR factors that influenced the behaviour of the implementing organisations (suppliers).
• Key considerations when contracting using a PbR modality – lessons for funders from the WASH Results Programme. The evaluation found that PbR appeared to be a viable option for funding WASH programmes but with some important design considerations which we examine in this blog.

This series of posts draws on the findings of the evaluation of the first round of the programme (May 2014 to March 2018). Undertaken by the e-Pact consortium, the evaluation included document analysis, literature reviews, verification data, key informant interviews and supplier case studies.

For a more detailed exploration about how the PbR mechanism functioned within the WASH Results Programme, how it shaped supplier behaviour and implications for donors, please see the Executive Summary of the programme evaluation, available along with other publications from the evaluation, at the DFID Research for Development website.

 

Answering questions on the WASH Results Evaluation

Over 180 people from NGOs, funding agencies and research organisations around the world with an interest in WASH and Payment by Results participated in a panel discussion last week on the evaluation findings of the WASH Results Programme.

Participants grilled the panellists on issues of sustainability, sector strengthening and the role of monitoring in WASH programmes.   If you missed the webinar, you can watch it online here.

After a short presentation of the key findings, the panelists shared their experience of the programme and responded to questions from the audience which focused largely on:

• The nature of the design of the Payment by Results modality and how the implementing organisations responded.
• Whether the WASH Results Programme was good value for money.
• The sustainability of the programme’s results.
• How the WASH Results Programme contributed to systems strengthening in the countries in which it operated.
• The role of monitoring in the programme.

The questions prompted lively debate during the Q&A session and continued in the webinar chat. 

For those who were unable to attend the live webinar, it can be viewed here with a record of the Chat. The synthesis report and other publications from the evaluation are available to download from the DFID website.

If you would like to comment on the evaluation findings or share your experience of comparable programmes, you can do so in the series of blog posts summarising the findings  :  Using Payment by Results to deliver WASH services at scale: lessons from the WASH Results Programme . 

 

 

WASH Results Programme Evaluation: findings and recommendations for the WASH sector

Evaluation of large-scale WASH programme reveals Payment by Results (PbR) is a viable option for funding WASH at scale, if used carefully.

Photo by Nitin Sharma on Pexels.com

This blog summarises findings of the evaluation by the e-Pact Consortium of the UK Department for International Development’s (DFID’s) Water, Sanitation, and Hygiene (WASH) Results Programme: a four-year programme (May 2014 to March 2018), which aimed to bring equitable and sustainable water and sanitation services and hygiene practices to millions of people in 11 countries. The programme was implemented by three suppliers contracted to DFID under a PbR financing modality (SAWRP, SWIFT and SSH4A).

As one of the first large-scale applications of PbR in the WASH sector and within DFID, the WASH Results Programme provides a rich source of learning about how PbR works at scale and how to use it in WASH in the future. Based on the evaluation, this blog explores the following areas:

1. Whether the programme successfully achieved its stated objectives;
2. The influence of programme design, including the PbR modality, on this achievement;
3. Lessons for applying PbR in WASH programming going forward;
4. Recommendations for future WASH PbR programmes.

This blog is an abridged version of the Evaluation Synthesis, which is available to download from DFID’s Research for Development online repository along with reports from all of the component studies of the evaluation.

Lessons from implementing the WASH Results Programme #3: Implications for funders

Our final blog post in a series, sharing lessons from implementing Payment by Results (PbR) to deliver WASH services at scale draws out implications for funders when designing future programmes.

The evaluation found that PbR appeared to be a viable option for funding WASH programmes but with some important design considerations that we explore in this blog post.

Lessons from implementing the WASH Results Programme #2: The influence of PbR

Our second blog post sharing lessons from implementing Payment by Results (PbR) to deliver WASH services at scale explores the influence of the PbR modality on suppliers’ behaviour. 

The shape of implementing consortia, the countries in which programmes were implemented, and the targets that were set were all influenced by the PbR modality. In this post we explore how the PbR modality of the WASH Results Programme shaped the behaviour of the organisations who were implementing the programme, known as “suppliers”.

Our second blog post sharing lessons from implementing Payment by Results (PbR) to deliver WASH services at scale explores the influence of the PbR modality on suppliers’ behaviour. 

The shape of implementing consortia, the countries in which programmes were implemented, and the targets that were set were all influenced by the PbR modality. In this post we explore how the PbR modality of the WASH Results Programme shaped the behaviour of the organisations who were implementing the programme, known as “suppliers”.

Lessons from implementing the WASH Results Programme #1: Intentions and key features

In this first post of a series, we look at what Payment by Results (PbR) in the WASH Results Programme was intended to achieve and key features of the programme.

“All happy families are alike; each unhappy family is unhappy in its own way.”  Leo Tolstoy memorably declared in the opening lines of Anna Karenina. Without wanting to suggest that PbR contracts are unhappy, they are certainly all different. An understanding of what the programme was trying to achieve and the characteristics of the way in which PbR was applied are important when thinking about the effects of the PbR modality.

In this first in a short series of blogs exploring the effects of the PbR mechanism on a large scale WASH programme,  we explore what PbR was intended to achieve in this instance and key features of the programme.

Rewards and realities of Payment by Results in WASH

If you missed the WASH Results Programme’s panel discussion at World Water Week 2018, then this webpage on the conference organiser’s website is for you.  You can read details of the presenters, find out where the group discussions took us and the types of questions the panel was asked about PbR in WASH.

The conclusions from the session were:

• There was some consensus that the data demands of PbR have served to strengthen suppliers’ data and monitoring systems, supported by an external verification system. This has led to better data on which to base evolving programming decisions.
• Those designing WASH Programmes using other funding modalities could usefully consider how they too could incentivise investment in and use of monitoring data whilst managing risks.
• The relatively limited experience of PbR in the WASH sector to date suggests a need for continued learning in this area if the funding modality is more widely adopted in future.

We’ll be sharing our reflections on the session, as the MV team for the WASH Results Programme, in a forthcoming blog post, but in the meantime please take a look at these reports by the programme’s Suppliers:

Selected key messages of our joint session (video by SNV)

Blog post on World Water Week, by SAWRP 

Related blog posts:

Oxfam’s experience with the SWIFT Consortium for Sustainable WASH in Fragile Contexts

Hard work pays off: access and service delivery results under a Payment by Results programme

WASH verification: in vino veritas, in aqua sanitas

 

 

WASH verification: in vino veritas, in aqua sanitas

Ahead of the WASH Results Programme’s session at World Water Week, Andy Robinson (Lead Verifier in the Monitoring and Verification Team for the DFID WASH Results Programme) looks at the importance of third party and internal verification in improving the quality and reliability of programme monitoring and evaluation.

‘In wine there is truth; in water there is health’. Sanitation is derived from sanitas, the Latin word for health; and veritas means truth, the root of the word verification – the process of establishing that something is true. Verification may be the missing ingredient in large-scale monitoring and evaluation systems – the wine that enables WASH programmes to hear the truth.

Despite the investment of considerable programme time, capacity and resources in the development and implementation of rural sanitation monitoring and evaluation systems, the reality is that many large-scale monitoring systems in developing countries do not work well. Large-scale sanitation monitoring systems rarely provide the detail that we need; and it can be hard to distinguish reliable data from unreliable. This is a huge sector problem: when you don’t know what is happening early enough, or where things are going wrong, it is hard to respond quickly or effectively.

Lessons learnt from the DFID WASH Payment by Results (PbR) programme suggest that verification may be an important part of the solution: both external (third party) and internal (within programme) verification appear to be important factors in improving the quality and reliability of programme monitoring and evaluation [1].

DFID contracted an independent monitoring and verification (MV) team to verify the results from the WASH PbR programme. PbR financing means that implementing agencies receive funding on the basis of verified results, which generates strong incentives for agencies to achieve results, and to provide solid evidence of these results to the MV team. As a result, agencies invest in more comprehensive and responsive programme monitoring and evaluation systems, which generate rapid and detailed information on programme performance and progress.

The MV team verifies the results based on the data and reports from the monitoring and evaluation systems set up by the implementing agencies. This arrangement requires the MV team to appraise how these systems work; to identify potential risks and weaknesses; and to review whether systems and processes are working as intended. In the multi-country SNV Sustainable Sanitation and Hygiene for All WASH Results Programme (SSH4A WRP), the verification role appears to have had a noticeable impact (see box).

---

Box 1     Impact of verification: both internal and external

Firstly, it has encouraged SNV to strengthen its monitoring and reporting systems. Household surveys are the main instrument used by SNV to evidence sanitation and hygiene outcomes. Third party verification confirmed that some survey enumerators did not do their work properly – perhaps by taking short cuts to save time, or failing to follow protocols correctly – and that some processes could be improved. The mobile-to-web survey approach used by SNV greatly facilitated the verification process, with GPS and time data available for household interviews, and toilet photographs where appropriate. The rich survey data facilitated a cost-efficient and largely remote verification process, with only a small sample of villages and households visited in person to check for systemic errors.

Many of the survey problems were identified at the start of the programme, and SNV worked to strengthen and professionalise its monitoring and reporting processes so that later surveys ran smoothly and efficiently. This brings us to the second impact of the verified PbR – the development of the verification process was transparent, with all checks discussed and agreed with SNV before use. In order to avoid any surprises during verification, SNV began to run similar checks on the surveys before verification. As a result, SNV spotted several monitoring issues early, and was able to take prompt action to remedy these problems.

---

The combined effect of the PbR imperative to achieve results, and the detailed verification of these results, have led to more professional and systematic approaches to monitoring and reporting. All of these have important effects on implementation – as more reliable data and more rapid feedback mechanisms encourage adaptive management.

The use and analysis of the monitoring data also strengthens the systems – data collectors know that someone will be analysing the data, someone else will be verifying the data, and that these data will form the basis of programme payments. The shift caused by the greater scrutiny and use of the monitoring data has been a game-changer. The value of reliable monitoring and evaluation systems is increasingly apparent, with programme managers and project staff now able to focus on the systems and processes that achieve sustained outcomes, rather than on the inputs and expenditures that drive conventional WASH programmes [2].

Verification has proved a useful tool to encourage more professional and systematic approaches to monitoring and evaluation, but it is only one small part of the wider challenge of achieving large-scale and sustained development outcomes – wine may bring out the truth, but turning truth into health is a much harder trick!

Suggestions for further reading: Visit e-Pact’s blog series, Learning about Payment by Results in Water, Sanitation and Hygiene (WASH) A blog by the team verifying and evaluating DFID’s Results Programme:  

[1] “Beyond a burden: what value does verification offer“

[2] “The payback and pains of Payment by Results (part 1)“. 

About the Author: Andy Robinson is an independent water and sanitation specialist based in the French Alps. Since 1987, Andy has worked on the design, implementation, and evaluation of WASH programmes in Asia and Africa for a diverse range of clients. In the last fifteen years, Andy has been heavily involved in the promotion of improved sanitation and hygiene, working with governments, development partners, and communities in more than 20 countries in Africa and Asia.     

This post was originally published on the SNV website and is one of a series leading up to World Water Week 2018.

Going to #WWWeek? Come to the WASH Results session on Sunday morning and find out more about the realities of Payment by Results in WASH.

Hard work pays off: access and service delivery results under a Payment by Results programme

In 2014, the ‘new kid on the block’ for WASH sector funding was introduced through DFID’s WASH Results Programme’s Payment by Results. The fund was designed to give impetus to meet the MDG goals on WASH, found to be lagging behind in many countries. The targets were ambitious, the model risky. SNV’s Anne Mutta shares SSH4A RP’s experience in implementing PbR.

The Sustainable Sanitation and Hygiene for All Results Programme (SSH4A RP) of SNV, financed by UKAID, was among the first organisations to engage in PBR programming in 2014. By end of December 2017, our four-year experience in implementing SSH4A RP in 62 districts across nine countries in Africa and Asia successfully engaged 7 million people. Independently verified results of the programme found that at least 2.7 million people have gained access to and are using new and improved sanitation facilities. Beyond meeting our pre-defined results at household level, SNV’s payment was tied to so-called ‘Sustainability Results’, measuring progress towards sustainable service delivery systems in each district.

Stephen Covey says ‘begin with the end in mind’, under a PBR system this translates to having a definite goal to work towards (the results) and constantly reviewing what you are doing (process) to see if it is getting you where you want to go!

SNV staff and government officials dialogue with kebele female sanitation group

Kebele residents count days before flag turns green signifying ODF achievement

Under a PBR contract, programme targets are fixed and underachievement is ‘penalised’. Because overly-optimistic targets are considered financial risks, organisations are challenged to be more realistic at the start of programme design. Building on and ‘growing’ our years of SSH4A work (developed in 2008) had helped us offer ambitious but realistic targets that can be delivered within a given time frame. Similarly, because we’ve been physically present in our SSH4A RP countries for years (for example, we’ve been present in Kenya since 1969) our knowledge base and partnership with local government and CSOs allowed us to prepare for potential risks and unforeseen events. The number of countries and districts, helped to balance out the effect of those unforeseen events, as there are many factors beyond our control, in particular affecting sustainability indicators.

Although targets are fixed under DFID’s WASH Results Programme (WRP), there is flexibility in implementation: programme activities may be changed to respond to evolving contexts. A lot of time savings were realised because there was no need for us to engage in a protracted consultation process with UKAID when funds had to be shifted. The WRP’s PbR gave us more control on how things can be done and allowed us to swiftly respond to shifting needs or priorities. Our country teams –  who had a better handle of the contexts they were working in – were able to adjust their interventions and search for those most suitable in supporting households to gain access to sanitation facilities. Experience and engagements on the ground guided our interventions, and facilitated actual ‘learning and doing’.

PbR increased attention on the need for sound monitoring systems within project designs. A defining characteristic of the WRP is the integration of an independent third party verification process, which is conducted prior to ‘pay out’. With quality monitoring and evaluation (M&E) becoming an integral part of programme conversations, we introduced robust M&E tools, our country teams became involved in conducting household surveys, and they organised regular stakeholder review/ reflection meetings on sustainability indicators. Constantly questioning whether progress had been made, why and how things can be done better and sustainably have helped us reach optimal results. This reflection process served as the main driver behind our achievements, and the results that we all take pride in.

By embracing this new working modality, and with hard work, we managed to surpass at least 95% of our SSH4A RP pre-defined targets. Without a reflective, evidence-based process of target setting and progress monitoring, it is foolhardy to imagine that we are able to get the same desired results.

About the Author: Anne Mutta is multi-country programme manager of SNV’s largest Payment by Results (PbR) programme to date – a multi-million programme that is being implemented across eight countries in Africa and Nepal. Anne, based in Kenya, has 20 solid years of experience leading action-oriented and evidence-based WASH programmes.

Photo credit: Anjani Abella/SNV

This post was originally published on the SNV website and is one of a series leading up to World Water Week 2018.

Going to #WWWeek? Come to the WASH Results session on Sunday morning and find out more about the realities of Payment by Results.

Oxfam’s experience with the SWIFT Consortium for Sustainable WASH in Fragile Contexts

What is Payment by Results like for Suppliers? In this post (first published by SNV) the SWIFT Consortium’s Global Programme Manager reflects on Oxfam’s experience.

In 2014 Oxfam made the decision to bid for a big commercial tender. It was not just any commercial tender but one that was structured along a Payment by Results (PbR) process. This meant that all programme funding had to be pre-financed by Oxfam and its partners. Once results have been proven, Oxfam gets paid by DFID (Department for International Development). This process introduced a new way of working for a lot of teams: we had to provide concrete evidence to prove our success.

Oxfam took a lead in this PbR-funded project as member of the SWIFT Consortium for Sustainable WASH in Fragile Contexts: a consortium with partners from Kenya, DRC and Liberia. Our ambition was to reach over 1 million people across these three countries over a four-year period. Unfortunately, the Ebola crisis in Liberia halted programming there, but we continued to work in DRC and Kenya.

Between 2014 and 2016 the programme had reached over 850,000 people in DRC and Kenya, providing safe water or undertaking sanitation and hygiene promotion activities. During this period, Oxfam achieved its first set of goals. Despite the immense pressure on our teams in countries, we received 100% payment for the initial phase.

Unlike many traditional implementation programmes, DFID included a sustainability phase in its programming. During this phase – between 2016 and 2018 – teams supported local capacity for the management of systems. Through working with water utility companies, we sought to improve services and introduce an e-billing service to improve efficiency and decrease corruption. Both water utility companies that Oxfam worked with now have improved revenue collection systems; staff of one company now have their own uniforms.

In Nairobi, Oxfam partner WSUP (Water and Sanitation for the Urban Poor) worked with Nairobi Water Company to bring water to individual compounds in some of the poorest neighbourhoods of the city. Oxfam also expanded Water ATMs, which allowed for people to access water whenever it is convenient, and ensured that payments are clear and transparent.

Arron Gesar holds token to collect water from a Water ATM in Hadado Kenya

In DRC Oxfam piloted ‘ASUREPs’, a professional water management model, which has made significant improvements to the maintenance of water systems, increased the proportion of people paying for the service and, in some places, raised money to extend the system. This model was implemented in semi-urban localities: small towns or high growth areas. It has been praised by the government, leading to an opportunity to influence WASH policies at national level.

Ms Kahindo Byamungu, Sake ASUREP worker, opens water point at scheduled times

The benefits of this programme, beyond its achievements, have been for all of us – from head offices to country teams and partners – to really think about sustainability, about WASH, and look at how we are measuring success.

All these activities took place during a drought in Kenya, insecurity in DRC, and heightened political tensions during the election period for both countries. The teams, throughout the programmes, went above and beyond to deliver a successful programme, which has been called a ‘high performing programme’ by DFID.

Finally, this risk has been paid off with DFID extending our DRC programme for another three years.

About the Author: Joanna Trevor has over 15 years of experience working in fragile states, holding a range of advocacy and programme management roles across East and Central Africa. She is currently Global Programme Manager for the SWIFT Consortium, which aims to deliver sustainable access to safe water and sanitation, and to encourage the adoption of basic hygiene practices in the Democratic Republic of Congo (DRC) and Kenya. Having joined Oxfam in 2013, Joanna has previously worked for World Vision, Mercy Corps and as a consultant for the British Government’s Stabilisation Unit. Most recently, she was Oxfam’s Programme Director in DRC. She holds a degree in Social Policy from Swansea University.

Photo credits: Banner and 1st photo by Katie G. Nelson/Oxfam | 2nd photo by Anne Cossutta/Oxfam

This post was originally published on the SNV website and is one of a series leading up to World Water Week 2018.

Going to #WWWeek? Come to the WASH Results session on Sunday morning and find out more about the realities of Payment by Results.