The WASH Results Evaluation Team (led by Oxford Policy Management) shared their midline evaluation findings with the programme’s Suppliers through a webinar earlier this year and the slides are now available online together with the full report.
The WASH Results Midline Evaluation was carried out mid-2016 and explored:
- The relevance of programme design (theory of change).
- To what extent the verification systems were fit-for purpose (relevance and efficiency).
- In what ways the Payment by Results (PBR) modality affected implementation.
- Which outputs were achieved (those paid for, and those outside the PBR framework).
The evaluation showcases how the three Suppliers designed their PBR systems in different ways, to suit the different WASH activities being implemented across 12 countries between them. As a result, a tailored verification system was also implemented by its Third Party Verifier (led by Itad) to fit this variation. This diversity is summarised nicely in the slides that accompanied the webinar.
The planned output targets of the programme were achieved by December 2015 – with significant over-achievement (though over-achievement was not paid for under the PBR modality). They were achieved off the back of results-oriented problem-solving by the Suppliers who adjusted activities to achieve agreed results. The increased scrutiny of the third-party verification mechanism also helped strengthen monitoring systems of the Suppliers, resulting in more regular and higher-quality monitoring data. However, the evaluators also found that the pressure of achieving results put significant burden on programme staff and partners.
Suppliers managed the risks of PBR in a variety of ways, including front-loading payments and moving output targets between countries if one was under-achieving. With a few exceptions, the Suppliers also opted to pre-finance their local implementation partners and thus hold the PBR risks at the headquarters level, instead of passing them down to local implementation partners.
The midline evaluation webinar concluded with a suite of recommendations for PBR suppliers (NGOs), for DFID and for independent verifiers. The endline evaluation will be carried out in 2018 and will focus on outcome achievement, impact and sustainability aspects.
If you would like to be notified when the WASH Results Endline Evaluation Report is available, please email Cheryl Brown, WASH Results MVE Team Communications Manager. You may also be interested in the Evaluation Team’s Short Reading List on PBR.