Tag Archives: design

WASH Results’ evaluation highlights differences in design

The WASH Results Evaluation Team (led by Oxford Policy Management) shared their midline evaluation findings with the programme’s Suppliers through a webinar earlier this year and the slides are now available online together with the full report.

The WASH Results Midline Evaluation was carried out mid-2016 and explored:

  • The relevance of programme design (theory of change).
  • To what extent the verification systems were fit-for purpose (relevance and efficiency).
  • In what ways the Payment by Results (PBR) modality affected implementation.
  • Which outputs were achieved (those paid for, and those outside the PBR framework).

The evaluation showcases how the three Suppliers designed their PBR systems in different ways, to suit the different WASH activities being implemented across 12 countries between them. As a result, a tailored verification system was also implemented by its Third Party Verifier (led by Itad) to fit this variation. This diversity is summarised nicely in the slides that accompanied the webinar.

The planned output targets of the programme were achieved by December 2015 – with significant over-achievement (though over-achievement was not paid for under the PBR modality). They were achieved off the back of results-oriented problem-solving by the Suppliers who adjusted activities to achieve agreed results. The increased scrutiny of the third-party verification mechanism also helped strengthen monitoring systems of the Suppliers, resulting in more regular and higher-quality monitoring data. However, the evaluators also found that the pressure of achieving results put significant burden on programme staff and partners.

Suppliers managed the risks of PBR in a variety of ways, including front-loading payments and moving output targets between countries if one was under-achieving. With a few exceptions, the Suppliers also opted to pre-finance their local implementation partners and thus hold the PBR risks at the headquarters level, instead of passing them down to local implementation partners.

The midline evaluation webinar concluded with a suite of recommendations for PBR suppliers (NGOs), for DFID and for independent verifiers. The endline evaluation will be carried out in 2018 and will focus on outcome achievement, impact and sustainability aspects.

If you would like to be notified when the WASH Results Endline Evaluation Report is available, please email Cheryl Brown, WASH Results MVE Team Communications Manager.  You may also be interested in the Evaluation Team’s Short Reading List on PBR.




Recent views and news on Payment By Results

New funding for learning, a new framework and a free webinar. In this post we highlight what’s happening online about Payment By Results


One of the challenges that the World Bank acknowledges it and its clients face, is “Choosing the incentive mechanism that best fits an intervention in a particular context and obtains the greatest impact”.  To that end, the World Bank has issued a call for proposals to learn more about the impact results-based financing can have on education systems at a district/regional level. One criticism often levelled at Results Based Finance (RBF) is that it incentivises “cherry-picking”, leaving the harder-to-reach behind in pursuit of easier ways to reach beneficiary targets. The World Bank is therefore requiring those submitting proposals to be able to measure the equity impact of RBF.

Could an “Expert Body” help overcome some of the problems experienced by past Payment By Results (PbR) programmes? Dr Laura Johnstone has just shared a framework for commissioning and implementing PbR programmes which she describes as “dynamic and applicable locally, regionally and nationally to all PbR provision across the public sector wherever target achievement and the provision of value to stakeholders is required”.  A key element of the framework is the establishment of an Expert Body that comprises key stakeholders for the programme “including the commissioner, the principal, the agent, the service users and academics with experience of PbR and the specific need”. The Expert Body plays the roles of governing body at the commissioning stage, and data and learning gatherer during implementation, to inform adjustments to delivery. Dr Johnstone invites feedback on the framework via the blog post or direct email.

Finally, even as this post goes live, a World Bank webinar is scheduled to discuss the findings of a study of results-based climate finance activities. The recording of ‘Achieving transformational change in climate change mitigation through Results-based financing‘ can be accessed after June 6th, from the World Bank’s Open Learning Campus.

Cheryl Brown, Communications Manager for WASH Results MVE