Truly exceptional? Handling misfortune within Payment by Results

An exceptional event or a predictable adversity? The difference matters more in a Payment by Results (PbR) setting, as this blog post explores.

Conflict, political upheaval, epidemic, drought, flooding and earthquake; the WASH Results Programme has been hit by a wide range of disasters across the 12 countries in which it operates. All these adversities had an impact on the populations involved: some hit the programme’s implementation, some the sustainability of its achievements and others have affected the ability to monitor and verify those achievements.

Discussions on how to deal with these events have involved considering what is within the reasonable expectation of a Supplier to anticipate and deal with, and what is a truly exceptional event for which there should be flexibility around what Suppliers are expected to deliver – whether in the quantity, scope or timing of results.

The challenge of responding to exceptional events is not new for development programmers, but like many challenges, it takes a different shape in the context of a PbR programme. In such programmes, payment is linked to achieving and verifying results, and an impact on results ultimately leads to a financial impact for one or more of the parties. This is particularly challenging in the second phase of WASH Results, where Suppliers are paid for sustaining results achieved in the first “Outputs” phase. The passage of time means that programme achievements (whether physical infrastructure or behaviour change) are more likely to be affected by exceptional events, and suppliers may not have the resources (such as programme field staff) in place to undertake substantial mitigating actions.

Members of our team (Monitoring and Verification for the WASH Results Programme) recently met to discuss their experience of exceptional events in the programme and how these were tackled. Here are three of the examples they discussed followed by the team’s reflections on the issues they raise:

1) The moving population In this example a conflict-affected community relocated out of the proposed project area. In response, the Supplier closed the project in that area but thanks to overachievement of results in other locations, the overall outputs of the programme (in terms of beneficiaries reached) were not reduced and the Supplier did not suffer financially. In this case, the flexibility of PbR meant the risk to Supplier, Donor and Verifier could be effectively mitigated, although some of the original intended beneficiaries were not reached.

2) Destroyed infrastructure, different decisions In one instance, WASH infrastructure built in the first (output) phase of the WASH Results Programme was destroyed when a river eroded part of a village.  Although there was a known risk of erosion, the timing of erosion could not be foreseen nor the risk mitigated. The people living in this area were some of the poorest and most vulnerable whom the Supplier did not want to exclude from the programme.  The erosion was considered extreme, as evidenced by newspaper reports and other local data and it was agreed the area with the destroyed infrastructure would not be included in the sample frame for outcome surveys and so would not affect outcome results.

Meanwhile, in the same country, infrastructure was damaged by flooding, but this was considered expected, not extreme. In contexts where flooding can be expected, the demand for sustained outcomes (in which payment is linked to the sustained use of infrastructure) requires that infrastructure is built in such a way that it can withstand expected levels of flooding, or that plans for reconstruction or repair in the case of damage should be integral to programming. Consequently, areas in which infrastructure was affected by flooding were to be included in the sample frame for the outcome survey, which was amended to include questions about flood damage and beneficiary priorities for reconstruction.

3) When verification is too risky  When conflict erupted in one project location, the programme was able to implement activities regardless and continued to achieve results. However, the security situation on the ground made it too risky (for programme staff and the verification team) for the results to be independently verified through a field visit. In this case, alternative and less representative forms of verification were accepted. In this example, there was no adverse impact on the results achieved, or reduction in payment to the Supplier, but there was increased risk around the confidence that could be placed in the results.

Making decisions about risk

In exceptional circumstances, decisions need to be made about who bears the risk (Donor, Supplier, Verifier, Beneficiaries) and what kind of risk (physical, financial, reputational). If financial risk falls exclusively on Suppliers, they need to factor that into their “price per beneficiary” across the programme. Alternatively, Suppliers may choose not to operate in riskier areas, with potential negative consequences for the equity of programme interventions. If donors accept all risk, there is little incentive for Suppliers to programme in ways that account for predictable risks, such as flooding, particularly over the longer term.

Reflections and suggestions emerging from the discussion of these cases included the following:

  • There are different types of impact to consider: effect on population, effect on ability to deliver activities, effect on achievement of results, and effect on ability to verify results. Being clear on the type of impact might aid decisions about who bears the risk and the mitigation strategy.
  • Discussions about risk need to happen during the design phase; one approach is to use a risk matrix that explores what level of risk each party is going to absorb (and so design into the programme) and what would be considered ‘exceptional’.
  • Programmes need to include within their design plans for responding to anticipated events e.g. in areas at risk of flood, include help for villages to cope with normal levels of flooding.
  • Suppliers can minimise their financial and operational risk by balancing their work across a range of fragile and more secure areas, so enabling them to pull out of more hazardous areas in extreme circumstances and achieve results elsewhere. However, if the Supplier commits to working with specific communities in conflict-affected areas, then incentives will need to be set up differently within the results framework.
  • In fragile contexts, a compromise may need to be made on rigour of verification and plans made for reliance on remote verification from the start, e.g. analysis of systems, remote data collection through phone or satellite, and beneficiary monitoring.

Our conclusions about exceptional events in the context of a PbR programme in WASH echo those in many of our previous blog posts. PbR throws a spotlight on an issue, raises questions of how risk is shared between stakeholders, and highlights the importance of planning at design phase and of flexibility, should the unforeseen occur.

If you have any ideas or observations about this topic, we encourage you to Leave A Reply (below), or email us

2 responses to “Truly exceptional? Handling misfortune within Payment by Results

  1. Excellent blog post.

    Accounting for extraordinary and exogenous factors is a big set of challenges.

    Are there any ways to develop baselines of what is ‘normal’? While natural disasters that can affect programs and implementation have historical records of periodicity (heck, flood and natural hazard insurance is based upon this sort of data), I have no idea how one would be able to determine a baseline of expected location-specific political instability risk.

    Keep us posted on this blog. I certainly would love to see where this discussion goes.

    Best,

    Ed Bourque
    http://www.edbourqueconsulting.com/blog/

    Like

  2. Thanks Ed for your positive comments and thought provoking question.

    We think it is valuable for donors to have an agreed framework for working out the legitimacy of implementer’s claims for pay out should results not be obtained due to exceptional events.

    For example, the Monitoring and Verification (MV) team was asked to suggest a framework for assessing whether an event was exceptional in relation to a drought in one of the operating countries. We proposed that it was relevant to consider whether a similar drought event could have been reasonably expected to happen within a period of two or three programme lifetimes. Thus, if the WASH Results Programme is 5 years long, and a drought of this severity might be expected within 10 or 15 years, then the supplier should reasonably have foreseen its likelihood and planned for it.

    In relation to assessing risk of insecurity, it looks as though there are some tools emerging that help to assess risk of conflict and violence eg New Methods to Assess Risk of Conflict and Violence https://reliefweb.int/report/​world/new-methods-assess-risk-​conflict-and-violence, although as the MV team we do not have direct experience of them. These may provide a useful basis for a conversation between donor and supplier either at the contracting stage when risk sharing is being discussed or in the unfortunate event of disruption of the programme.

    In the end, though, it will always be up to the donor and implementer to agree on what essentially is some kind of force majeure, and that could well be quite different in different regions of a country – particularly environmental risk, but also political security. This means that assessment would often need to be at a region level, so it would have to be done on a case-by-case basis based on whatever evidence is available.

    Catherine Fisher, (Learning Advisor for the Monitoring and Verification Team)

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s