Ahead of the WASH Results Programme’s session at World Water Week, Andy Robinson (Lead Verifier in the Monitoring and Verification Team for the DFID WASH Results Programme) looks at the importance of third party and internal verification in improving the quality and reliability of programme monitoring and evaluation.
‘In wine there is truth; in water there is health’. Sanitation is derived from sanitas, the Latin word for health; and veritas means truth, the root of the word verification – the process of establishing that something is true. Verification may be the missing ingredient in large-scale monitoring and evaluation systems – the wine that enables WASH programmes to hear the truth.
Despite the investment of considerable programme time, capacity and resources in the development and implementation of rural sanitation monitoring and evaluation systems, the reality is that many large-scale monitoring systems in developing countries do not work well. Large-scale sanitation monitoring systems rarely provide the detail that we need; and it can be hard to distinguish reliable data from unreliable. This is a huge sector problem: when you don’t know what is happening early enough, or where things are going wrong, it is hard to respond quickly or effectively.
Lessons learnt from the DFID WASH Payment by Results (PbR) programme suggest that verification may be an important part of the solution: both external (third party) and internal (within programme) verification appear to be important factors in improving the quality and reliability of programme monitoring and evaluation .
DFID contracted an independent monitoring and verification (MV) team to verify the results from the WASH PbR programme. PbR financing means that implementing agencies receive funding on the basis of verified results, which generates strong incentives for agencies to achieve results, and to provide solid evidence of these results to the MV team. As a result, agencies invest in more comprehensive and responsive programme monitoring and evaluation systems, which generate rapid and detailed information on programme performance and progress.
The MV team verifies the results based on the data and reports from the monitoring and evaluation systems set up by the implementing agencies. This arrangement requires the MV team to appraise how these systems work; to identify potential risks and weaknesses; and to review whether systems and processes are working as intended. In the multi-country SNV Sustainable Sanitation and Hygiene for All WASH Results Programme (SSH4A WRP), the verification role appears to have had a noticeable impact (see box).
Box 1 Impact of verification: both internal and external
Firstly, it has encouraged SNV to strengthen its monitoring and reporting systems. Household surveys are the main instrument used by SNV to evidence sanitation and hygiene outcomes. Third party verification confirmed that some survey enumerators did not do their work properly – perhaps by taking short cuts to save time, or failing to follow protocols correctly – and that some processes could be improved. The mobile-to-web survey approach used by SNV greatly facilitated the verification process, with GPS and time data available for household interviews, and toilet photographs where appropriate. The rich survey data facilitated a cost-efficient and largely remote verification process, with only a small sample of villages and households visited in person to check for systemic errors.
Many of the survey problems were identified at the start of the programme, and SNV worked to strengthen and professionalise its monitoring and reporting processes so that later surveys ran smoothly and efficiently. This brings us to the second impact of the verified PbR – the development of the verification process was transparent, with all checks discussed and agreed with SNV before use. In order to avoid any surprises during verification, SNV began to run similar checks on the surveys before verification. As a result, SNV spotted several monitoring issues early, and was able to take prompt action to remedy these problems.
The combined effect of the PbR imperative to achieve results, and the detailed verification of these results, have led to more professional and systematic approaches to monitoring and reporting. All of these have important effects on implementation – as more reliable data and more rapid feedback mechanisms encourage adaptive management.
The use and analysis of the monitoring data also strengthens the systems – data collectors know that someone will be analysing the data, someone else will be verifying the data, and that these data will form the basis of programme payments. The shift caused by the greater scrutiny and use of the monitoring data has been a game-changer. The value of reliable monitoring and evaluation systems is increasingly apparent, with programme managers and project staff now able to focus on the systems and processes that achieve sustained outcomes, rather than on the inputs and expenditures that drive conventional WASH programmes .
Verification has proved a useful tool to encourage more professional and systematic approaches to monitoring and evaluation, but it is only one small part of the wider challenge of achieving large-scale and sustained development outcomes – wine may bring out the truth, but turning truth into health is a much harder trick!
Suggestions for further reading: Visit e-Pact’s blog series, Learning about Payment by Results in Water, Sanitation and Hygiene (WASH) A blog by the team verifying and evaluating DFID’s Results Programme:
About the Author: Andy Robinson is an independent water and sanitation specialist based in the French Alps. Since 1987, Andy has worked on the design, implementation, and evaluation of WASH programmes in Asia and Africa for a diverse range of clients. In the last fifteen years, Andy has been heavily involved in the promotion of improved sanitation and hygiene, working with governments, development partners, and communities in more than 20 countries in Africa and Asia.
This post was originally published on the SNV website and is one of a series leading up to World Water Week 2018.
Going to #WWWeek? Come to the WASH Results session on Sunday morning and find out more about the realities of Payment by Results in WASH.