The Millennium Challenge Corporation (MCC), through its Compact with the government of Lesotho (GoL), awarded $164-million over five years for investment in improved water supplies and sanitation facilities for rural and urban domestic, commercial, and industrial users. As part of its commitment to transparently and thoroughly monitor and evaluate its activities, the MCC contracted NORC in 2007 to conduct an impact evaluation of its water sector activities. This report presents the Impact Evaluation of the Rural Water Supply and Sanitation Activity (RWSSA).
RWSSA originally included 250 rural water supply points and 10,000 VIP latrines and had a budget of $30.2 million (18 percent of the $164-million Water Project in the Compact). In order to increase the coverage of VIP latrines in participating villages, MCC subsequently increased the budget to $40.1 million and the Government of Lesotho (GOL) contributed $17.1 million to RWSSA. In addition, the target for VIP latrines coverage was increased from 10,000 to 27,245 in the Lesotho M&E Plan. When the Lesotho Compact ended in September 2013, 175 water systems (70% of the target) and 29,352 VIP latrines (108% of the target) had been installed.
Implementation continued post-Compact with approximately $5.3 million of additional funding from the GOL; ultimately, 250 water systems (100% of the target), and 31,768 VIP latrines (117% of the revised target), were completed. The total cost of RWSSA, including MCC and GOL funding during the Compact and after, was approximately $60 million.
Households impacted by the program are located in villages that were identified by the Department of Rural Water Supply (DRWS) as lacking access to safe drinking water and adequate sanitation. To identify the effects of the program on the outcomes of interest, eligible villages were randomly assigned to treatment and control groups. The list of outcomes analyzed includes toilet use, type of water source used, time spent collecting water, diarrhea incidence and income, among others.
To evaluate the impact of the program we use data from the baseline and follow-up Impact Evaluation Multipurpose Surveys (IEMS). The IEMS is a longitudinal analytic survey specifically designed to collect data for the impact evaluations of the MCA-Lesotho Compact health and water (rural and urban) activities. The baseline version of the survey included items on basic sociodemographics, household access to water and sanitation, water consumption, hygiene and health profiles. The follow-up version of the survey added a few labor outcome items to the instrument.
During program implementation construction delays in some treatment villages meant that construction works ended after follow-up data collection. As a consequence, randomization was compromised because the villages that were actually treated before follow-up data collection were a subset of the villages that were assigned to the treatment group originally. To tackle this problem the evaluation used Instrumental Variables (IV) methods in order to evaluate the effect of the program. This approach exploits the fact that treatment assignment was randomized, but it also addresses the fact that treatment was not provided as planned in all treatment villages.
The impact evaluation shows that the program has had significant effects on key wellbeing indicators. It found that households in the treatment group are more likely than in the control group to use as their main water source an improved water source, such as a public standpipe or a protected spring, as opposed to an unimproved source, such as an unprotected spring or surface water. They are also more likely to use a toilet and spend less time collecting water. However, the evaluation did not find any impacts significant at standard levels of confidence for any of the diarrhea incidence indicators analyzed, although most of the estimated effects have the (negative) expected sign.
The evaluation also did not find any effects for any labor outcomes, or income. An important exception to this is that it found that the program has a positive and significant effect on female labor participation. The evaluation discusses the mechanisms that can explain why the effects on labor outcomes are not more apparent. In particular, it is possible that time availability does not translate into better labor outcomes because the latter are not restricted by time availability but by other conditions, like the labor market itself.
In terms of policy implications, the results described in this report imply that this type of program can have major impacts on households wellbeing via reductions on time spent collecting water, but limited effects on higher level outcomes, like diarrhea incidence. Furthermore, even if household members spend less time collecting water as a result of the program, it is not clear that this will translate into a 1:1 increase in the number of hours they participate in the labor market, as labor outcomes may depend on more factors than just greater available time.
