The evaluation will address the following evaluation areas:
Evaluation Area 0 examines whether the SNRDP was implemented according to plan. The analysis will focus on highlighting any ways that implementation deviated from the original Compact design to fully understand how the SNRDP was implemented. The evaluation team will review program documents to identify any changes made to the original design.
Evaluation Area 1 tests the economic viability of MCC-funded roads by conducting a cost-benefit analysis (CBA) to estimate the economic rate of return (ERR) and net present value (NPV) of the roads. The CBA will employ the Highway Development and Management (HDM)-4 model, an analytical tool developed by the World Bank. The post-Compact CBA will re-evaluate the validity of the initial assumptions made prior to the Compact and the economic viability of the SNRDP.
Evaluation Area 2 will evaluate the road maintenance regime within the Philippines to test the sustainability of improvement in road infrastructure. Examining the political and economic factors shaping road maintenance decisions and practices will improve MCC's assumption on post-Compact maintenance and project-life assumptions about its infrastructure investments. In particular, Evaluation Area 2 will assess whether MCC's investment in improving maintenance practices were effective in improving the Philippines' maintenance practices.
Evaluation Area 3 is a study of road users to understand the type of beneficiaries from the SNRDP. The data collected for Evaluation Area 3 will inform the HDM-4 model. Information such as the cost and duration of trips and value of goods being transported will be analyzed. This evaluation area is also intended to understand any change over time in road users and their travel patterns before and after the road improvement and how they differ among the road users.
Evaluation Area 4 is an analysis of the transportation market structure. This evaluation area will analyze transportation market structure, both formal and informal, to understand how cost savings from road improvements have passed on to transport consumers who do not own their own vehicles. The analysis of the formal and informal institutions of the transportation market will inform whether vehicle operating cost (VOC) savings are passed on to road users who do not own their own vehicle, such as farmers transporting their goods to market and public transportation users.
Below are the key evaluation questions for each evaluation area:
Evaluation Area 0
0) Were there any deviations from the original project design? [Result: Road Rehabiltation, Construction and/or Improvement]
Evaluation Area 1
1) What is the economic return of the road investment? [Result: Time Savings and Lower Vehicle Operating Costs (modelled by HDM-4), and Reduced Road Maintenance Cost]
Evaluation Area 2
2A) What are the relevant road authority's current maintenance practices and what is the likelihood that MCC's investment will remain adequately maintained for the life of the investment? Specifically, what maintenance regime reflects current practices and will therefore be applied in HDM-4? What maintenance practices most influenced your selection of this regime? [Result: Assumption: Maintenance]
2B) In cases where MCC investments included targeted maintenance improvements, how were these implemented and what were the effects of those efforts? [Assumption: Maintenance]
Evaluation Area 3
3A) Who is traveling on the road, why, what they are transporting, what they are paying for transport, and how long does it take to move along key routes? [Reduced Transportation Costs (actual), Generated and Diverted Traffic] How does road usage vary by road-user's income and gender?
3B) Have road usage patterns changed, in terms of who is traveling on the road, why, what they are transporting, what they are paying for transport, and how long it takes to move along key routes? [Results: Reduced Transportation Costs (actual), Generated and Diverted Traffic]
Evaluation Area 4
4) Given the existing transportation market structure, what portion of VOC savings will be passed on to consumers of transportation services; and if not all savings are passed on, could this project have cost effectively addressed these inefficiencies? [Result: Reduced Transportation Costs (actual)]