This evaluation is an observational study and a longitudinal, performance evaluation, meaning that we are not able to estimate a counterfactual. Instead we attempt to triangulate among diverse data sources to measure changes over time between 2014 and 2019 and then use qualitative data to consider TCP contribution to any observed changes. Data sources include (1) monitoring and administrative data, (2) documents generated by TCP implementers, (3) qualitative interviews, (4) a survey of vendors who sell goods and services to the government, (5) a survey of public employees in three key government agencies, and (6) online surveys with government officials that participated in the TCP-promoted PPP and procurement training courses.
Strengthening budget and treasury management: On the treasury aspects of the intervention, the evaluation team considers that both the Honduran treasury, General Treasury of the Republic (TGR), and partner pilot institutions (the secretariats of health, education, and public works) achieved some tangible results, but these did not translate into significant improvements in the effectiveness and efficiency of invoice processing and cash management. With TCP support, the finance ministry (SEFIN) worked to close the loopholes that allow institutions to procure goods and services without financial commitments. Nonetheless, there is still little evidence that secretariats are promptly entering and approving invoices. In addition, we do not find a significant reduction in arrears or borrowing costs by the end of the TCP. Building on the existing foundation to achieve these medium-term outcomes will require additional government efforts and technical assistance post-TCP.
On the budget aspects of the intervention, TCP support has focused on supporting a medium-term approach to budgeting, known as a medium-term expenditure framework (MTEF). These efforts started on a strong footing with local ownership and consistency with International Monetary Fund (IMF) requirements. With TCP support, the GoH has consolidated a Medium-Term Macroeconomic and Fiscal Framework (MTMFF), a key part of the MTEF, and introduced “baseline budgeting,” an approach to better predict revenue needs, which was rolled out to 43 institutions (out of a total of 109). Nonetheless, these baseline budgets are not reconciled with top-down budget ceilings. In addition, SEFIN is currently working to improve “programmatic budgeting,” prior to further advancing baseline budgeting and the incomplete MTEF. Similarly, the TCP made progress in developing SEFIN capacity to conduct Fiscal Impact Analysis, but application remains limited.
Improving procurement capacity: This activity has been one of the more successful parts of the TCP. With TCP support, the Regulatory Office of Contracting and Acquisitions of Honduras (ONCAE) established an evaluation unit and conducted 21 evaluations of GoH institutions, successfully implemented 180-hour procurement training courses to 195 participants, created a process for certifying public purchasers, increased offerings in the e-catalogue, and took steps towards strengthening the overall regulatory framework. It is not clear that ONCAE and the GoH will continue to build upon TCP-promoted efforts, however. The evaluation unit lost momentum with the departure of staff and one of the TCP-supported consultants, and the evaluations were not used to incentivize change in assessed institutions. While universities still offer the procurement course, high cost remains a potential barrier. The status of the certified public purchasers remains uncertain and will depend on future legal reforms. The e-catalogue has likely reduced costs, but framework agreements need to better maximize value for money. In addition, an effort to update the procurement platform HonduCompras has not moved forward as planned.
Building the capacity of the Supreme Audit Tribunal (TSC): With TCP support, the TSC produced five pilot performance audits. Of the four assessed, the evaluation team can only confirm that one led to meaningful change. In the absence of a stronger follow-up process, use of sanctioning authority, broader dissemination, or civil society or media interest, performance audits might not have an adequately strong theory of change to affect reform. While some form of performance auditing appears likely to continue in the TSC, it is unlikely to be a major focus. The TCP also supported efforts to improve TSC investigations of illicit enrichment and made some advances in the investigations process. However, given the low number of investigations, long bottlenecks, and few cases referred to prosecutors, among other factors, the TSC remains unable to make a meaningful dent in the issue of illicit enrichment.
Grants for social accountability: The TCP awarded grants to four Honduran civil society organizations (CSOs) to conduct citizen oversight and monitoring of government. The evaluation focused on the main grant to the Association for a More Just Society (ASJ) to undertake social audits of key government institutions, an innovative intervention that combined bottom-up accountability with top-down presidential support. The intervention appears to have had a positive impact on how procurement and human resources processes are carried out. Yet, both the qualitative and survey data suggest that ASJ's impact has been somewhat limited by larger governance challenges, including dependence on agency leadership, turnover in personnel, and patronage-based administrative staff appointments. ASJ intends to continue its activities beyond the TCP and has started on a fourth round of evaluations, assuring a degree of sustainability despite concerns over funding.
Improving the capacity for and design and implementation of PPPs: The TCP deployed several international advisors, who provided extensive technical assistance and support to varying degrees on approximately 11 PPPs. However, the TCP was unable to gain adequate cooperation from the key PPP stakeholder COALIANZA (Commission for the Promotion of Public-Private Partnerships or from conceding authorities. This severely limited the activities that could be undertaken and the reach of those activities. As a result, technical assistance to the public works agency, Secretariat of Infrastructure and Public Services (INSEP), did not lead to meaningful changes in the institution, legacy contracting problems could not be resolved, and no TCP-supported PPPs can be held up as a model. The TCP adapted by concentrating its support on supporting a Fiscal Contingencies Unit (UCF), carrying out a more general training course for GoH personnel, and identifying opportunities to improve PPPs and their governance where possible. In addition, after INSEP's continual delay in establishing a PPP unit, the GoH transferred INSEP's role as the roads contract management authority to Strategic Investment for Honduras (Inversión Estratégica de Honduras - INVEST-H), a government institution that originated as and still partially functioned as MCC's counterpart: Millennium Challenge Account-Honduras (MCA-H), which was originally established as part of MCC's Honduras Compact (2005-2010). Overall, MCC and MCA-H pressure on the government has led to a recognition that the existing PPP regime is in need of radical reform. In 2019, the GoH decided to undertake a full-scale review of the national PPP program with support from the Inter-American Development Bank (IADB), which led to a resolution of the Council of Ministers identifying the changes that would be introduced. In short, while the PPP activity may not be a success for what it has achieved directly, it might be a success for preventing future liabilities.