The evaluation employs a mixed-methods approach in which qualitative techniques and quantitative analysis support each other, recognizing that the techniques used will depend on the evaluation question to be addressed.
The source of information for the qualitative analysis is through Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs) with the household and conservancy or PPO member-households and management, as well as with stakeholders in the tourism sector from the private-sector and associated regulatory bodies.
In the case of the quantitative analysis, control groups are not available for the evaluation of either the CS activity or INP sub-activities. The CS activity is taking place in most of the conservancies of the Northern Communal Areas (NCAs), which were selected for their tourism potential. Conservancies outside of this activity are generally in areas with differing natural endowments and market access and, as such, cannot serve as a comparable set of non-intervention conservancies. In the case of the INP sub-activities, it is not feasible to establish a valid comparison group because the intervention covers nearly the entire INP producer population. Instead, a type of a reflexive (before-and-after) design called a dose-response model is employed whereby each conservancy or PPO at baseline contributes to our understanding of the counterfactual by allowing us to infer whether differences in the amount of Compact assistance (the "dosage") influence and, therefore, impact on CS or INP performance.
The model identifies likely program impacts by estimating the marginal effects of different intervention levels (e.g., intensity of training or number and type of grants) on outputs and outcomes of interest at critical points along the causal chain from the short to medium run. Originally, program impact on household income, the ultimate expected result by MCA-N, was to be a focus of examination, but it is now accepted that such changes would not likely be large enough to be detected over the relatively short evaluation period.