Activity 1:
We will estimate Activity 1's causal impacts using a randomized controlled trial. The Ministry of Education (MINEDUC) grouped all eligible schools within the five study departments into school districts-clusters of neighboring schools with eight schools each, on average. We randomly assigned 80 percent of those districts to a treatment group, in which schools would be invited to participate in Activity 1. We assigned the remaining 20 percent of districts to a control group, in which the implementer would not intervene.
We will gather baseline and endline data from a randomly selected sample of schools from the treatment and control groups. We will estimate Activity 1's causal effects by comparing outcomes from the two groups at endline and controlling for chance differences observed at baseline. We will report two types of estimates: intent-to-treat estimates, which reflect differences between treatment and control schools regardless of their participation in treatment, and local average treatment effects, which are adjusted to account for participation rates in both groups. We describe our sample selection, randomization, and data collection methods elsewhere in sampling and data collection methods sections.
We will describe the implementation of the activity and identify its facilitators and barriers via an implementation study. We describe our sample selection and data collection methods in the sampling and data collection methods sections.
Activity 3:
We will conduct a trend analysis to understand changes in key outcomes related to this activity. This approach will not show the activity's causal impacts, but we will combine the trend analysis with an analysis of related concurrent events to see the potential relationship between the activity and outcomes of interest.
We will also conduct a qualitative analysis, relying on focus groups, interviews, and implementation documentation. We will use the drivers-of-change (DOC) framework and political economy methods to structure the qualitative component. The DOC framework is an analytic framework for applying political economy analysis, which enables evaluators to systematically assess how project design and implementation decisions addressed contextual factors that may affect whether project goals are met. Specifically, the DOC framework calls for an analysis of the relationship between structural features (such as the structure and history of education in Guatemala and social and demographic trends), institutions (such as the legal framework, government policies, and informal rules that affect behavior), and agents (such as organizations and people who influence and participate in lower-secondary education).
Note:
Under Evaluation Methodology, we selected Randomization, which is the method for the impact evaluation of Activity 1. We will also conduct implementation evaluations of Activities 1 and 3 and political economy analyses of Activity 3.
Under data type, we selected Sample Survey Data, which is the main method for the impact evaluation of Activity 1. For both activities, we will also use administrative data, monitoring and evaluation data from the implementer, and qualitative data.