Позитивные изменения. Том 2, № 2 (2022). Positive changes. Volume 2, Issue 2 (2022) - Редакция журнала «Позитивные изменения»

the box one by one to ensure they match the applicants who did not turn up for the draw. If N spots were available for the program, the applicants having drawn the lowest numbers were selected for the program. The draw was organized separately for men and women. The public lottery process was well accepted by participants, and helped provide an image of fairness and transparency to the program in a post-conflict environment marked by social tensions. After several years of operations, researchers used this allocation rule, already integrated in the program, to conduct impact evaluation.

In Niger, the government started to roll out a national safety net project in 2011 with support from the World Bank. Niger is one of the poorest countries in the world, and the population of poor households eligible for the program greatly exceeded the available benefits during the first years of operation. Program implementers relied on geographical targeting to identify the departments and communes where the cash transfer program would be implemented first. This was feasible, as data was available to identify relative poverty or vulnerability status of the various departments or communes. However, within communes, very limited number of people could enroll in the program based on objective criteria. For the first phase of the project, program implementers decided to use public lotteries to select beneficiary villages within targeted communes. This decision was made in part because the available data to prioritize villages objectively was limited, and in part because an impact evaluation was being embedded in the project. For the public lotteries, all the village chiefs were invited in the municipal center, and the names of their villages were written on a piece of paper and put in a box. A child would then randomly draw beneficiary villages from the box until the quotas were filled. The procedure was undertaken separately for sedentary and nomadic villages to ensure representation of each group. After villages were selected, a separate household-level targeting mechanism was implemented to identify the poorest households, which were later enrolled as beneficiaries. The transparency and fairness of the public lottery was greatly appreciated by the village authorities, as well as by program implementers — so much so that the public lottery process continued to be used in the second and third cycle of the project to select over 1,000 villages throughout the country. Even though public lottery was not necessary for an impact evaluation at that point, its value as a transparent, fair, and widely accepted operational tool to allocate benefits among equally deserving populations justified its continued use in the eyes of program implementers and local authorities.

Sources: Bertrand, Marianne, Bruno Crépon, Alicia Marguerie, and Patrick Premand. Impacts à Court et Moyen Terme sur les Jeunes des Travaux à Haute Intensité de Main d’oeuvre (THIMO): Résultats de l’évaluation d’impact de la composante THIMO du Projet Emploi Jeunes et Développement des compétence (PEJEDEC) en Côte d’Ivoire. Washington, DC: Banque Mondiale et Abidjan, BCPEmploi. 2016

Premand, Patrick, Oumar Barry, and Marc Smitz. "Transferts monétaires, valeur ajoutée de mesures d’accompagnement comportemental, et développement de la petite enfance au Niger. Rapport descriptif de l’évaluation d’impact à court terme du Projet Filets Sociaux." Washington, DC: Banque Mondiale. 2016.

WHY DOES THE RANDOMIZED ASSIGNMENT METHOD WORK WELL?

As discussed, the ideal comparison group should be as similar as possible to the treatment group in all respects, except with respect to its participation in the program that is being evaluated. When we randomly assign units to treatment and comparison groups, that randomized assignment process in itself will produce two groups with a high probability of being statistically identical — as long as the number of potential units to which we apply the randomized assignment process is sufficiently large.

Figure 3 illustrates why randomized assignment produces a comparison group that is statistically equivalent to the treatment group.

To estimate the impact of a program using randomized assignment, we simply take the difference between the outcome under treatment (the mean outcome of the randomly assigned treatment group) and our estimate of the counterfactual (the mean outcome of the randomly assigned comparison group). We can be confident that our estimated impact constitutes the true impact of the program, since we have eliminated all observed and unobserved factors that might otherwise plausibly explain the difference in outcomes.

Inserts 2 and 3 give real-world applications of randomized assignment to evaluate the impact of a number of different interventions around the world.

Insert 2: RANDOMIZED ASSIGNMENT AS A PROGRAM ALLOCATION RULE: CONDITIONAL CASH TRANSFERS AND EDUCATION IN MEXICO

The Progresa program, now called "Prospera," provides cash transfers to poor mothers in rural Mexico conditional on their children's enrollment in school and regular health checkups. The cash transfers, for children in grades 3 through 9, amount to about 50 percent to 75 percent of the private cost of schooling and are guaranteed for three years. The communities and households eligible for the program were determined based on a poverty index created from census data and baseline data collection. Because of a need to phase in the large-scale social program, about two-thirds of the localities (314 out of 495) were randomly selected to receive the program in the first two years, and the remaining 181 served as a comparison group before entering the program in the third year. Based on the randomized assignment, Schultz (2004) found an average increase in enrollment of 3.4 percent for all students in grades 1–8, with the largest increase among girls who had completed grade 6, at 14.8 percent. The likely reason is that girls tend to drop out of school at greater rates as they get older, so they were given a slightly larger transfer to stay in school past the primary grade levels. These short-term impacts were then extrapolated to predict the longer-term impact of the Progresa program on schooling lifetime and earnings.

Source: Schultz, Paul. "School Subsidies for the Poor: