Page 1 of 61
VOTER OPINION AND INVOLVEMENT THROUGH CIVIC EDUCATION (VOICE) 
The VOICE Impact Evaluation in the Democratic Republic of the Congo 
Final Report 
USAID Contract No: DFD-I-00-05-00225-00, Task Order Number 8 
APRIL 2012
This publication was produced for review by the United States Agency for International 
Development. It was prepared by the International Foundation for Electoral Systems.
The VOICE Impact Evaluation 
in the Democratic Republic of the Congo: 
Final Report 
Submitted to IFES by: 
Steven E. Finkel, PhD 
Department Chair and Daniel H. Wallace Professor 
Department of Political Science 
University of Pittsburgh 
finkel@pitt.edu
April 6, 2012 
ii
The VOICE Impact Evaluation in the Democratic Republic of the Congo: Final Report 
Table of Contents 
Executive Summary .........................................................................................................................iii
Chapter I. The Voice Impact Evaluation in the Democratic Republic of Congo ............................. 1
A. Study Introduction and Goals ................................................................................................ 1
B. The Boîtes à Images Sessions................................................................................................. 2
C. Framework for the Evaluation ............................................................................................... 5
Chapter II. Study Methodology ...................................................................................................... 7
A. The Research Design .............................................................................................................. 8
1. Comparison of Individuals in Treatment and Control Villages .......................................... 8
2. The Randomized “Encouragement” Component ............................................................ 11
B. Survey Instrument and Variables Used in the Analysis ........................................................ 13
C. Field Work ............................................................................................................................ 18
D. Statistical Analyses ............................................................................................................... 18
1. Estimating Village-Level Effects of the Boîtes Sessions ................................................... 18
2. Estimating the Effects of Attending Boîtes Sessions ........................................................ 21
3. Additional Explorations .................................................................................................... 24
Chapter III. The Impact of the Boîtes à Images Sessions: Basic Results ...................................... 26
A. Subjective Assessments of Session Quality ......................................................................... 26
B. Village-Level Effects of the Boîtes Sessions .......................................................................... 28
C. The Effects of Attending Boîtes Sessions ............................................................................. 34
D. “Spillover” Effects of the Boîtes Sessions on Treatment Village Non-Attendees ................. 36
E. Effects of Boîtes Session Quality and Timing ....................................................................... 38
1. Session Quality .................................................................................................................. 38
2. Session Timing.................................................................................................................. 42
F. Effects of Boîtes Sessions on Different Kinds of Individuals ................................................. 46
Chapter IV. Conclusions ................................................................................................................ 48
A. Major Findings ..................................................................................................................... 49
B. Implications for Civic Education Design, Implementation, and Evaluation ......................... 51
References ..................................................................................................................................... 54
iii
Executive Summary 
This document reports the results of the Voice Impact Evaluation, a study conducted by IFES to 
assess the effects of its Voter Opinion and Involvement through Civic Education Program (VOICE) 
in the Democratic Republic of the Congo. The VOICE program was designed to improve the 
capacity of the Congolese people to participate effectively in the ongoing process of 
decentralization in the DRC. This process, mandated by law in 2007, calls for the reorganization 
of the country’s administrative units, and increases in the powers of Local and Provincial 
governments in order to bring greater accountability and inclusion of subnational units in the 
country’s political, economic, and social development. 
VOICE consisted of a range of activities aimed at enabling ordinary Congolese citizens to better 
understand and engage in the decentralization process, motivating individuals to participate in 
government and politics more generally, and providing community-based organizations the 
capacity to implement civic and voter education campaigns. The focus of this evaluation is on 
the central civic education-related activity of the VOICE program, the Boîtes à Images
sensitization sessions in which facilitators from local community-based organizations presented 
community audiences with a variety of images that were designed to stimulate discussion and 
learning about decentralization and, to a lesser extent, broader issues of political, economic, and 
democratic development. These sessions, lasting approximately two hours with audiences of 
approximately 100 individuals, were conducted throughout 2010 and 2011 in four target 
provinces --- South Kivu, Maniema, Kantanga, and Bandundu. Due to budget limitations, our 
evaluation is limited to assessing the impact of the Boîtes à Images sessions conducted during 
the summer of 2011 in Bandundu province. 
The evaluation was designed primarily to determine the extent to which the Boîtes à Images
sessions led to changes in ordinary citizens’ knowledge and attitudes about decentralization and 
democratic politics more generally, and whether the sessions prompted greater cognitive and 
behavioral engagement with the Congolese political process. Moreover, it sought to assess the 
conditions under which the sessions had greater or lesser impact, focusing on a number of 
factors that potentially affect the impact that the Boîtes sessions had on individuals, including 
the quality of the instruction and organization of the event, the length of time since the Boîtes
session took place, and the individuals’ own prior political, social, and informational resources. 
Finally, the evaluation sought to determine the relative magnitude of direct effects on 
individuals who may have attended the sessions, as well as indirect or “spillover” effects on non￾attendees in treatment villages who may have learned about decentralization issues through 
discussion with others in their social networks who did attend the Boîtes events. 
To accomplish these task, we implemented a complex research design with two components: 
first, a comparison between respondents from 8 villages in Bandundu province where Boîtes à 
Images s sessions took place and 8 villages from similar “control villages” in neighboring 
groupements where sessions did not take place; and second, a field-experimental 
“encouragement design” whereby 100 individuals in each of 8 villages were randomly 
encouraged at the end of a baseline interview to attend the upcoming Boîtes session, while 40 
randomly-selected individuals in those “treatment villages” received no information about the 
workshop. Follow-up interviews were conducted with all respondents between one day and 
one month after the workshops in the treatment villages took place. The study is one of the 
first evaluations of adult civic education programs to make use of experimental methods, which 
iv
provide greater confidence in the attribution of causal effects than has been possible in the 
mostly passive observational designs in previous research in the field. To our knowledge, it is 
the first that implements an “encouragement manipulation”, one that preserves the benefits of 
experimentation in a situation where it is not possible to randomly assign individuals to 
experience a treatment, nor to restrict exposure to those specifically chosen individuals. 
The study points to a series of important findings about the kinds of effects that the Boîtes à 
Images sessions had, and the conditions under which the sessions had their strongest impacts. 
There were large differences between individuals in control and treatment villages in terms of 
their changes in factual knowledge about decentralization before and after the Boîtes events 
took place in the treatment villages. While respondents in control villages increased to some 
extent on decentralization-related knowledge (with the percentages of correct responses 
increasing between 5-10% for most questions), respondents in treatment villages increased at 
substantially higher rates (between 12 and 45% increases in correct responses for each 
question). On an overall scale of “correct decentralization knowledge” (ranging from 0 correct 
to 6 correct answers), the average respondent in control villages increased by only .45 correct 
answers over time, while the average respondent in treatment villages increased by 1.5 correct 
answers from the pre- to post-Boîtes interviews. These effects remains strong in the context of 
regression models where the treatment and control samples are carefully balanced on possible 
confounding variables, and in models that control for the individuals’ pre-Boîtes levels of the 
decentralization knowledge. These effects are impressive, especially given that they stem from a 
single civic education event conducted at the village level. To the best of our knowledge, these 
are some of the largest effects registered in the several adult civic education evaluation studies 
previously conducted, and certainly the largest registered in these studies on political 
knowledge (Finkel 2003; Finkel and Smith 2011; Finkel et al. 2012). 
We also found village-level effects of the Boîtes sessions on other knowledge-related items, such 
as the individual’s general knowledge about DRC political institutions and actors, and on the 
individual’s perception of their knowledge about decentralization and the DRC constitution. 
These effects were of smaller magnitude than those observed on the general decentralization 
knowledge scale, but they do indicate that the Boîtes sessions were beneficial to some degree in 
stimulating general political learning, and in stimulating greater confidence among individuals in 
their own store of decentralization and constitutional knowledge. 
However, we found few corresponding effects on individual attitudes about, or support for, the 
decentralization process. There were no differences between the changes in support for 
decentralization among respondents in control villages and respondents in treatment villages, 
no differences between treatment and control respondents in terms of their perceptions of the 
“good” and “bad” features of decentralization, or their satisfaction with the current pace of the 
decentralization process. Individuals in treatment villages, moreover, tended to change slightly 
in the direction of greater, not lesser, national authority in a series of policy areas (roads and 
transportation, schools and education, police and military, taxation), compared to comparable 
individuals in control villages. There were also no effects of the Boîtes session on more general 
orientations about democracy, on political interest, discussion, or efficacy, nor on political 
participation. The effects of the Boîtes events appear to have been more or less exclusively 
concentrated in the realm of decentralization and general political knowledge, with no 
concomitant effects on decentralization attitudes or other political orientations or behaviors. 
v 
We exploited the “randomized encouragement” feature of the design in order to determine the 
direct effects of attending the Boîtes event among individuals in treatment villages. Using a 
variety of “intent to treat” and “instrumental variables” models to overcome possible biases 
related to self-selected exposure to the treatment, we found consistent evidence that Boîtes 
attendees registered large and significant gains on decentralization knowledge, relative to non￾attendees, with these effects being even larger than the overall village effects mentioned above. 
Boîtes attendees also were found to have gained significantly on the other variables (general 
knowledge and perceptions of decentralization and the DRC constitution, support for national 
authority in roads, police and military affairs) shown to be significant at the village level as well. 
Further analysis showed minimal “spillover effects” from attendees to others in their family, 
neighborhood or friendship networks. Taken together, the results suggest that virtually all of 
the village-level impacts on the knowledge and decentralization orientations were concentrated 
among those who attended the Boîtes sessions. It seems reasonable to speculate that the 
single, relatively brief Boîtes exposure was not sufficient to stimulate the kind of post-treatment 
diffusion via interpersonal discussion necessary for spillover effects to occur, especially given 
the breakdown of the decentralization process at the elite level and its relatively low salience in 
the DRC at the time the study was conducted. 
We further examined the conditions under which the Boîtes sessions produced larger impacts. 
First, the quality of the Boîtes session mattered a great deal. Events with higher reported 
satisfaction of the participants regarding the organization, information delivery and competence 
of the facilitators showed substantially stronger effects on decentralization knowledge and 
many of the other knowledge-related items of lesser perceived quality. Importantly, individuals 
who attended the highest quality events often showed significant increases in the cluster of 
orientations related to political engagement: interest, efficacy, political discussion and 
participation, relative to comparable individuals in control villages. As was found in previous 
research, however, the numbers of individuals in DRC treatment villages who were trained at 
the highest levels of session quality were relatively small. Thus, the results indicate both the 
strong potential for effects from high quality civic education, as well as the more limited actual
impacts the sessions had on decentralization and political engagement outcomes. 
Second, the time since the event mattered as well. Treatment village respondents interviewed 
closer to a month after the Boîtes event showed substantially less improvement in 
decentralization knowledge, relative to comparable individuals in control villages, than 
respondents interviewed in days immediately following the event. Treatment effects decline by 
over 60% from the day after the event to 26 days later on the key variable of decentralization 
knowledge, with even steeper declines on the other knowledge-related political engagement 
orientations. This pattern is consistent with a “fade-out” effect, such that individuals are 
forgetting after several weeks a large portion of what they may have learned in the days 
following the treatment. We cannot show this conclusively with the data at hand, however, 
since individuals interviewed at different times after the sessions had been exposed to different 
treatments. Nevertheless, this pattern shows strong support for the notion — consistent with 
previous work — that the single-shot Boîtes treatments may not have produced lasting impact 
on the orientations they were designed to affect. 
Finally, we found no consistent evidence that the effects of the Boîtes sessions differed across 
individuals in treatment villages with different demographic, political, or social characteristics. 
There were similar effects of the Boîtes session for men and women, for individuals with 
vi
different educational attainment, for individuals with varying levels of secondary group 
membership, and for individuals who had and had not lost family members in the recent DRC 
armed conflict. However, we did find suggestive evidence that the Boîtes treatment had greater 
effects on those with lower initial levels of decentralization and general political knowledge. In 
every instance, knowledge gains from the Boîtes events were greater among those with the 
greatest “need” for the information. 
Based on these findings, we propose the following recommendations for future civic education 
design, implementation, and evaluation in the DRC: 
 Future civic education programs should continue to use “Boîtes à Images” and other 
visual materials. 
 Future programs must go beyond “one-off” village-level treatments in order to maximize 
individual level impact. 
 Future programs must give greater emphasis to the training of facilitators and to 
improving the general organization of civic education treatments.
 Future programs should emphasize post-treatment discussions of participants to 
maximize the possibilities of spillover effects.
 Future evaluations should continue to exploit experimental designs in order to make 
more credible causal inferences about program effects, while recognizing their potential 
limitations in certain settings as well.
1 
Chapter I. The Voice Impact Evaluation in the Democratic Republic of Congo 
A. Study Introduction and Goals 
This document reports the results of the Voice Impact Evaluation, a study conducted by IFES to 
assess the effects of its Voter Opinion and Involvement through Civic Education Program (VOICE) 
in the Democratic Republic of the Congo. The VOICE program was designed to improve the 
capacity of the Congolese people to participate effectively in the ongoing process of 
decentralization in the DRC. This process, mandated by law in 2007, calls for the reorganization 
of the country’s administrative units, and increases in the powers of Local and Provincial 
governments in order to bring greater accountability and inclusion of subnational units in the 
country’s political, economic, and social development. 
VOICE consisted of a range of activities aimed at enabling ordinary Congolese citizens to better 
understand and engage in the decentralization process, motivating individuals to participate in 
government and politics more generally, and providing community-based organizations the 
capacity to implement civic and voter education campaigns. The focus of this evaluation is on 
the central civic education-related activity of the VOICE program, the Boîtes à Images
sensitization sessions in which facilitators from local community-based organizations presented 
community audiences with a variety of images that were designed to stimulate discussion and 
learning about decentralization and, to a lesser extent, broader issues of political, economic, and 
democratic development. These sessions, lasting approximately two hours with audiences of 
approximately 100 individuals, were conducted throughout 2010 and 2011 in four target 
provinces --- South Kivu, Maniema, Kantanga, and Bandundu. Due to budget limitations, our 
evaluation is limited to assessing the impact of the Boîtes à Images sessions conducted during 
the summer of 2011 in Bandundu province. 
The evaluation was designed primarily to determine the extent to which the Boîtes à Images
sessions led to changes in ordinary citizens’ knowledge and attitudes about decentralization and 
democratic politics more generally, and whether the sessions prompted greater cognitive and 
behavioral engagement with the Congolese political process. Moreover, it sought to assess the 
conditions under which the sessions had greater or lesser impact, focusing on a number of 
factors that potentially affect the impact that the Boîtes sessions had on individuals, including 
the quality of the instruction and organization of the event, the length of time since the Boîtes
session took place, and the individuals’ own prior political, social, and informational resources. 
Finally, the evaluation sought to determine the relative magnitude of direct effects on 
individuals who may have attended the sessions, as well as indirect or “spillover” effects on non￾attendees in treatment villages who may have learned about decentralization issues through 
discussion with others in their social networks who did attend the Boîtes events. 
These goals were accomplished by conducting interviews (through BERCI, a respected Congolese 
survey company) with a random sample of 1,120 individuals in 8 “treatment” villages where 
Boîtes à Images sessions took place, and comparing their responses to those of a random 
sample of 480 individuals in 8 comparable “control” villages where Boîtes sessions did not take 
place. Baseline interviews were conducted with all respondents before the sessions took place 
in treatment villages, and all respondents were re-interviewed some days or weeks after the 
Boîtes sessions took place. The survey instrument contained an extensive battery of questions 
to measure the individuals’ factual knowledge of the decentralization process and of political 
2 
institutions in the DRC, support for decentralization and for changes in the authority of local 
governments in various policy areas, support for democratic values and processes, perceived 
political knowledge and competence, and levels of political discussion and participation. In 
“treatment” villages, individuals were also asked in the follow-up interview about whether they 
had attended the Boîtes à Images session that had recently taken place, and, if so, about their 
experiences and satisfaction with the event. 
The study also represents a major advance over previous work evaluating the impact of adult 
civic education programs in developing democracies that have been conducted by the author 
and others over the past decade and a half (e.g., Bratton et al. 1999; Finkel, 2002; 2003, Finkel 
and Smith 2011, Finkel et al. 2012). All of the previous evaluations of civic education impact 
have been based on observational research designs, i.e., with data collected on non-randomized 
treatment and control groups, often at one point in time long after the activities have taken 
place. These studies have provided valuable information, but the observational nature of the 
designs means that one cannot rule out the possibility that the “self-selection” of individuals 
into the civic education workshops led to biased inferences about the causal effects of the 
treatments. The current study includes a randomized experimental component, whereby 
randomly-selected individuals in the treatment villages were “encouraged” in the baseline 
interview to attend the upcoming Boîtes event. As we discuss in chapter II, this experimental 
manipulation gives us leverage with which to make inferences about the effects of exposure to 
the Boîtes sessions with much greater confidence than has been possible in previous civic 
education research (for other field experimental work on civic-education related topics, see 
Paluck and Green 2010, Fafchamps and Vicente 2010; Collier and Vicente 2009).
The report is organized as follows. In the remainder of this chapter, we discuss the content of 
the Boîtes à Images “treatment” and our framework for analyzing its potential effects. In 
Chapter II, we discuss the study’s methodology, including the sampling design, the experimental 
manipulation, the survey instrument and details about the field work, and the statistical 
methods that we utilize to estimate treatment effects. Chapter III presents the results of the 
analyses. Chapter IV concludes and draws implications for the implementation of future 
decentralization and other civic education programs in the DRC, as well as implications for the 
design of future impact evaluations in the field. 
B. The Boîtes à Images Sessions 
In 2007 the Government of the Democratic Republic of Congo (DRC) committed to a process of 
decentralization, which is to entail the passage of a law on so-called Entités Territoriales 
Decentralisées (ETDs), creating 26 provinces from the existing 11 and more than 6,000 
subprovincial electoral constituencies from the existing 189. The motivations behind 
decentralization are to bring greater accountability to existing governmental institutions, to 
reduce levels economic and political inequality across different geographic regions, to enhance 
the inclusion of broader strata of Congolese civil society, and to provide greater input from 
communities into decisions related to economic development in their localities (Dizolele 2010). 
Over the past several years, the process advanced fitfully, and at the moment the ongoing 
political crisis and tensions between various factions within the DRC government have 
effectively stalled formal implementation. Nevertheless, at the time of the study (summer 
2011) the subject was at least on the agenda of DRC civil society, and officially on the 
government’s agenda as well.
3 
The VOICE project initiated by IFES employed a wide range of tools designed to promote greater 
citizen involvement and understanding of the decentralization process, including information 
sheets, topical roundtables, and targeted information towards different demographic segments 
of the Congolese population. The central tool was the Boîtes à Images community workshop, in 
which facilitators used a series of “picture boxes” to illustrate aspects of decentralization and 
more general political and economic processes to audiences of approximately 100 persons in 
villages throughout the country. The use of images as a civic education delivery mechanism was 
designed specifically by IFES as a means to convey messages in the DRC context, where adult 
literary reaches only 67% in large cities and is likely far less in rural and more remote 
populations. According to IFES materials provided to Boîtes facilitators during their training 
sessions, the images were designed to be “a tool that triggers dialogue”, one that “encourages 
participants to share what they know, hear, live and understand about decentralization” in “a 
space where men and women express themselves freely”. To this extent, the Boîtes sessions 
were designed to be highly participatory forums where active learning took place, stimulated by 
the specific images about decentralization that were presented. The Boîtes sessions lasted 
approximately two hours each. 
The specific 13 Boîtes à Images were organized into more general “modules”, corresponding to 
different aspects of the decentralization process and different aspects of civic education 
emphasized by the VOICE programs. VOICE documents describe the modules as follows: 
Module 1: “Let’s understand our new institutions”. Introduce concepts of local governance and 
importance of participation in local elections and government; explains decentralized 
institutions and decentralization law (as available); responsibilities of urban and municipal 
counselors; responsibilities of sector and chefferie counselors. 
Module 2: “Let’s be a part of the new Congo”. Introduces the roles of the actors in the process 
(election commission, politicians, political parties and opposition, judges, civil society); explain 
rationale for decentralization and benefits of active participation in civic life. 
Module 3: “Let’s work together for peaceful and fair local elections”. Introduce procedures for 
local elections; explain voting procedures, and encourage peaceful conduct. 
The images that formed the basis of this phase of the VOICE program primarily emphasized 
topics in Modules 1 and 2, with less material related to the theme of local elections, as those 
issues are to be covered more thoroughly in subsequent phases of VOICE. Module 1’s theme of 
understanding new DRC political institutions is illustrated by two images reproduced below: 
Image #2 is entitled “Decentralization in the Democratic Republic of the Congo”, and shows the 
map of the 11 current provinces and a large arrow pointing to the envisioned 26 provinces 
(including Kinshasa) post-decentralization. Image #8, entitled “Transfer of Resources”, depicts 
the revenue flow envisioned in a decentralized DRC from the local Entités Territoriales 
Decentralisées (ETDs) to the central government and back again, with some of the funds to the 
localities coming from the newly-created Caisse Nationale de Péréquation, an institution that 
will provide funds for local development in order to equalize resources across different 
geographic regions. Module 2’s theme of political participation is exemplified in Image #11 
(shown below), “The Role of the Citizen”, which depicts ordinary individuals engaging in various 
acts of political participation, including attending community meetings, submitting a petition to 
4 
an elected official, and participating in a peaceful demonstration. Module #3’s emphasis on 
local elections is illustrated in Image #12 below, “The Selection of Leaders”, which depicts 
individuals going to an enrollment center with their required identification documents in order 
to obtain new voter registration cards. All of the images are accompanied by sample questions 
to pose to the audience in order to stimulate discussion and learning (e.g. “ask participants to 
share their experiences in participating in peaceful protests”, “ask participants if they know how 
much revenue their entity mobilizes and what efforts have been made to develop their 
communities”, “ask participants to share lessons learned from previous elections operations”). 
All 13 images, along with the discussion guidelines and learning objectives for each one, are 
reproduced in Appendix A of this report. 
Image #2: Decentralization in the DRC 
Image #8: Transfer of Resources
5 
Image #12: The Selection of Leaders 
Image #11: The Role of the Citizen 
C. Framework for the Evaluation 
The Boîtes à Images sensitization sessions sought to increase individuals’ level of awareness and 
knowledge of decentralization, political institutions and political actors, increase individuals’ 
awareness of their role in the political process, promote more informed and effective citizen 
participation, and promote the values and dispositions that would facilitate individuals’ 
contributing to the political and economic development of the Democratic Republic of the 
6 
Congo. An assessment of the impact of the sessions, therefore, should focus on these specific 
orientations and how they may have changed as a result of an individual attending a Boîtes 
session themselves, or as a result of a Boîtes session taking place in their village. Further, the 
assessment should examine the conditions under which the Boîtes sessions led to more or less 
impact on individuals, and the kinds of individuals who responded most strongly to the 
messages conveyed. The objectives of the study, then, are to determine whether the Boîtes à 
Images sensitization sessions were successful in changing orientations related to 
decentralization knowledge and awareness, civic skills, democratic values, and political 
engagement, and if so, under what conditions, and for which kinds of individuals. 
Most of the orientations that the Boîtes sessions sought to influence — including awareness of 
politics, skills, support for democratic values, and political participation — are well known to 
political scientists as precisely those aspects of public opinion that are most important for the 
development and consolidation of democratic political culture. They have been researched 
extensively in both established and emerging democratic systems for over fifty years, and have 
also been examined specifically in the context of assessing the impact of civic education 
programs in many settings (e.g., Diamond 1999; Almond and Verba 1963; Finkel and Smith 2011; 
Finkel et al 2012). Hence, we have a good deal of knowledge of exactly how these orientations 
should be conceptualized and measured, and we include indicators used in the past for many of 
these attitudes, values, and beliefs in the current study. We also develop new questions as 
appropriate for the DRC context, in particular in gauging citizens’ knowledge, awareness and 
attitudes about the decentralization process, a topic which, to the best of our knowledge, has 
received very little attention in political attitude research and no attention whatsoever in 
previous civic education evaluation research. 
The study seeks to determine, then, the effects of the Boîtes à Images sessions on the following 
orientations: 
 Knowledge about the Decentralization Process in the DRC; 
 Knowledge about Political Institutions, Actors and Processes in the DRC; 
 Attitudes about Decentralization, including perceived benefits of the process, support 
for the process, and support for decentralized authority in specific policy areas; 
 Political Interest and Political Efficacy, or perceptions of individuals of their political 
competence, and their ability to participate effectively in political life; 
 Political Participation; 
 Support for Democratic Processes and Values, such as satisfaction with democracy and 
political tolerance 
The study further seeks to assess the conditions under which the effects of the Boîtes sessions 
were stronger or weaker, focusing specifically on two general sets of factors shown to be 
relevant in previous evaluations. One set concerns the quality and timing of the sessions 
themselves. Previous work has found substantial differences in the effects of civic education 
exposure, to the extent that workshops or other events are better organized, and led by more 
competent, knowledgeable, and likeable facilitators (e.g., Finkel and Smith 2011). Moreover, it 
may be the case that some of the initial effects of civic education exposure may “fade out” over 
time. We therefore develop measures of the quality (organization, facilitator competence, and 
the like) of the Boîtes session to which individuals were exposed, and test for differential effects 
7 
of the sessions based on “session quality”, and based on the length of time that may have 
passed since the Boîtes event took place in the individual’s village. 
The second set of factors that may influence the magnitude of Boîtes effects are characteristics 
related to the individual’s political and social resources. Some previous work (e.g., Finkel and 
Smith 2011) has found greater impact of civic education on those with greater “need” for the 
information, i.e., individuals with lower levels of education, and lower levels of social and 
economic resources. We test for these kinds of differential effects in the analyses below as well. 
Finally, it is important to consider that the Boîtes à Images sessions may have effects that reach 
well beyond the 80-100 individuals who were directly exposed to the messages conveyed at the 
event itself. Individuals who did not attend the sessions may nevertheless have discussed 
decentralization-related topics with individuals who did, thereby becoming exposed to the 
messages conveyed at the Boîtes event indirectly. Recent civic education evaluations have 
shown the increasing importance of these kinds of “secondary” or “spillover” effects (Finkel and 
Smith 2011; Fafchamps and Vicente 2010), and there is strong reason to expect the same 
pattern of findings here as well. We therefore attempt to show the overall effects of the Boîtes 
sessions among individuals living in villages where the events took place, but also devote 
considerable attention to disentangling the “direct” versus “indirect” effects of the Boîtes 
sessions, on participants and non-participants, respectively. 
Chapter II. Study Methodology 
The research design of the study has two components: 1) a comparison of changes in 
decentralization orientations from a pre-event baseline survey and a post-event survey of 1,120 
randomly-selected individuals in 8 “treatment” villages in Bandundu province where Boîtes à 
Images events took place, and 480 randomly-selected individuals in 8 comparable “control”
villages where Boîtes à Images events did not take place; and 2) a randomized “encouragement” 
experiment, whereby 800 of the treatment village respondents were randomly encouraged to 
attend the upcoming Boîtes à Images event after completing the baseline survey, with their 
responses compared in several different ways to the “non-encouraged” and “control village” 
individuals in order to estimate the causal effects of the Boîtes events. As we will show, the 
design allows us to estimate the overall effect of the Boîtes sessions at the village-level 
(component #1), as well as to isolate the causal effect of attending the event itself (component 
#2). Taken together, the two components also allow us to make some headway in assessing 
“secondary” effects of the events, such that individuals who were directly exposed to the 
program may have gone on to discuss the messages of the events with others in their social 
networks. 
In this chapter, we first discuss the research design and its implementation in the field in more 
detail. We follow with a description of the survey instrument used to assess the respondents’ 
decentralization knowledge, attitudes, and political engagement (the full survey instrument is 
provided in Appendix B). We conclude with a discussion of the statistical methods used to 
estimate of the effects of the Boîtes events. As we will show, this “pre-test/post-test design with 
randomized encouragement” allows inferences about the causal effects of the Boîtes events on 
decentralization orientations to be made with greater confidence than in the mainly non￾randomized studies used in the field to date. At the same time, the design departs from an 
8 
“ideal” experimental design in several important ways, and these limitations need to be taken 
into account in evaluating the results of the study as well. 
A. The Research Design 
1. Comparison of Individuals in Treatment and Control Villages 
The most basic estimate of Boîtes à Images impact comes from a comparison of responses from 
randomly-selected individuals in villages where Boîtes events took place with randomly-selected 
individuals in comparable villages where the events did not take place. Given the potential for 
messages about the event to spread to individuals in a given “treatment” village, regardless of 
whether they personally attended the Boîtes event or not, we can estimate an overall impact of 
the event—including both “direct” effects on attendees and “indirect”, “secondary” or 
“spillover” effects on non-attendees. This is accomplished by comparing individuals in treatment 
villages with a control group consisting of individuals from comparable villages where Boîtes
events did not occur. These effects are of critical importance in evaluating the effects of the 
Boîtes events, because the sessions were designed at least in part to influence both individuals 
who attended the events as well as to stimulate changes among others in the attendees’ 
neighborhoods or social networks. 
This component of the study was thus designed to produce two samples of individuals: those 
residing in 8 villages where a Boîtes event was to take place and a comparable sample of 
individuals residing in 8 villages in nearby (but not too close) groupements where Boîtes events 
would not take place. Following the procedures discussed below, the Congolese survey 
company BERCI interviewed 140 randomly-selected individuals within each of the treatment 
villages and 60 randomly-selected individuals within control village at two points in time: once 
within one week before the Boîtes event was to take place in the treatment village, and once 
between one week and one month after the sessions. The basic test of the effects of the Boîtes
events, then, is a comparison of the changes in decentralization orientations registered by the 
1120 individuals from the treatment villages with the changes in decentralization orientations 
registered by the 480 individuals from the control villages. We call these the “village-level” 
effects of the treatment, as we are determining in these analyses the extent to which the Boîtes
event stimulated changes in decentralization orientations across the entire village, whether the 
interviewed individuals attended the event or not. These village-level results are shown in 
section B of chapter III below. 
We began by randomly selecting 8 different groupements in Bandundu province where we had 
knowledge that Boîtes events were to occur1
, and then asked the NGO that implemented Boîtes
events in that groupement to provide BERCI with a list of villages that were specifically targeted 
for an event. BERCI selected one village at random from the list to serve as the initial 
“treatment” village. 
BERCI then selected a village to serve as a “control” for each of the selected “treatment” 
villages. The control villages were to have the following characteristics: they were to be in 
another groupement in the same territory as the treatment village, sufficiently nearby to ensure 
 
1
Cost considerations prevented us from including a greater number of provinces (our initial design called for including 
villages in Bandundu, Katanga, South Kivu, and Maniema provinces) and a greater number of villages (our initial 
design called for 24 treatment and 24 control villages). 
9 
general similarities to the treatment village in terms of size, ethnicity, and socio-economic 
conditions, but sufficiently far away so that there would be little chance of “spillover” effects 
from the Boîtes event taking place in the treatment village. BERCI initially sampled the 
neighboring groupements for each treatment village and then randomly sampled villages from 
those groupements to arrive at the initial list of control villages. If the sampled village was closer 
than 10 kilometers to the treatment village, a substitute village from the same groupement was 
chosen as a replacement. This produced an initial sampling of 8 treatment and 8 control villages 
as shown in Table II-1 below. 
Table II-1 Initial Sample of Treatment and Control Villages 
TERRITOIRE GROUPEMENTS IFES Village/IFES (Treatment) POPULATION
Oshwe Badjakamba Popombo 1,540
Idiofa Ebiala Ebiala kandolo 1,325
Bagata Mbaya Salikoko Mbaya 347
Bulungu Nkata Kwanga mulungu PLZ 646
Gungu Lozo Munene Lozo Munene 2 1,718
Masi Manimba Bangalala Mutulu 343
Kasongo Lunda Pelende Tombola 1,087
Inongo Mbelo Mpole 1,398
TERRITOIRE GROUPEMENTS NON IFES Village/NON IFES (Control) POPULATION
Oshwe Etuali Bokala Beact (camp) 2,159
Idiofa Bangoli Maboko 743
Bagata Falwano Ngomuna 488
Bulungu Kikongo Kikongo 1,381
Gungu Gatundo Luvuji 1 165
Masi Manimba Kalunga Mikinzi Kingola poste 543
Kasongo Lunda Kiabamba Kahungula/Kingete 617
Inongo Bokote Bokote/Ilanga 1,073
It became apparent, however, that several of these villages were too remote for BERCI’s 
interviewers to have reasonable access, and it was also apparent that the distances between the 
“treatment” and its respective “control” village was in some cases so great that travel costs for 
the surveys would have been prohibitive. With our approval as well as the approval of the local 
IFES team, three “control” and four “treatment” villages were substituted for the original 
sampling. We show the final set of villages included in the study in Table II-2 below, with the 
substituted villages highlighted in boldface. The date that the Boîtes event took place in the 
treatment village is provided as well. 
The fact that the resultant sample of treatment and control villages is no longer strictly 
randomly selected introduces potential biases in the results, to the extent that the substituted 
villages had characteristics that differentially predisposed individuals towards positive or 
negative decentralization orientations. That is, with a pure random sample, we can be more 
confident that the treatment and control villages were matched on important factors that relate 
to decentralization orientations, and that any differences we observe in the changes on these 
10
orientations over time could be attributable to the Boîtes event “treatment”.2
 If the sample of 
treatment villages had, for example, higher levels of education or social integration than the 
sample of control villages, we could not be certain that differences in decentralization 
orientations were the result of the Boîtes sessions and not due to these other factors. 
Table II-2 Final Sample of Treatment and Control Villages 
Territoire Groupements 
IFES
Village/IFES
Treatment
Boîtes
Date
Groupement 
Non IFES
Village/Non 
IFES Control
Ifes
Estimated 
Distance
Oshwe Badjakamba Nkole Etat
(Pop. 812)
8/21/11 Etuali Bokala Wambia
(Pop. 619)
40km
Idiofa Ebiala Ebiala 
Kandolo
7/25/11 Bangoli Maboko 40km
Bagata Mbaya 
Salikoko
Mbaya 7/23/11 Falwano Ngomuna 100km
Bulungu Nkata Kwanga 
mulungu
6/22/11 Kikongo Kikongo 16km
Gungu Lozo Munene Kalumbu
(Pop. 435)
7/16/11 Gatundo Luvuji 1 50km
Masi 
Manimba
Bangalala Mutulu 6/12/11 Kalunga 
Mikinzi
Kingola Poste 75km
Kasongo 
Lunda
Pelende Munganda
(Pop. 522)
8/7/11 Kiabamba Lubombo
(Pop. 198)
27km
Inongo Mbelo Ilungu
(Pop. 328)
9/3/11 Bokote Bokoloko
(Pop. 90)
19km
We cannot overcome this potential problem completely, but we do attempt to deal with it in 
two ways. First, our design explicitly models change in decentralization orientations over time 
between respondents in the treatment and control villages. To the extent that the (non￾randomly selected) treatment and control villages have characteristics that would lead 
individuals to show differences on decentralization orientations at a given point in time, and to 
the extent that these characteristics are stable, these factors will be subtracted out when 
individuals’ orientations are compared from the baseline to the post-test survey. That is, the 
use of difference scores in the analysis will control for possibly confounding village-level effects, 
as the procedure any stable village-level characteristic that distinguishes “treatment” and 
“control” villages (over and above the randomization process) in terms of their average level of 
decentralization orientations. 
Second, as will be discussed in more detail below, we also conduct additional individual-level 
“matching” of treatment and control group individuals on a set of variables that could be related 
to decentralization orientations and hence confound the inferences we seek to make about the 
effects of Boîtes sessions. This procedure resulted in a treatment and control sample of 
individuals who were effectively “balanced” in terms of age, sex, education, political interest 
and exposure to mass media before the treatment (Boîtes event) was introduced in the 
treatment villages. This increases our confidence that whatever deviations from pure random 
 
2
 It is also the case that the Boîtes/Voice program itself was not randomly implemented at the village level. This 
introduces the possibility that the village chosen by IFES for “treatment” were potentially more receptive to the 
messages in the Boîtes session than villages not chosen for treatment. Some of the procedures we implement 
here can help control these potential biases but whenever treatment placement is non-random they cannot be 
eliminated altogether (see Khandker et al. 2010).
11
sampling the village-level substitutions may have produced, the treatment and control 
individuals in the analysis were nevertheless effectively identical on variables known to be 
related to decentralization orientations, attitudes, and engagement. 
2. The Randomized “Encouragement” Component
a. The Logic of the Design 
We supplement treatment and control comparisons at the village level with an additional 
component designed to assess the specific effects of attending a Boîtes à Images sensitization 
session. This is an important aspect of the design for several reasons. First, it may be the case 
that no village-level effects will be detectable following the procedures described above; to the 
extent that the proportion of individuals who attend the events in a particular village is low, a 
random sample of 140 villagers may not produce sufficient numbers of attendees to show 
significant differences from the control villages, even if significant effects of exposure to the 
treatment exist. Second, we want to determine what the effect of direct exposure to the 
treatment actually is, aside from any spillover effects resulting from post-exposure discussion 
within individual social networks.3
 To determine the “direct effects” of exposure to the Boîtes
events, we therefore need to compare those individuals who attended the Boîtes events with 
individuals who did not attend. 
The most straightforward way to make these comparisons is simply to ask individuals in the 
treatment villages whether or not they attended the Boîtes event that had recently taken place 
during the post-test survey. There are two significant difficulties in relying solely on this 
approach (though of course we did collect this information and it does figure prominently in the 
subsequent analysis). First, as noted above, we typically have little idea a priori how many 
individuals will attend a given civic education event. If few individuals in a village attend, 
relative to the number of individuals interviewed, we will have a very small sample of “treated” 
individuals to analyze, and concomitant difficulties in demonstrating the direct effects of the 
treatment. And since the proportion of attendees is not known to us in advance, we have no 
way of knowing how large a random sample of a village would need to be in order to produce 
sufficient numbers of attendees.4
 This presents a practical difficulty in estimating the effect of 
Boîtes attendance. 
A more fundamental difficulty, however, is that individuals choose to attend the Boîtes events 
(as in virtually all civic education programs), and this “self-selection into treatment” introduces 
serious potential biases in the assessment of treatment effects. Self-selection into treatment 
means that it is impossible, even after introducing multiple control variables within a statistical 
matching or regression context, to rule out the possibility that unobserved factors leading 
individuals to attend the Boîtes events also produced higher levels of knowledge, engagement, 
or other decentralization orientations. “Selection on unobservables” such as motivation or 
 
3
 It may even be the case that spillover effects are negative, which again could produce small or negative 
village-level treatment effects in the analyses above, even if there were significant direct effects on 
individual attendees. 
4
 Finkel and Smith (2011) “solved” this problem by interviewing randomly-selected attendees at Kenya 
civic education workshops at the event sites before the workshop began. This procedure guaranteed a 
sufficient number of attendees, but potential biases stemming from the self-selected nature of workshop 
attendees nevertheless remained. 
12
personality is likely in the case of most civic education programs, as those individuals who are 
“predisposed” towards democracy or who are already “trending” towards democratic 
engagement are likely to be those individuals who voluntarily attend community workshops or 
other activities associated with democracy. As a result, a simple comparison of “attendees” 
versus “non-attendees” is likely to produce a biased estimate of program impact to the extent 
that these “unobservables” confound the estimates of interest. 
Ideally, we would be able to randomly assign individuals to the treatment, so that we would 
select a random sample of villagers to comprise the participants in a given Boîtes event, and 
compare their responses to a random sample of villagers who did not attend (along with a 
random sample of individuals in control villages as described above). Such a design was 
impossible to implement in this case, because, as with virtually all civic education programs 
outside of the school context, we can neither randomly assign individuals to the treatment, nor 
can we mandate compliance with any manipulation that would be imposed. That is, we cannot 
compel individuals to attend workshops, and in the usual case we cannot deny access to the 
event to another person in a given village who is part of our survey sample. 
To overcome these difficulties, we implemented a variation of the "encouragement design" 
described in Duflo, Glennerster and Kremer (2007), in which some randomly selected individuals 
are encouraged through a baseline interview process to experience a given kind of treatment 
while others are not given such encouragement. The encouragement design works by 
exogenously increasing the probability of attending some event as a result of the randomly￾assigned invitation to take the treatment. If the encouragement is successful, we can then 
compare the overall attitudes, values, or behaviors among individuals in the "encouraged to 
treatment" group with both the “not encouraged” and the control village groups in various ways 
to obtain estimates of program impact that preserve the core advantage of random 
manipulation in making valid causal inferences. As we will show below in the section on 
statistical modeling, we make initial comparisons of these groups in what is referred to as 
“intent to treat” effects, and subsequent analyses will go further in using the “encouraged to 
treatment” variable as what is known as an “instrumental variable” that proxies for self￾reported treatment exposure. The randomization process ensures that this proxy variable is 
unrelated to all factors —including unobservables— that are correlated with both treatment 
exposure and decentralization orientations. Thus, provided that the encouragement works as 
intended, the design allows for the estimation of treatment effects controlling for the possible 
confounding influences of self-selection.5
 
Figure II-1 below summarizes both components of the research design. We interviewed 140 
individuals in each of the 8 treatment villages, and make initial comparisons between the 1,120 
total treatment village responses and the 480 total respondents interviewed in the 8 control 
villages (Groups A+B versus Group C). Within each treatment village, moreover, we divide the 
respondents randomly between an “encouraged” group (Group A) and a “not encouraged”
group (Group B), and we use comparisons between all the groups, along with information about 
their exposure to the Boîtes event, in order to arrive at estimates of the direct effect of the 
treatment. 
 
5
 More technically, the design identifies the average treatment effect among “compliers” with the 
manipulation, that is, among those individuals who were induced to experience the treatment through 
the invitation but who otherwise would not have attended.
13
We note two other advantages of the design. First, it may be possible to compare the village￾level effects of the Boîtes event with the specific effect for attendees to arrive at an estimate of 
“secondary” or “spillover” effects of the sessions. These estimates may be further augmented 
through comparison of the non-encouraged portion of the treatment village sample with the 
control village sample (i.e., Groups B versus C) and comparisons of the non-attendees in the 
treatment village with individuals similar on background characteristics from the control villages 
(see chapter III, section D below). 
Figure II-1 Summary of Research Design 
Second, the encouragement design is also beneficial in overcoming one of the problems in 
assessing the effects of Boîtes attendance discussed above, i.e., the potential lack of attendees 
in the analysis sample. By introducing a random encouragement to attend the Boîtes event in 
the pre-test (baseline) survey, we also increase the likelihood that a sufficient number of 
individuals among those sampled were actually in attendance at the event. 
B. Survey Instrument and Variables Used in the Analysis
The survey instrument included questions relating to the main themes of the Boîtes à Images
program: decentralization, democracy, and political engagement. Key questions relate to 
individuals’ knowledge, attitudes, and opinions about the decentralization process and policies 
in the DRC, and we categorize these items under the general rubric of Decentralization 
Knowledge and Attitudes. Other questions relate to individuals’ general knowledge about 
politics, preferences about democracy, and engagement with politics. We categorize these items 
under the rubric of Political Knowledge, Political Attitudes, and Engagement. The survey 
instrument also asked Basic Demographic information. Finally, in the baseline survey a random 
sample of individuals in treatment villages were provided the “encouragement to attend” the 
upcoming Boîtes event. In the post-Boîtes survey, all individuals were asked if they had 
attended the session as well, with those who answered “yes” being asked a further series of 
questions related to their experiences and satisfaction with various aspects of the event, as will 
be discussed below. Many of the survey questions were drawn from previous civic education 
Respondents Per Village Total Respondents
Treatment Villages (8 Total) 140 1,120
Encouraged to 
Attend Boîte Event
(Group A)
100 800
Not Encouraged to 
Attend Boîte Event 
(Group B)
40 320
Control Villages (8 Total)
(Group C) 
60 480
Totals: Baseline Survey 200 1,600
Totals: Post-Test Survey 200 1,600
14
evaluations; others represent new items developed for the DRC and decentralization process, 
about which there is little survey-based research from which to draw. 
The questions were vetted through the IFES office in Washington and in Kinshasa, and were pre￾tested with a small sample of 100 individuals by BERCI in December 2010 (and subsequently 
modified as appropriate). The questionnaire was developed originally in English by Steven 
Finkel; it was translated into French by BERCI personnel and verified by French speaker Rola 
Adbul-Latif in the IFES Washington office. The final questionnaire was translated into two local 
languages, Lingala and Kikongo, for administration in the field. 
Decentralization Knowledge and Attitudes 
Respondents were asked a series of questions regarding their general and specific knowledge of 
the decentralization process in the DRC, its positive and negative aspects, their general support 
for decentralization, and their satisfaction with the pace of the process. Respondents were also 
asked their views on whether specific policy areas should be the responsibility of the central or 
local governments. 
Decentralization Knowledge. Respondents were asked two general and four specific questions 
about knowledge of the decentralization process in the DRC. The two general questions were: 
1. Do the decentralization laws give more power to the central government in Kinshasa, 
less power to the central government in Kinshasa, or does it not change the amount of 
power the central government in Kinshasa has? 
2. Do the decentralization laws give more power to the local and Provincial governments, 
give less power to the local and Provincial governments, or does it not change the 
amount of power the local and Provincial governments have? 
The four specific questions were: 
1. Do you happen to know how many provinces there are to be in the country, including 
the city of Kinshasa, once the laws about decentralization are passed? Is the number 
less than 10, between 10 and 20, between 20 and 30, or more than 30? [CORRECT 
ANSWER: 26] 
2. The laws about decentralization have created smaller units within each province that 
are called Entités Territoriales Decentralisées or ETD. One of these entities is la 
Chefferie. Do you happen to know who makes the laws for la Chefferie? [CORRECT 
ANSWER: le Conseil de Chefferie]
3. Decentralization has also changed the way that provinces and local governments get 
money to provide services for the people. Some of the money is supposed to come 
from local taxes, while some is supposed to come directly from the central government 
in Kinshasa. Think about a scale that runs from 0 to 100. If 0 means that none of the 
money for local governments comes from the central government in Kinshasa, and 100 
means than all of the money for local governments comes from Kinshasa, could you tell 
me what share of the money for local governments you think is supposed to come from 
Kinshasa? [CORRECT ANSWER: 40%] 
4. Have you ever heard of something called the “Caisse Nationale de Péréquation?” If yes, 
“is the Caisse Nationale de Péréquation something that is supposed to give money for 
people who don’t have enough food to eat”, “give money to village chiefs to settle land 
disputes and other problems in the villages,” “give money to local governments to 
support development projects” [CORRECT ANSWER], or “give money to returning 
combatants from the war to help them start new lives”
15
We summed respondents’ correct answers to create a Decentralization Knowledge scale ranging 
from 0 to 6. 
Perceived Knowledge of Decentralization. We asked respondents the extent to which they felt 
informed about the decentralization process. The question was coded as “1” for “not informed 
at all” to “4” for “very informed.”
Positive and Negative Aspects of Decentralization. The survey instrument included questions on 
the extent to which respondents agreed or disagreed with the following statements: (1) 
Decentralization will be good for the country because the local governments will be more able 
than the Kinshasa Government to find solutions to the population; (2) Decentralization will be 
good for the country because the DRC is too big to be controlled by the central power only; (3) 
Decentralization is not a good thing for the country because it will generate more conflict than 
economic and political resources; (4) Decentralization will be good for the country because it 
will give more control to ordinary citizens on economic development; and (5) Decentralization is 
not a good thing for the country because it will bring more corruption by authorities than 
before. Responses were scored as “1” for “strongly disagree” to “4” for “strongly agree.” We 
created two scales referring to the positive aspects of decentralization (items 1, 2, and 5) and 
the negative aspects of decentralization (items 3 and 4). 
General Support for Decentralization. We measured respondents’ support for decentralization 
by asking respondents the extent to which they agreed or disagreed with the following 
assertion: “In general, I support the decentralization process in the DRC.” Responses were 
scored as “1” for “strongly disagree” to “4” for “strongly agree.”
Pace of Decentralization. We asked respondents whether they thought decentralization process 
was going too fast, too slow, or at the appropriate pace. Responses were coded as “1” for “too 
fast,” “2” for “about the right pace,” and “3” for “too slow.”
Policy responsibilities. Respondents were asked about their opinions on whether the central 
government in Kinshasa or local governments should be responsible for different policy areas 
including: (1) roads and transportation; (2) schools and education policy; (3) economic policy 
and taxation; (4) the police; and (5) the military. For each item, responses were scored on a 
four-point scale, with “1” for “completely the central government,” “2” for “mainly the central 
government,” “3” for “mainly the local governments,” and “4” for “completely the local 
governments.” We collapsed the scales (1 & 2 = central government and 3 & 4 = local 
governments) coded them as “0” for “central government” and “1” for “local government.” We 
combined the responses for police and military authority into a single measure. 
Political Knowledge, Political Attitudes, and Engagement 
We asked a series of questions concerning the respondent’s general knowledge of how the 
political system of the DRC works, their perceived knowledge of the constitution, their support 
for democracy as a form of government and their level of satisfaction with it, and their support 
for political values associated with democratic governance. Respondents were also asked about 
their perceived ability to participate in politics, their interest in political matters, how often they 
discuss politics with others, and the extent to which they participate in political activities. 
General Political Knowledge. Respondents were asked four questions about knowledge of 
political matters in the DRC: (1) whether parents have to pay for their children to go to 
elementary school [CORRECT ANSWER: NO]; (2) who appoints members of the Supreme Court 
[CORRECT ANSWER: THE PRESIDENT]; (3) which political party got the most votes in the 2006 
election [CORRECT ANSWER: PPRD]; and (4) whether amendments to the constitution of the 
Democratic Republic of Congo made by a declaration of the President, by a simple majority of 
the national assembly, by a two-thirds majority of the national assembly, or by a majority vote 
16
by the people. [CORRECT ANSWER: TWO-THIRDS MAJORITY]. We summed respondents’ 
correct answers to create a Political Knowledge scale ranging from 0 to 4. 
Perceived Knowledge of the Constitution. We asked respondents the extent to which they felt 
informed about the content of the Constitution of the DRC. The question was coded as “1” for 
“not informed at all,” “2” for “somewhat informed,” and “3” for “very informed.”
Democracy is Best. We asked respondents, “Sometimes democracy does not work. When this 
happens, some people say that we need a strong leader that does not have to bother with 
elections. Others say that even when things do not work, democracy is always the best option. 
Can you tell us which statement is closest to your view?” Responses were coded as “0” for those 
who say that the country needs a strong leader and “1” for those who say that democracy is 
always best. 
Satisfaction with Democracy. We asked respondents whether they were satisfied with the way 
democracy works in the DRC. Responses scored as “1” for “not at all satisfied,” “2” for 
“somewhat satisfied,” and “3” for “very satisfied.”
Political Tolerance. We included a battery of questions testing respondents’ willingness to 
extend basic political rights to “people whose ideas are considered bad and dangerous.” 
Respondents were asked whether (1) a person who does not like their tribe should be allowed 
to participate in political activities of the village, and (2) a person who would abolish elections 
and install a military dictatorship should be allowed to participate in elections. The questions 
were scored from “1” for “strongly disagree” to “4” for “strongly agree.” We averaged the two 
items to create a Political Tolerance scale. 
Political Efficacy. We probed respondents about their perceived influence in politics, or political 
efficacy, by asking individuals whether they agreed with the following statements: (1) “I feel 
very prepared to participate in the political life of my country,” and (2) “The new 
decentralization law is too complicated to understand for people like me.” Responses were 
scored as “1” for “strongly disagree” to “4” for “strongly agree.” We averaged respondents’ 
answers to create a Political Efficacy scale ranging from 1 to 4. 
Political Interest. Respondents were asked about their interest in political matters in the DRC. 
Responses were scored as “1” for “not at all interested,” to “4” for “very interested.”
Political Discussion. We asked respondents whether they have discussed political matters with 
friends, family, or work colleagues in the past year and how many times they had done it. 
Responses were coded as “0” for “not done,” “1” for “once,” and “2” for “several times.”
Political Participation. The survey instrument included questions on whether the respondent 
had done any of the following activities in the past year: (1) participated in a collective effort to 
solve the problems of the neighborhood or community; (2) filed a complaint with a government 
agency or civil society organization about an injustice or violation of their rights; and (3) taken 
part in a protest, march, or demonstration on some national or local issue. For each item, 
responses were scored on a three-point continuum, with “0” for “not done,” “1” for “once,” and 
“2” for “several times.” We created a scale of political participation by summing individuals’ 
positive responses to the questions (“once” and “several times”). 
Demographic Variables 
Respondents were asked basic demographic questions such as age, sex, and level of education, 
and also whether anyone in their household had been victims of violence during the recent 
conflict in the DRC. These variables were measured as follows: 
Age. We asked respondents about their age in years. Responses were collapsed and coded in 
the following categories: “1” for “18 to 24,” “2” for “25 to 34,” “3” for “35 to 49,” and “4” for 
“50 to 86.”
17
Sex. Responses were coded as “0” for female and “1” for male.
Education. Respondents were asked about the highest level of formal education attained. We 
coded responses as “1” for “no schooling and cannot read and write,” “2” for “never attended 
school but can read and write,” “3” for “some elementary schooling,” “4” for “finished 
elementary schooling,” “5” for “some secondary classes,” “6” for “secondary school finished,” 
“7” for “some college,” and “8” for “college graduate.”
Violence Experienced. We included a question that asked whether respondents had lost a 
member of their immediate family during the recent armed conflict in the DRC. Responses were 
scored as “0” for “no” and “1” for “yes.”
Encouragement to Attend the Boîtes Session. Within each treatment village, BERCI selected 100 
individuals at random to form an “encouraged” sample, and 40 individuals at random to form a 
“not-encouraged” sample. Individuals in the “encouraged” group were told the following:
Thank you very much. We’ve finished the interview, but before we leave, I’d like to tell you about 
an event that is going to take place this _DATE OF BOITE EVENT_ at _PLACE OF BOITE EVENT_. 
This gathering will be an animated civic discussion with images related to the process of 
decentralization that is now going on in the country. It is organized by _NGO NAME_, an 
organization that is not affiliated with the government nor with any political party. The event will 
take place at the following address: _ADDRESS OF BOITE EVENT_.
There won’t be room for everyone to come, so our company has asked us to invite some of the 
people we are talking to participate in the event. Do you think it would be possible that you 
would participate? (IF YES): After the event takes place, if you give them your name, the 
facilitators will reimburse you any of your travel expenses.6
Attendance at Boîtes Session. In the post- Boîtes survey, all respondents were asked whether 
they had attended the Boîtes session that had recently taken place in the treatment village. The 
wording was as follows:
Last week (month), there was a meeting in _VILLAGE OF BOITE EVENT_ sponsored by _NGO 
NAME_ that talked about issues related to decentralization in the DRC. Did you participate in this 
event? 
We attempted to verify the participation of each individual in our survey who claimed to have 
attended a particular Boîtes event, but this proved virtually impossible given the incomplete lists 
available, transcriptions of individuals’ names that proved difficult to read, and other problems 
in the field. Nevertheless, we were able to make some headway in this regard by asking all 
individuals who claimed to have had attended the event to answer the following question: 
“Could you please tell how the Boîtes à Images civic education event was conducted? Was it 
through: an oral presentation by a leader, a presentation and comments about images, or 
through presentations of films and videos? 
We treated individuals who did not answer “a presentation and comments about images” as 
having a strong likelihood of not being in attendance despite their initial claim. This resulted in a 
 
6
 Importantly, all individuals who attended the event were given the equivalent of $5 to offset travel 
expenses, whether or not they were encouraged to attend or not. This prevented any bias resulting from 
resentment of non-encouraged attendees who may nevertheless have been in our analysis sample. 
18
77% attendance rate among individuals in the treatment sample, compared with the 87% 
attendance rate as measured by the initial claims of the respondents.
Session Quality. We included a battery of questions inquiring about individuals’ impression of 
the quality of the Boîtes session they attended. Respondents were asked the extent to which 
they were satisfied with a) the organization of the session; b) the information they learned at 
the session; and c) the competence of the facilitators who led the session. Each question was 
coded from “1” for “not at all satisfied” to “4” for “very satisfied.” Were averaged the three 
items to create a Session Quality scale. 
C. Field Work
 
Baseline interviews were begun by BERCI in the period between June 8, 2011 (Bulungu 
Territorie) and July 23, 2011 (Kusango Lunda Territorie). In each territory, the baseline survey 
was conducted within the week prior to the Boîtes à Images session taking place in the 
treatment village. Interviewers followed standard random route household interviewing 
procedures (described in pages 8-9 of the technical appendix from BERCI, found in Appendix C of 
this report). Detailed information about the respondent and how s/he could be recontacted in a 
future follow-up survey was collected. The procedures used to verify that the respondents were 
identical in the baseline and post-Boîtes interviews are also found in Appendix C of the report. 
Following the Boîtes session, interviewers attempted to reinterview all respondents from the 
baseline wave. This proved highly successful, as BERCI achieved a remarkable 98% reinterview 
rate. There were no differences in panel attrition rate between treatment and control villages. 
Post-Boîtes interviews were conducted between July 8, 2011 and September 4, 2011; this period 
represented a time of between 1 and 26 days following the Boîtes session in the treatment 
villages. The overall time between pre-event and post-event interviews ranged from 3 to 39 
days. 
D. Statistical Analyses 
1. Estimating Village-Level Effects of the Boîtes Sessions
We estimate the effects of the Boîtes events at the village level through comparisons of the pre￾and post-event levels of each dependent variable among individuals in the “treatment” and 
“control” village samples. Using standard notation so that subscript i refers to an individual, 
subscript t refers to the time or “wave” of observation, and subscript j refers to a village, we can 
express, for example, the individual’s knowledge about the decentralization process in the DRC 
at a given point in time as: 
(1) 
ijt t it i j ijt 0 1 2 3 y D I J          
where Dit represents prior exposure to the Boîtes event, (coded as “0” for all respondents at 
time t=1 and for control village respondents at time t=2, and coded as “1” for individuals in 
treatment villages only at time t=2), Ii
 represents all individual-level characteristics that are 
unique to individual i and that are stable over time, and Jj represents all stable village-level 
characteristics that are unique to the given village j, the 

 represent regression coefficients 
linking the independent variables to the outcome variable y, and 

is an error term. The 
equation models the individual’s score on Decentralization Knowledge (y) at a given point in 
19
time as a function of an overall average score for all individuals at that time β0t plus some effect 
β1 from having been exposed to the Boîtes event, along with effects from stable factors unique 
to each individual – either observed factors such as age or education level, or unobserved 
factors such as “personality” – and stable factors (e.g. observed socio-economic level or 
unobserved level of “social capital”) unique to the village.7
 This equation can be expressed for 
each individual in the treatment and control villages at both points in time, and an expression 
for the difference or change in decentralization knowledge (and other dependent variables) can 
be obtained by subtracting the equation at time 1 (pre-Boîtes) from the equation at time 2 
(post-Boîtes). This results in the following basic equation for estimating the effect of the 
treatment at the village level: 
(2) 
 
1 01 1 1 2 3 1
2 02 1 2 2 3 2
0 1 2 3
ij i i j ij
ij i i j ij
ij i i i ij
y D I J
y D I J
y D I I
    
    
    
    
    
          
which, since the individual and village level I and J terms are assumed to be stable, reduces to 
the following basic estimation equation for assessing the effects of the Boîtes event: 
 (3) 
ij 01 i ij        yD   
Alternatively, since ΔDi
 equals 0 for all individuals in the control villages and 1 for all individuals 
in treatment villages, the equation can be expressed as modeling the “difference in the average 
difference” in the dependent variable for the treatment and control villages: 
(4)
 
ij 01 i ij     y TREATMENT    
where Δβ0 then represents the average change over time in y for control villages, and β1
represents any additional change in y for individuals in treatment villages. If the β1 coefficient is 
statistically significant, this indicates that the change in decentralization knowledge (or other 
dependent variables) in treatment villages is statistically different from the change in control 
villages. And because both individual and village level “fixed effects” I and J have been included 
and then differenced out of consideration, this model represents a very powerful means of 
estimating causal effects of the treatment, independent of unobserved or unmeasured factors 
at the individual or village levels that may also influence decentralization knowledge, provided 
that those factors can be assumed to be stable over time. 
All of the models are estimated using basic ordinary least squares (OLS) regression methods and 
so-called “robust clustered” standard errors for purposes of statistical inference. Robust 
clustered standard errors take into account the fact that observations from individuals within a 
given village are likely to be related to one another to a significantly greater extent than are 
observations from individuals in different villages, thus violating the OLS assumption that 
sampled observations are statistically independent. These robust clustered standard errors 
 
7
 The unobserved components of the I and J terms are also commonly referred to as individual and village 
level “fixed effects”.
20
provide a more conservative test of statistical significance, as the tests are effectively based on 
the much smaller number of independent village-level observations (16), compared to the total 
number of individual observations (1,600).8
The model in equations (3) and (4) easily can be extended to include additional control 
variables, for example, educational attainment or age, and we report the results of models that 
control for these factors, along with sex, political interest, and media attentiveness, in 
subsequent analyses as well. These models, in effect, relax the assumption in (2) above that the 
regression coefficient linking these variables to y is the same in waves 1 and 2, and thus test for 
whether the Boîtes session affects changes in knowledge, over and above differential changes 
over time shown by individuals with different levels of education (or sex or political interest).
The main limitations of these analyses stem from the lack of complete randomization of 
treatment and control villages in the study, as noted above. This means that the treatment and 
control villages may differ on important factors related to decentralization orientations before 
the Boîtes treatment, and these pre-existing differences, and not the treatment itself may be 
responsible for differential changes observed between the treatment and control villages over 
time. Some of this problem is taken into account through the differencing process shown in (2), 
and some of the problem is further taken into account by including additional control variables 
in the estimation of equation (4). Nevertheless, we implement an additional method known as 
“coarsened exact matching” (CEM) to further account for possible pre-existing differences 
between individuals in treatment and control villages (see Finkel, Horowitz, and Rojo-Mendoza 
(2012) for a recent application in civic education evaluation). 
Table II-3 Covariate Imbalance, Pre- and Post-CEM balancing 
Pre-matching Means Post-matching Means
Variable Control Treatment Control Treatment Scale
Sex 0.55 0.53 0.56 0.53 0 = female
1 = male
Education 3.32 3.28 3.27 3.28 0 = no schooling
7 = university graduate
Age 37.74 35.78 35.80 35.77 18-86 years old
Media Attentiveness 0.95 1.04 1.02 1.04 0 = never
3 = everyday
Political Interest 2.84 3.12 3.11 3.12 1 = very little interest
4 = great deal of interest
Discuss Politics 0.82 0.82 0.86 0.81
0 = never
1 = once
2 = several times
N of Cases 471 1,101 470 1,084
Note: Boldfaced comparisons indicate significant differences at .05 level between treatment and control 
villages. 
 
8
 Clustered standard errors may be biased when the number of groups (villages) is relatively small, as in 
the current study. As a further robustness check, we use a more conservative procedure known as 
“randomization inference” in subsequent models whenever our initial estimates of equation (4) show a 
significant effect of the Boîtes treatment. (See Rosenbaum 2002). 
21
The CEM matching procedure produces different “bins” or strata consisting of the combinations 
of characteristics in the specified variables (e.g., women with high education, medium interest, 
young age), and then balances the treatment and control groups as completely as possible by 
eliminating treatment group cases that have no corresponding or “matched” control group 
member in their “bins”, and eliminating control group cases that have no corresponding or 
“matched” treatment group member in their “bins” as well. In our case, this procedure 
eliminated only 18 individuals, indicating that there was very strong initial balance between the 
treatment and control villagers on the “coarsened” variables. Table II-3 above shows the 
degree to which individuals in these groups were balanced before and after the CEM matching 
procedure. 
As the table shows, whatever differences existed between the treatment and control village 
samples on age, media attentiveness, and political interest in the baseline surveys were 
eliminated through the CEM matching and balancing procedures. In subsequent analyses, we 
match further on the pre-treatment level for each dependent variable, such that the estimation 
sample is fully balanced on the pre-Boîtes outcomes between the treatment and control 
villages. This means that we are estimating the effect of the Boîtes treatment on individuals in 
the treatment and control villages who are similar on important demographic and political 
characteristics and who have equivalent initial values of decentralization knowledge and the 
other outcomes of interest in the study. All of these procedures ensure that we are estimating 
the effect of the Boîtes events while controlling for as many pre-existing differences between 
the treatment and control villages as possible, given the lack of full randomization of treatment 
at the village level in the study design. 
2. Estimating the Effects of Attending Boîtes Sessions 
Following the logic of the exposition above, we also seek to estimate the effect of the Boîtes 
events on the specific individuals who attended the sessions in the treatment villages. We 
exploit the features of the “encouragement” design, whereby 100 randomly-selected treatment 
village individuals were invited to the Boîtes session in the baseline wave of data collection, 
while 40 randomly-selected treatment village individuals, along with the 60 randomly-selected 
control village respondents, were not told about the Boîtes event. The encouragement is 
designed to (randomly) increase the encouraged individuals’ probability of attending the Boîtes
session, and this “exogenously-induced” increase can be used in several ways to estimate the 
impact of attendance. The fact that the increase is exogenously or randomly-induced means 
that attendance, at least for some segment of the encouraged group, can be viewed as 
unrelated to any other factor — either observable or unobservable — that may be correlated 
with both attendance at the Boîtes sessions and decentralization orientations and other 
dependent variables. We may then use this information to estimate of the effects of the Boîtes
sessions, over and above the potential biases stemming from the self-selection of individuals 
into the treatment. In our case, the randomization of encouragement worked nearly perfectly 
in the field, as there were virtually no differences between “encouraged” and “non-encouraged” 
individuals from the treatment villages on demographic, political, or decentralization-related 
variables. We thus produced groups that were statistically identical aside from their being 
encouraged to attend the upcoming Boîtes event. 
We begin by estimating the “difference in differences”, i.e., the difference in each outcome from 
the baseline interview to the post-event interview, between individuals who were encouraged 
22
to attend the Boîtes event and individuals who were not encouraged to attend. This so-called 
“intent to treat” effect estimates the impact of Boîtes attendance on the randomly-selected 
treatment group without considering possible “non-compliance” with the manipulation. As 
non-compliance is likely to be related to factors associated with the outcomes in the study (e.g., 
political interest, motivation, overall knowledge about politics), the analysis of the randomized 
encouraged group versus the randomized non-encouraged group provides a “pure” estimate of 
the potential impact of the treatment. The “intent to treat” estimate results from a 
straightforward substitution of an “Encouragement to Attend” (Eit) indicator variable for the 
treatment variable D in equation (2) above, as: 
(5) 
1 01 11 2 3 1
2 02 12 2 3 2
0 12 2 3
0 12
ij i i j ij
ij i i j ij
ij i i i ij
ij i ij
y E I J
y E I J
y E I J
yE
    
    
    
  
    
    
         
     
where Ei represents the randomized encouragement to attend the Boîtes session and with all 
other terms as above.9
 Since the E term is randomly determined, and hence is unrelated to all 
factors in the unobserved error term ε (or Δε) that may also be related to the dependent 
variables, β12 provides an estimate of the Boîtes treatment that is based on the logic of 
randomized assignment.10
 
At the same time, as the amount of non-compliance increases, i.e., the more that encouraged 
individuals do not attend and non-encouraged individuals do attend, the estimated “intent to 
treat” effects will be correspondingly weaker. In our case, we observed significant amounts of 
non-compliance: 83% of the encouraged sample was coded as having attended the event, while 
a full 62% of the non-encouraged sample (within treatment villages) attended the events as 
well. This means that the randomized manipulation to attend was successful in influencing the 
likelihood of participation in the event, but the difference in self-reported attendance between 
the encouraged and non-encouraged groups in the treatment villages was only about 20%. This 
means that the encouragement manipulation has less statistical power than it would have had 
the “compliance” with the manipulation among non-encouraged individuals been higher; it also 
means that it is likely that the “intent to treat” estimate will underestimate the “true” causal 
effect of the treatment, given dilution from non-compliance.11
 
9
 Note that the E variable does not drop out through the differencing process because its causal effect on 
the outcome variable is assumed to change over time. In the baseline wave of observation E is assumed 
to have no effect on the outcome variable, i.e., being encouraged or not to attend the Boîtes event is 
assumed to have no direct influence on (pre-Boîtes levels of) decentralization orientations. This means 
that β11 equals 0 in (5a), while β12 will be greater than 0 in (5b), to the extent that encouragement does 
raise the probability of attending the Boîtes event and to the extent that the events have effects on 
individuals’ decentralization orientations.
10 As in the village-level analyses, we also control for other potential confounding variables, including the 
lagged value of the dependent variable, by including them in the regression model and by balancing them 
at the start of the analysis through the CEM procedure described above. In the “intent to treat” models, 
we balance the encouraged and not-encouraged groups on the relevant covariates. 
11 To the extent that non-encouraged individuals may discuss the Boîtes event with individuals who do 
attend, these potential “spillover” effects will also weaken the “intent to treat” estimate in (5). 
23
We can obtain a second estimate of the effects of Boîtes attendance by comparing the 
responses of individuals who report that they attended the session (in either the encouraged or 
non-encouraged groups) with all individuals who did not attend. We may call this the “self￾reported attendance” model, which takes into account non-compliance with the encouragement 
manipulation above, but ignores the advantages that result from the randomization process. 
This model is identical in form to the previous village level and “intent to treat” models, but with 
an indicator variable “ATTENDED” (Ai) used as the key independent variable. The “self-reported 
attendance” model is similar in design to previous observational evaluations of the effects of 
civic education, which compared self-reported attendees and non-attendees in estimating 
causal effects of the programs (e.g., Finkel 2003; Finkel et al 2012).12
 One major advantage of 
this design is that we have both baseline and post-event data for all respondents, so we are able 
to exploit the longitudinal nature of the data and difference out both individual and village level 
fixed effects as noted above (see also Finkel and Smith 2011). Still, there are potential self￾selection biases in the “self-reported attendance” model, to the extent that attendance at the 
event is associated with time-specific unobserved factors, or unobserved factors that also are 
associated with changes (as opposed to simple levels) in the dependent variables over time. For 
example, Boîtes attendees may have been trending upwards in their decentralization knowledge 
already, and this increased knowledge may have led them to attend the event and also show 
subsequent changes on knowledge that would have occurred in any case, with or without having 
attended the Boîtes session. We thus report the results of the self-reported attendance model 
but rely on it cautiously as an accurate estimate of the causal effects of exposure to the Boîtes
events. 
The final estimate of the impact of Boîtes attendance attempts to overcome the self-selection 
biases on the self-reported attendance model by exploiting the randomized nature of the initial 
encouragement design. In the self-reported attendance model, 
(6) 
i 01 i i       yA   
the problem stems from the potential relationship between attending the event (Ai) and 
unobservables captured in the error term εi
, which may bias the ordinary least squares 
estimation of the effects of Ai on Δyi
. To overcome this potential problem, we need an “outside”
exogenous variable, called an “instrumental variable”, which is related to attendance (Ai) at the 
event but unrelated to any of the unobservables or other components of the error term. 
Because the study design implemented here was successful, the randomized encouragement 
variable Ei
 satisfies these criteria and can serve as a valid “instrument” in assessing causal 
effects. Since Ei
 did increase the probability of attending the Boîtes event (as noted above), it 
satisfies the first criteria. And since individuals in treatment villages were randomly assigned to 
receive the encouragement, Ei
 is therefore statistically independent of any other variable, either 
observed covariates or unobserved factors in εi
. It therefore satisfies the second criteria for 
serving as an appropriate instrumental variable as well. 
The estimation for this “instrumental variable” (IV) model involves a two-stage procedure. In 
the first stage, we regress Ai
 on the instrumental variable Ei and all control variables Zk that may 
 
12 We also note that there are fewer “non-attendees” in the sample than we would like (N=252, or 23% of 
the treatment village sample), but as mentioned above it was not possible to estimate accurately a priori
the proportions of likely attendees.
24
be related to attendance at the Boîtes events and generate a “predicted” attendance variable
A*i
: 
(7) 
01
*
01
i i k k i
i i k k
A E Z u
A E Z
  
  
   
  
Because Ai
is predicted by exogenous or random variables that are unrelated to the outcome 
equation’s error term εi
, then A*I
 is also unrelated to those error terms. In the second stage, we 
use A*I as the “proxy” for Ai
 in the self-reported attendance equation to arrive at the 
“instrumental variables” estimate of the causal effect of the Boîtes exposure, “purged” of the 
potential bias that may have resulted from the original correlation between Ai
 and εi
.
13
: 
(8) 
*
i 01 i k k i        y A Z    
Importantly, the instrumental variables procedure does not provide an estimate of the causal 
effect of Boîtes exposure for all individuals who would potentially attend the events. Rather, 
the estimated effect is limited to those individuals who attended the session as the result of the 
randomized encouragement itself, i.e., those individuals who were exogenously “pushed” into 
Boîtes exposure. This group constitutes 20% of individuals in treatment villages (as noted 
above, calculated by subtracting the 61% of attendees in the non-encouragement group from 
the 82% of attendees in the encouragement group). We cannot estimate the effects of the 
session on individuals who would always have attended, whether or not they were encouraged, 
nor the effects on individuals who would never have attended, regardless of encouragement. 
The so-called “Local Average Treatment Effect” from the instrumental variables model, 
nevertheless, is informative substantively, as it provides a causal effect of the treatment on 
individuals who are pushed into attendance as a result of a relatively mild effort at recruitment, 
which mimics to some extent the kind of mobilization from NGOs or other civil society 
organizations that typically precedes a civic education event.14
 
3. Additional Explorations 
The foregoing analysis form the core of our concerns in the study: estimating the overall village￾level effects of the Boîtes events, and estimating the effects of exposure to the events among 
those individuals who attended the Boîtes sessions (or, in the case of the instrumental variables 
model, those individuals who attended as a result of the “push” from the encouragement 
 
13 One additional assumption of the IV procedure is that the randomized encouragement variable cannot 
have a direct effect on the outcome variables, as in that case the second stage estimation procedure will 
break down completely (given that A* as well as all of the variables that predict A* would all be included 
in the outcome equation). This assumption would be violated if the encouragement to attend the Boîtes 
session prompted individuals to learn about decentralization, independent of attending the event. This 
seems unlikely to have occurred, given the objective lack of progress in the actual implementation of the 
decentralization process in the DRC and the relatively low salience of the issue at the time of the study.
14 We use CEM matching to balance the “encouraged” and “non-encouraged” individuals on relevant 
covariates before the instrumental variables estimation. We also include the village-level 
treatment/control variable in the first stage estimation in order to arrive at a more accurate prediction of 
self-reported attendance, though none of the substantive conclusions we draw depend on the inclusion of 
this variable. 
25
manipulation). Subsequent analyses extend our focus to three other areas that are also 
important for the overall evaluation of the Boîtes events. 
First, we seek to assess the extent to which the Boîtes events generated “spillover” or 
“secondary” effects among non-attendees in the treatment villages. We accomplish this by 
comparing the changes in decentralization orientations among the individuals in the control 
villages to different groups of individuals in the treatment villages. An initial comparison could 
be between the control villages and those in the treatment villages who were randomly not 
encouraged to attend; this comparison would yield a kind of “intent to treat effect” (or more 
precisely, the intent “not to treat” effect). Yet we know that some 60% of individuals in the not￾encouraged group from treatment villages nevertheless took up the treatment, so we cannot 
legitimately claim that effects using this method would produce estimates of “spillover”. We 
conduct instead a set of analyses comparing the self-reported non-attendees in treatment 
villages to individuals in control villages, and attempt as best as possible to take the self-selected 
nature of individuals’ non-attendance into account. That is, individuals in the treatment villages 
who opted out of the treatment are likely to be different from the control individuals on factors 
that may affect changes in the outcome variables over time, regardless of whether they came 
into contact with or otherwise learned about the messages of the Boîtes sessions. We do not 
have an instrumental variable available to deal with this potential bias directly, but we are able 
to match the non-attendees to the control respondents on as many observed covariates as 
possible through the CEM procedures described above. This produces an estimate of spillover 
among non-attendees who are balanced as fully as possible with control respondents on 
demographic, political, and motivational factors that may have otherwise confounded the 
comparisons, as well as balanced on the pre-Boîtes levels of each dependent variable. However, 
the estimates of spillover within the treatment villages are not as precise as they might be, due 
to the relatively small number of individuals in our sample who did not attend the Boîtes events 
(N=252). 
Our second set of additional analyses concerns possible differences in the causal effects of the 
Boîtes “treatments” depending on factors related to the quality and timing of the sessions. 
Specifically, we hypothesize that Boîtes events that were led by more competent facilitators, 
that contained more complete information, and that were better organized had stronger effects 
on individual decentralization orientations than events that were of lesser “quality”. To test this 
hypothesis, we extend the initial village level model in equation (4) by including a variable that 
interacts the treatment village indicator variable with Session Quality, the variable discussed 
above that represents the average level of satisfaction with the event on the competence, 
information, and facilitator dimensions among individuals who attended the event. 
We test further for the possibility of “fade-out” effects of the Boîtes events, such that individuals 
interviewed immediately after the event took place may show greater effects than individuals 
interviewed some weeks later. These models are estimated by interacting the treatment village 
indicator variable with the length of time since the Boîtes event in the respondent’s village.
Finally, we test whether the Boîtes sessions had “heterogeneous effects,” that is, differential 
effects on different kinds of individuals, in particular whether the sessions had greater effects on 
those with lower levels of political and social resources, or on those with lower initial levels of 
the decentralization knowledge and other outcomes of interest. All of these models were 
26
estimated by constructed interaction variables with the treatment indicator and including these 
interactions in the equation (4) above. 
Chapter III. The Impact of the Boîtes à Images Sessions: Basic Results 
A. Subjective Assessments of Session Quality
We begin our assessment of the Boîtes à Images sessions by presenting the subjective views of 
the program among individuals who attended the events in the treatment villages. Judged 
according to this criterion, the events were highly successful. The figures below show that 
feelings of satisfaction with the sessions were nearly universal among the participants. 
Figure III-1a
Figure III-1b
For instance, half of the participants report being “very satisfied” with the organization of the 
session (Figure III-1a), and some 43% more report being “satisfied” on this dimension. Similar 
distributions —approximately half being “very satisfied” and half being “satisfied” are seen for 
satisfaction with the “information learned during the session” and the “competence of the 
facilitators”, with very few responses on any of the questions being registered in the 
“dissatisfied” categories. These results suggest that the Boîtes à Images events were popular 
2% 4%
43% 51%
Organization of the sessions
Not at all satisfied
Not so satisfied
Satisfied
Very satisfied
1% 6%
51%
42%
Information learned in the sessions
Not at all satisfied
Not so satisfied
Satisfied
Very satisfied
27
among those taking part, with a high degree of satisfaction with the sessions’ organization and 
facilitators, and a high degree of perceived effectiveness in terms of transmitting information. 
Figure III-1c
It should be noted, however, that the level of satisfaction differs considerably across 
participants from the eight different events. Though levels of “dissatisfaction” are never more 
than 10% in any of the locales, the levels of “very satisfied” versus “satisfied” respondents do 
show substantial variation. Figure III-2 shows the proportion of “very satisfied” respondents 
across the eight locations for each of the three satisfaction questions. As can be seen, 
proportions of respondents who were “very satisfied” with the organization of the session 
ranged from a high of 97% in Ilungu (in Inongo Territorie) to a low of 30% in Ebiala Kandolo (in 
Idiofa Territorie). 
Figure III-2: Percent 'very satisfied' by village
Variations of similar magnitude are seen for proportions very satisfied with “information 
learned” and “facilitator competence”: the event in Ilungu rated highest, the events in Ebiala 
Kandolo and Mbaya generally the lowest, with the other events falling somewhere in between. 
It is also the case that events rated higher on one dimension generally show higher values on 
the other dimensions as well: the average correlation between the measures is approximately 
.40, and the reliability coefficient (alpha) is approximately .66. We conclude, then, that 
2% 5%
46%
47%
Competence of facilitators
Not at all satisfied
Not so satisfied
Satisfied
Very satisfied
0
20
40
60
80
100
Organization
Information
Competence
28
satisfaction with the events among participants was generally quite high, but that there was 
nevertheless significant variation across the eight events in terms of perceived quality, 
organization, and effectiveness of the sessions. We investigate in section E below the degree to 
which these “quality” dimensions may have been related to the strength of the sessions’ 
impacts on individuals’ decentralization orientations.
B. Village-Level Effects of the Boîtes Sessions 
Demonstrating the effectiveness of the Boîtes à Images events, however, must go beyond 
whether attendees were satisfied with the session or with what they believe they learned. The 
core analyses of the study attempt to determine whether the events led to significantly greater 
changes in decentralization knowledge, attitudes, and political engagement among individuals in 
villages where the Boîtes events took place compared to individuals in comparable control 
villages and to determine the magnitude of the effects on the specific individuals who attended 
the events as well. We begin by showing the distribution and changes in all the orientations 
from the baseline to the post-test wave of observation for all individuals in the control and 
treatment villages. Table III-2a shows the results for the series of questions related to 
knowledge of the decentralization process, perceived knowledge of decentralization, general 
political knowledge, and perceived knowledge of the DRC constitution; Table III-2b shows the 
results for other decentralization orientations examined in the study, including support for the 
process, opinions about the pace of decentralization, support for greater responsibility of local 
governments in a series of policy areas, and discussion about the issue. Table III-2c shows the 
results for more general attitudes about democracy and overall political engagement. These 
comparisons represent the “raw” differences between the treatment and control villages before
implementing the CEM matching and balancing procedures in subsequent regression analyses. 
Table III-2a shows the percentages of individuals in the baseline and post-Boîtes surveys who 
answer each of the decentralization knowledge questions correctly. It can be seen that 11% 
more individuals in control villages showed increase over time in correct responses to the 
general question about decentralization giving less power to the central government, with a 
comparable 11% change in the increase in correct responses among individuals in treatment 
villages. On all other decentralization knowledge questions, however, there were dramatically 
greater changes in correct responses among individuals where Boîtes à Images events took 
place than in the comparable control villages. For example, in the post-Boîtes wave of 
observation, roughly 60% of individuals in the treatment villages correctly identified the Caisse 
Nationale de Péréquation and correctly noted the proportion of funds that would be coming to 
localities from the central government; these represented increases of approximately 40% from 
the baseline survey on these items, compared to changes of approximately 10% in control 
villages. Changes in treatment villages for correct responses on the other decentralization 
knowledge questions, related to the number of provinces envisioned in the decentralization 
process and on who would execute the laws in la Chefferie, were also of much greater 
magnitude in treatment than control villages over time. 
We show the general differences between changes in decentralization knowledge for the 
treatment and control villages by constructing a summary scale of the number of correct 
responses given by the individual on the five items. As can be seen, the proportion who respond 
correctly on 3 or more of the 6 items changes by only about 10% among control village 
individuals (from 38% to 46%), while the corresponding change is 35% among individuals in 
29
villages where Boîtes à Images events took place. The effect is even more dramatic if one 
examines the proportions of individuals who increased at all in their decentralization knowledge 
over time (i.e., ignoring individuals whose knowledge stayed the same or decreased from the 
baseline to the post-Boîtes wave of observation). In the control villages, just over 1/3 of all 
individuals (36%) registered some increase over time; this figure nearly doubles to 69%, or more 
than 2/3 of all individuals in treatment villages. These are large differences, and strong initial 
evidence that the Boîtes events were effective in their primary goal of educating individuals 
about the decentralization process. 
Table III-2a. Decentralization and General Knowledge Pre- and Post-Boîtes Sessions
Control Villages Treatment Villages
Variable Pre Post ∆ Pre Post ∆
Decentralization gives less 
power to the central gov’t 
in Kinshasa
42.7% 52.9% 10.2% 53.0% 63.5% 10.5%
Decentralization gives 
more power to the 
provinces and local gov’t
41.8% 43.3% 1.5% 47.8% 59.8% 12.0%
No. of provinces after 
decentralization 52.9% 62.6% 9.8% 52.4% 81.9% 29.5%
Who makes the laws of 
the chiefdom 38.9% 42.9% 4.0% 40.1% 54.5% 14.4%
Proportion of funds for 
provinces and local gov’t 
coming from central gov’t 
15.9% 25.3% 9.3% 23.5% 61.9% 38.3%
Caisse Nationale de 
Péréquation funds local 
development projects
3.4% 13.8% 10.4% 13.4% 58.6% 45.1%
No. of correct responses 
to decentralization 
questions
0 = 16.6%
1 = 23.4%
2 = 24.6%
3 = 21.0%
4+ = 14.4%
(
x
= 1.96)
0 = 7.2%
1 = 21.0%
2 = 26.1%
3 = 22.9%
4+ = 22.7%
(
x
= 2.41)
0.45
0 = 11.7%
1 = 20.2%
2 = 25.0%
3 = 21.4%
4+ = 21.7%
(
x
= 2.30)
0 = 4.7%
1 = 6.9%
2 = 10.5%
3 = 19.2%
4+ = 58.7%
(
x
= 3.80)
1.50
Perceived knowledge of 
decentralization 
1 = 15.3%
2 = 18.3%
3 = 38.1%
4 = 28.4%
(
x
= 2.80)
1 = 14.7%
2 = 11.1%
3 = 42.6%
4 = 31.7%
(
x
= 2.91)
.12
1 = 15.8%
2 = 14.3%
3 = 35.0%
4 = 34.9%
(
x
= 2.89)
1 = 7.0%
2 = 7.9%
3 = 33.3%
4 = 51.8%
(
x
= 3.30)
.42
General political 
knowledge (No. of correct 
responses) 
0 = 3.0%
1 = 20.6%
2 = 41.4%
3 = 28.7%
4 = 6.4%
(
x
= 2.15)
0 = 4.5%
1 = 22.1%
2 = 34.4%
3 = 33.8%
4 = 5.3%
(
x
= 2.13)
-.01
0 = 6.1%
1 = 23.2%
2 = 36.5%
3 = 28.4%
4 = 5.8%
(
x
= 2.05)
0 = 3.6%
1 = 14.6%
2 = 35.3%
3 = 34.8%
4 = 11.6%
(
x
= 2.36)
.31
Perceived knowledge of 
the constitution
0 = 57.3%
1 = 29.3%
2 = 13.4%
(
x
= 0.56)
0 = 55.4%
1 = 30.4%
2 = 14.2%
(
x
= 0.59)
.03
0 = 48.0%
1 = 32.7%
2 = 19.4%
(
x
= 0.71)
0 = 34.9%
1 = 41.1%
2 = 24.1%
(
x
= 0.89)
.18
N of Cases 464 to 471 1,039 to 1,101
30
Table III-2a also shows that individuals in treatment villages showed somewhat greater changes 
in general political knowledge than did those in control villages, and somewhat greater increases 
in their perceived knowledge of decentralization and constitutional issues in the DRC. For 
example, on the general factual knowledge scale, the proportions of individuals in the control 
villagers who answered 3 or 4 of the 4 questions correctly increased by 4%; this figure was 12% 
in the treatment villages. The proportion of control villagers who believe they are “very” or 
“somewhat informed” about decentralization and about the DRC constitution does not change 
much over time, while perceived knowledge on both dimensions rise by 13-15% in treatment 
villages. These differences, however, are much lower in magnitude than those observed on the 
decentralization knowledge questions. Still, they provide some initial suggestion that the Boîtes
events may have stimulated general gains about decentralization, political and constitutional 
knowledge. 
Table III-2b. Attitudes Toward Decentralization Pre- and Post-Boîtes Sessions
Control Villages Treatment Villages
Variable Pre Post ∆ Pre Post ∆
General support for 
decentralization 78.8% 86.0% 7.2% 74.9% 87.3% 12.4%
Positive aspects of 
decentralization
0 = 8.7%
1 = 15.1%
2 = 24.4%
3 = 51.8%
(
x
= 2.19)
0 = 2.3%
1 = 8.3%
2 = 20.8%
3 = 68.6%
(
x
= 2.56)
.36
0 = 8.2%
1 = 12.0%
2 = 23.8%
3 = 56.0%
(
x
= 2.28)
0 = 3.5%
1 = 5.7%
2 = 17.1%
3 = 73.7%
(
x
= 2.61)
.33
Negative aspects of 
decentralization
0 = 50.3%
1 = 32.3%
2 = 17.4%
(
x
= 0.67)
0 = 47.8%
1 = 24.0%
2 = 28.2%
(
x
= 0.80)
.13
0 = 48.2%
1 = 31.3%
2 = 20.4%
(
x
= 0.72)
0 = 61.4%
1 = 20.6%
2 = 18.0%
(
x
= 0.57)
-.16
Pace of decentralization is 
too slow 62.4% 63.5% 1.1% 61.9% 58.5% -3.4%
Decentralized policy 
responsibility: military 
and police
0 = 66.7%
1 = 22.1%
2 = 11.3%
(
x
= 0.45)
0 = 64.8%
1 = 25.7%
2 = 9.6%
(
x
= 0.45)
.00
0 = 73.0%
1 = 18.3%
2 = 8.7%
(
x
= 0.36)
0 = 82.8%
1 = 11.2%
2 = 6.0%
(
x = 0.23)
-.13
Decentralized policy 
responsibility: roads and 
transportation
30.6% 41.5% 10.9% 27.4% 21.7% -5.7%
Decentralized policy 
responsibility: schools 
and education
32.5% 41.0% 8.3% 40.1% 37.1% -2.7%
Decentralized policy 
responsibility: taxation 46.3% 58.9% 12.3% 56.9% 53.0% -4.2%
N of Cases 463 to 471 1,078 to 1,101
However, similar gains were not observed on orientations aside from decentralization and 
political knowledge. As Table III-2b shows, the changes in attitudes about decentralization 
observed over time in control and treatment villages were all relatively small. Individuals in 
both control and treatment villages were highly supportive of decentralization at both waves of 
observation, with the treatment villages showing only a slightly larger increase. Both groups 
31
increased modestly in their views of the positive aspects of decentralization, and both groups 
stayed roughly the same on their views of the negative aspects of decentralization and their 
(generally negative) views of the pace with which decentralization was going in the country. 
There were some modest changes in the groups’ views on which level of government should be 
responsible for different policy areas, with individuals in the treatment villages being more likely 
over time to ascribe greater responsibility for the national government for issues such as police 
and the military, while individuals in the control villages were more likely over time to ascribe 
greater responsibility for local governments on issues such as schools and taxation. All of this 
indicates that, whether or not a Boîtes event took place in the village, changes on 
decentralization attitudes were, for the most part, generally minimal. 
Table III-2c. Democratic Attitudes and Political Engagement Pre- and Post-Boîtes Sessions
Control Villages Treatment Villages
Variable Pre Post ∆ Pre Post ∆
Democracy is the best 
option 79.4% 78.8% -0.1% 75.4% 80.1% 4.7%
Satisfaction with 
democracy 
0 = 54.7%
1 = 34.6%
2 = 10.7%
(
x
= 0.56)
0 = 49.5%
1 = 37.2%
2 = 13.4%
(
x
= 0.64)
.07
0 = 36.1%
1 = 39.3%
2 = 24.6%
(
x
= 0.88)
0 = 34.4%
1 = 47.6%
2 = 18.0%
(
x
= 0.84)
-.04
Political tolerance
0 = 26.8%
1 = 29.3%
2 = 44.0%
(
x
= 1.17)
0 = 20.4%
1 = 28.7%
2 = 51.0%
(
x
= 1.31)
.13
0 = 22.6%
1 = 28.0%
2 = 49.4%
(
x
= 1.27)
0 = 13.9%
1 = 17.3%
2 = 68.9%
(
x
= 1.55)
.28
Political efficacy
0 = 15.1%
1 = 42.9%
2 = 42.0%
(
x
= 1.27)
0 = 10.6%
1 = 43.5%
2 = 45.9%
(
x
= 1.35)
.08
0 = 14.7%
1 = 46.9%
2 = 38.4%
(
x
= 1.24)
0 = 13.2%
1 = 48.0%
2 = 38.9%
(
x
= 1.26)
.02
Political discussion
0 = 49.0%
1 = 20.0%
2 = 31.0%
(
x
= 0.82)
0 = 53.5%
1 = 17.6%
2 = 28.9%
(
x
= 0.75)
-.07
0 = 49.0%
1 = 20.2%
2 = 30.8%
(
x
= 0.82)
0 = 46.0%
1 = 26.6%
2 = 27.4%
(
x
= 0.81)
.00
Political participation 
0 = 22.9%
1 = 50.3%
2 = 17.8%
3 = 8.9%
(
x
= 1.13)
0 = 26.5%
1 = 53.1%
2 = 15.5%
3 = 4.9%
(
x
= 0.99)
-.14
0 = 21.4%
1 = 51.5%
2 = 20.5%
3 = 6.5%
(
x
= 1.12)
0 = 23.8%
1 = 52.9%
2 = 15.2%
3 = 8.2%
(
x
= 1.08)
-.04
N of Cases 468 to 471 1,056 to 1,101
Table III-2c shows even smaller differences in the gains observed in control and treatment 
villages for political orientations not directly related to decentralization. Changes in control and 
treatment villages were small and of similar magnitude in treatment and control villages in 
terms of individuals’ support for democracy (very high at both time periods in both sets of 
villages), their satisfaction with the way democracy is going in the DRC (mixed views in both sets 
of villages), their levels of political and social tolerance (mixed with slightly higher gains in the 
treatment villages), and in terms of cognitive and behavioral political engagement (virtually 
identical values in treatment and control villages over time). We conclude that, whatever 
effects the Boîtes sessions had on political knowledge, they appear not to be associated with 
general gains in democratic orientations or engagement with the political process. 
32
We estimate the effects of the Boîtes sessions at the village level more formally through the 
regression models outlined in Chapter II (equation 4). In these models, we predict changes in 
each orientation over time with an indicator variable signifying whether the individuals was in a 
treatment or control villages, with the estimation sample of treatment and control village 
individuals being balanced through the CEM procedure on the initial value of each orientation 
(so as to equate the treatment and control groups’ “starting points” in assessing change). We 
also balance the villages on covariates such as age, sex, political interest and media 
attentiveness so as to minimize these variables’ potentially confounding biases. The coefficients 
can then be interpreted as the effect of residing in a treatment village on an individual’s change 
in each orientation over time, compared to an individual with similar initial levels of the 
orientation and with similar demographic and political characteristics who resides in a control 
village. We present two models: one with only the “treatment” variable included, and one with 
the treatment variable along with additional control variables -- the initial level of the 
orientation and the other covariates that went into the CEM balancing process. There are 
virtually no differences between the estimates obtained from the two models, which gives us 
added confidence that the CEM matching procedure worked as expected in balancing the two 
samples on relevant covariates. The results are shown in Table III-3 above. 
It can be seen that the patterns observed in the basic cross tabulations above are confirmed in 
the regression analyses. By far the strongest effect of the Boîtes sessions is observed for the 
decentralization knowledge scale, with individuals in treatment villages increasing by 
approximately 1 and 1/3 “correct” answers after the Boîtes event took place compared to 
comparable individuals in control villages. This is an effect of sizeable substantive magnitude as 
well, with the single exposure of the Boîtes event producing an effect that represents over 20% 
of the distance of the entire knowledge scale, and one that represents a change of nearly ¾ of a 
standard deviation on the change in knowledge over time. To the best of our knowledge, this is 
one of the largest effects registered in the several civic education evaluation studies previously 
conducted on behalf of USAID, and certainly the largest registered on political knowledge (Finkel 
2003; Finkel and Smith 2011; Finkel et al., 2012). The Boîtes sessions do appear to have 
stimulated substantial gains in knowledge about the decentralization process at the village level 
from the baseline wave of observation to the post-Boîtes interviews.15
The results in the rest of the table also largely replicate the patterns seen in the cross￾tabulations. Significant effects of residing in a treatment village are observed for general 
knowledge and perceived knowledge of the constitution, though the effects are of substantially 
smaller magnitude than for decentralization knowledge (with the effects for both variables 
representing distances of approximately 10% on the knowledge scales and approximately .30 in 
standard deviation terms).16
But the effects on decentralization and overall political knowledge are not associated with 
significant effects on any of the other variables tested in the study, save for two small effects on 
attitudes towards decentralized authority in specific policy areas. Individuals in the treatment 
 
15 The effect remains strongly significant in the randomization inference robustness check (p<.01). 
16 The randomization inference test indicates further that only the general knowledge effect remains 
significant in this robustness check, though perceived knowledge of decentralization and perceived 
knowledge of the constitution are borderline in statistical significance (p=.07 and .08, respectively). 
33
villages are more likely to favor national authority in the areas of police and military and more 
likely to favor national authority in the areas of roads and transportation as well. 
Table III-3 Village-Level Effects of Boîtes Sessions
Variable w/o controls w/ LDV and controls
Overall decentralization knowledge 1.32***
(.33)
 1.38***
(.27)
Perceived knowledge of decentralization .32*
(.12)
 .35**
(.12)
General political knowledge .32**
(.10)
 .30*
(.12)
Perceived knowledge of the constitution .27*
(.11)
 .27*
(.10)
General support for decentralization .07
(.11)
.06
(.12)
Positive aspects of decentralization .08
(.16)
.08
(.08)
Negative aspects of decentralization -.17
(.19)
-.17
(.18)
Pace of decentralization is too slow -.01
(.09)
.00
(.09)
Support for decentralization: military and police -.18*
(.08)
-.18
(.09)
Support for decentralization: roads and transportation -.18**
(.06)
-.18*
(.05)
Support for decentralization: schools and education -.09
(.07)
-.10
(.09)
Support for decentralization: taxation -.10
(.08)
-.10
(.06)
Democracy is the best option .01
(.06)
.01
(.04)
Satisfaction with democracy .02
(.11)
.02
(.10)
Political tolerance .22
(.17)
.23
(.17)
Political efficacy -.08
(.12)
-.07
(.11)
Political discussion -.04
(.11)
-.04
(.12)
Interest in politics .17
(.14)
.17
(.14)
Political participation .12
(.09)
.09
(.09)
All estimates are CEM-weighted with clustered robust standard errors in parentheses. N ranges from 1,305 to 1,515 
***p < .001 (two-tailed); **p < .01 (two-tailed); *p < .05 (two-tailed) 
34
These effects indicate that, to the extent that treatment villages showed any changes in their 
decentralization preferences after the Boîtes sessions, they tended in the direction of more 
national as opposed to local authority.17
 No other variable shows statistically significant 
differences between the changes observed in treatment and control villages over time. We 
conclude that there are strong and significant effects of the Boîtes sessions at the village level on 
knowledge about decentralization, weak but significant effects of the Boîtes sessions at the 
village level on general political knowledge and preferences for more national authority over 
roads and transportation, and essentially no effects on any other orientation regarding 
decentralization, democracy, or political engagement. 
C. The Effects of Attending Boîtes Sessions 
The village-level effects estimated above represent the total impact of the Boîtes à Images 
sessions on individuals in treatment villages, compared to demographically and political 
comparable individuals in control villages with similar baseline levels of each orientation. We 
can view these “total effects” as resulting from the session’s immediate direct effect on 
individuals who attended the event, plus any diffusion of the information from the session 
among individuals in the village after the event took place. We seek in the next several sections 
to disentangle these effects, that is, to estimate the separate effects of direct Boîtes exposure
from effects on non-attendees from post-Boîtes discussion and diffusion through village social 
networks. As described in Chapter II above, we estimate three different models, each with 
advantages and limitations, in order to assess the direct effect of Boîtes exposure: 1) the intent￾to-treat model that uses the randomized “encouragement to attend” manipulation as the key 
independent variable; 2) the self-reported attendance model that compares individuals who 
attended with individuals who did not attend; and 3) the instrumental variable model that uses 
the randomized encouragement as an exogenous proxy for attendance so that the effects of 
attendance, purged of potential self-selection bias are estimated (for the subset of individuals 
who were “pushed” into the treatment by the encouragement manipulation). We show the 
results of these models in Table III-4.
The results of all of these analyses show consistent effects of attending the Boîtes à Images 
sessions on the individual’s knowledge about the decentralization process. The intent-to-treat
(ITT) estimate is 1.23, indicating that randomly “encouraged to attend” individuals improved on 
the decentralization knowledge scale by nearly 1 and ¼ point compared to randomly “not 
encouraged” individuals. This value is somewhat smaller than the overall village estimate in 
Table III-3, likely due to the large number of “non-compliers” in the comparison category, i.e., 
individuals who were not encouraged to attend in the treatment villages but nevertheless 
attended the event. When comparing “randomly encouraged” individuals only to individuals in 
the control villages, the intent to treat estimate rises to 1.62. This is strong initial evidence of
direct Boîtes impact in stimulating individual learning about the decentralization process. 
This conclusion is reinforced by the results obtained from the two other models. Individuals 
who report having attended the event (column 3 in Table III-4) register a full 1.61 greater 
change in decentralization knowledge over time compared to those who did not attend, with 
this difference again being statistically significant. As noted above, this estimate may be 
 
17 The randomization inference tests show that the police-military effect is borderline significant at the .08 
level, while preferences for more national authority in transportation remains significant at the .05 level. 
35
inflated, to the extent that individuals who self-selected into the event were already increasing 
on knowledge, or whose unobserved characteristics led to learning changes due to some other 
aspect of the environment aside from their having attended the Boîtes event. 
Table III-4: Effects of Boîtes Session Attendance
Variable Encouraged vs. All 
non-encouraged
Attendees vs. All 
non-attendees
IV w/ Two 
instruments
Overall decentralization knowledge 1.23***
(.28)
 1.61***
(.23)
 1.95***
(.32)
Perceived knowledge of 
decentralization
 .26**
(.08)
 .33***
(.08)
 .53***
(.13)
General political knowledge .21
(.11)
 .28**
(.10)
 .36*
(.15)
Perceived knowledge of the 
constitution
.13
(.18)
 .30**
(.08)
 .35***
(.10)
General support for decentralization -.01
(.10)
.11
(.10)
.10
(.17)
Positive aspects of decentralization .06
(.07)
.11
(.06)
.05
(.09)
Negative aspects of decentralization -.11
(.13)
-.19
(.13)
-.30
(.19)
Pace of decentralization is too slow -.03
(.06)
.06
(.07)
-.02
(.11)
Support for decentralization: military 
and police
 -.15**
(.05)
-.14*
(.07)
 -.25**
(.09)
Support for decentralization: roads 
and transportation
-.10
(.06)
-.08
(.06)
 -.24***
(.07)
Support for decentralization: schools 
and education
-.05
(.06)
-.03
(.09)
-.11
(.10)
Support for decentralization: taxation -.06
(.05)
-.09
(.06)
-.12
(.07)
Democracy is the best option -.05
(.04)
.02
(.04)
.00
(.05)
Satisfaction with democracy -.03
(.06)
.04
(.09)
.00
(.09)
Political tolerance .11
(.12)
.11
(.12)
.24
(.21)
Political efficacy .03
(.06)
.03
(.09)
-.07
(.11)
Political discussion .08
(.08)
.05
(.09)
.02
(.13)
Interest in politics .02
(.08)
 .25*
(.12)
.16
(.15)
Political participation .14*
(.06)
-.02
(.09)
.17
(.10)
All estimates are CEM-weighted with clustered robust standard errors in parentheses. N ranges from 1,329 to 1,512. 
***p < .001 (two-tailed); **p < .01 (two-tailed); *p < .05 (two-tailed) 
36
The instrumental variables estimate in column 4 of the table, however, shows that among the 
“purely exogenous” portion of Boîtes attendance, that is, the portion of the treatment village 
individuals who were affected by the randomization manipulation, the Boîtes effect is even 
larger, with a gain in knowledge of close to 2 more correct questions than among non￾attendees. The results show, consistently and through multiple estimation procedures, that 
there were substantial direct effects of attending the Boîtes à Images sessions on individual 
learning about decentralization. 
The table shows further that the same variables —and only those variables— that were 
significant at the village level in Table III-3 show consistently significant effects for Boîtes 
attendance as well. Individuals in treatment villages showed greater gains in general political 
knowledge, in perceived knowledge about decentralization and the DRC constitution, and 
greater support for national authority in the areas of police, military, and roads and 
transportation than individuals in control villages. These effects are largely replicated among 
Boîtes attendees. The self-reported attendance model (column 3) shows significant effects of 
attendance on general and perceived knowledge, support for national authority in policy and 
military matters, and overall political interest. Controlling for the self-selection effects in the 
instrumental variables model shows effects only on the knowledge and national authority 
support items, as the effects of attendance on political interest (and the intent-to-treat effect on 
political participation) are no longer significant. We note that the magnitude of all of these 
effects are relatively modest and substantially weaker than those observed for decentralization 
knowledge. There are no detectable effects of Boîtes attendance on any of the other 
orientations measured in the study. 
 D. “Spillover” Effects of the Boîtes Sessions on Treatment Village Non-Attendees 
The results presented thus far allow us to draw initial conclusions about the extent to which the 
Boîtes sessions may have stimulated “secondary effects” in treatment villages, i.e., among 
individuals who did not attend the event but who may have learned from individuals in their 
families, neighborhood, work or friendships networks who did. We know from Table III-3, for 
example, that the overall village-level effect of the treatment on decentralization knowledge 
was 1.38, so that the average individual in the treatment villages increased on the 
decentralization knowledge scale by that amount compared to the average (comparable) 
individual in the control villages. We also know from Table III-4 that our best estimate of the 
direct effect of the session on those who attended model is somewhere between 1.61 and 1.95. 
Given our estimate of approximately 77% of the treatment village sample attended the event, 
we can then estimate the effect among the remaining 23% as being approximately zero, ranging 
from a high of .38 if the attendee effect is 1.61 to a low of -.52 if the attendee effect is 1.95. 
These calculations yield similar effects on the other knowledge-related variables that were 
shown to have significant village-level effects in Table II-3: general political knowledge (spillover 
effect ranging from .10 to .27); perceived knowledge of decentralization (spillover effect ranging 
from (-.19 to .42) to the constitution (spillover effect ranging from -.03 to .17). In nearly every 
case, the effect of the treatment on non-attendees is substantially smaller than the estimated 
direct effect on attendees, and in most cases we cannot rule out the possibility that there are no 
spillover effects whatsoever. 
At the same time, we argued earlier that individuals who choose not to attend the sessions are 
different on many other relevant factors from self-selected attendees, and they may be very 
37
different from an “average” individual in the control villages as well. We need as a further test 
to compare the changes in knowledge observed among this (self-selected) non-attendee group 
with as comparable a group of individuals in the control villages as possible. We therefore re￾estimate the “difference-in-differences” between the sample of non-attendees and control 
villagers that are matched on all of the political and demographic covariates used in the study 
thus far (age, education, political interest, media attentiveness) and matched on the baseline 
level of each dependent variable.18
 We show these results in Table III-5 below. 
Table III-5: Spillover Effects of Boîtes Sessions
Variable Non-attendees vs. 
Control Village
Overall decentralization knowledge .37
(.26)
Perceived knowledge of decentralization .02
(.17)
General political knowledge .15
(.13)
Perceived knowledge of the constitution .06
(.11)
Support for decentralization: military and police -.15
(.13)
Support for decentralization: roads and transportation -.26***
(.05)
All estimates are CEM-weighted with clustered robust standard errors in parentheses. N ranges from 497 to 687. 
***p < .001 (two-tailed); **p < .01 (two-tailed); *p < .05 (two-tailed) 
The results show that there are no significant spillover effects on any of the variables tested, 
aside from support for national authority over roads and transportation. Our best estimate of 
the effect of the Boîtes session on non-attendees in treatment villages is .37 for the key 
dependent variable of decentralization knowledge, a reduction of some 80% from the effect 
observed on attendees. Reductions of upwards of 67-75% are seen on the other knowledge 
variables as well. We note that if these effects were statistically significant, it would provide 
some evidence of positive spillover, which would be notable, given that it would have resulted 
from single civic education event taking place in the village. 
It may be that a larger study with more villages and more individuals who did not attend the 
event would provide an estimate with greater power. It may also be the case that spillover 
effects were present for a subset of non-attendees, e.g., family members or close friends of the 
participants. Again, a larger study with more detailed information on the social networks of the 
villagers, both attendees and non-attendees would be needed to make these determinations. 
For now, however, we conclude from the analyses that were possible to conduct with our data 
that spillover effects from the Boîtes sessions onto individuals who did not attend the sessions 
were negligible. 
 
18 As in previous analyses, the differencing procedure eliminates stable unobserved factors at both the 
individual and village level that may confound the inferences as well. 
38
E. Effects of Boîtes Session Quality and Timing 
We turn now to investigating the potential differential impacts of the Boîtes sessions, in this 
section focusing on the characteristics or timing of the events themselves. In Section F we 
explore potential differential effects of the sessions on different kinds of individuals. We focus 
on village-level impacts, as they are the most straightforward to estimate (following the models 
in Section A above), and they provide a sense of the total impact that different kinds of Boîtes
sessions may have on individuals in treatment villages. 
As noted above, we have two main hypotheses: that Boîtes sessions of higher “quality” – i.e., 
that are better organized, that have more competent facilitators, and that led to greater 
satisfaction on the part of the participants – would have stronger impacts than sessions that 
were of lower quality. We hypothesize further that some of the gains registered in treatment 
villages may potentially lessen over time, such that individuals in treatment villages whose post￾Boîtes interviews took place closer to the event would show greater impacts than individuals in 
treatment villages who were interviewed well after the event took place. The results of both of 
these tests are of interest not only for assessing the Boîtes (VOICE) program, but also have 
important implications for the implementation of future civic education programs in the DRC 
and elsewhere. 
1. Session Quality 
We showed in Section A above that the events were rated as being of relatively high quality by 
participants: the sessions were thought to be well-organized, the facilitators were through to be 
competent, and there was general satisfaction with the information imparted during the 
sessions. But we also noted significant variation among the sessions in different villages as well. 
We therefore use the average value of the participants’ ratings for each of the sessions as a 
proxy for overall Session Quality, and we exploit the variation in this measure across villages by 
interacting the quality variable with the treatment indicator variable to produce the following 
estimation equation: 
(9) 
0 1 2 3
*
ij i i j i ij     y TREATMENT TREATMENT QUALITY Z         
where Z are control variables including the baseline value of the knowledge variable or other 
orientation tested.19
 It can be seen that the effect of the treatment will differ for every level of 
session quality to the extent that β2 is statistically significant. It will also be possible with this 
model to provide a precise estimate of the effect of the treatment at each level of overall 
quality, and to determine whether this effect is statistically different from zero. We estimated 
this model for all orientations examined in the study, as it may be possible that significant 
effects would be registered at very relatively levels of session quality even for orientations that 
did not up until now show significant overall effects. 
The estimation of this model results in several important findings. First, there is a strong, 
positive interaction effect for the key decentralization knowledge variable, indicating that 
individuals in treatment villages with higher quality Boîtes sessions learned significantly more 
 
19 As in previous analyses, the estimation sample is also “pre-balanced” on the covariates and baseline 
value of the dependent variables through the CEM procedure. 
39
from the event, relatively to comparable individuals in control villages, than did individuals in 
treatment villages with lower quality sessions. We show this effect in Figure III-3 below, with 
the dark line representing the estimated treatment effect and the dotted lines representing the 
95% confidence interval for these estimates. 
It can be seen that the effect of the treatment is statistically significant at all levels of session 
quality, but that the magnitude of the effect rises considerably in villages with higher quality 
treatments. At the highest level of session quality, treatment villagers registered gains of nearly 
2 ½ more correct responses on the decentralization scale than comparable control villagers, 
compared to gains of just less than 1 more correct response among treatment villagers where 
the session quality was lowest. This is strong evidence that proper implementation of the 
session through quality organization and competent facilitators matters greatly in determining 
the magnitude of the session’s impact at the village level.
Figure III-3 
The Effects of Boîtes Sessions of Varying Quality on Changes in Decentralization Knowledge
Second, there are generally no significant differences in the treatment effects on other 
decentralization orientations depending on the quality of the Boîtes session. There is some hint 
that the very highest quality treatments stimulate overall support for decentralization, but we 
cannot statistically rule out the notion that the differences in effects on this variable from 
sessions of different quality is zero. There is also some hint that sessions of higher quality lead 
to greater support for the current pace of decentralization, but regardless of session quality, all 
treatment effects for this variable are statistically indistinguishable from zero. The sole instance 
aside from overall decentralization knowledge where treatment effects do differ meaningfully 
depending on session quality is on the individual’s perceived knowledge of the DRC constitution, 
and we show these results in Figure III-4 below. It can be seen that when session quality is 
lowest, the treatment effect is only .13, with this value being statistically insignificant. As 
session quality rises to its average level, the effect of the treatment doubles to a significant 
value of .24, and as session quality rises to its highest value, the effect of the treatment more 
than doubles from that amount to a significant .51 value. This represents a relatively large 
effect in standard deviation terms of .57. 
.00
.50
1.00
1.50
2.00
2.50
3.00
3.50
3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9
Boîtes
Treatment 
Effect
Session Quality
40
The third, and perhaps most striking, result from these analyses is that there are consistently 
positive interactions of session quality on the treatment effects for the cluster of general 
political engagement variables. That is, sessions that are of higher quality stimulate greater, and 
in many cases, statistically significant gains in treatment villagers’ political interest, participation, 
efficacy, and discussion, relative to comparable control villagers, than sessions of lower quality. 
We show these results in Figures III-5 to Figures III-8. 
Figure III-4 
The Effects of Boîtes Sessions of Varying Quality on Perceived Knowledge of the Constitution
Figure III-5 
The Effects of Boîtes Sessions of Varying Quality on Political Efficacy 
The figures show consistently that high quality sessions affect the cluster of political 
engagement variables – the individuals’ interest in politics, perceived competence in political, 
and levels of discussion and participation-- while low quality sessions do not. Sessions of the 
highest quality are associated with treatment effects of substantially greater magnitude than 
low quality sessions: .36 versus -.05 for participation, for example, .54 versus -.02 for interest, 
.24 versus -.24 for efficacy, and .20 versus -.17 for discussion. This indicates that, beyond the 
-.20
.00
.20
.40
.60
.80
3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9
Boîtes
Treatment 
Effect
Session Quality
-.60
-.40
-.20
.00
.20
.40
.60
3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9
Boîtes
Treatment 
Effect
Session Quality
41
effects that sessions may have on the immediate content of the curriculum, i.e. knowledge 
about the decentralization process, high quality delivery of the session also stimulates the 
individuals’ general engagement with the democratic process. But these effects only took place 
in villages at relatively high levels of session quality, with few of the actual sessions examined in 
the study reaching those levels of quality necessary for these generalized effects to occur. 
Figure III-6 
The Effects of Boîtes Sessions of Varying Quality on Political Discussion
Figure III-7 
The Effects of Boîtes Sessions of Varying Quality on Political Interest
-.50
-.30
-.10
.10
.30
.50
3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9
Boîtes
Treatment 
Effect
Session Quality
-.45
-.25
-.05
.15
.35
.55
.75
3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9
Boîtes
Treatment 
Effect
Session Quality
42
Figure III-8 
The Effects of Boîtes Sessions of Varying Quality on Political Participation
2. Session Timing 
We test the possibilities of “fade-out” effects here, such that the effects of Boîtes sessions may 
dissipate in the days and weeks following the event. The timing of the post-Boîtes interviews, 
as noted above, ranged from a few days to a just under a month, giving us some variation to 
exploit though not enough to determine truly longer-term effects of the sessions (see Finkel et 
al., 2012). We estimate these models by interacting the treatment indicator variable with the 
days since the event took place, and including both of those variables in a model similar to 
equation (9) above for Session Quality. In these models, we also include a variable that 
measures the days between the baseline and post-Boîtes interviews, so that any tendency of the 
change in any of the orientations to occur for individuals in control village depending on the 
timing of the interview would also be taken into account (along with potential changes from the 
baseline interview to the date of the session for individuals in treatment villages as well).20
 We estimated these models for all variables considered in the study. The results strongly 
parallel the findings from the Session Quality models: there are significant “fade-out” effects 
seen for the key knowledge variables related to the decentralization process and perceived 
knowledge of decentralization and the DRC constitution, and significant “fade-out” effects for 
the political engagement variables of interest and participation as well. We show these results 
in graph form in Figures III-9 to III-11 for the knowledge items, and Figures III-12 and III-13 for 
the engagement variables. 
It can be seen in Figure III-9 that the positive effects of the Boîtes sessions on decentralization 
knowledge are much stronger immediately following the event than they are after one month 
has passed between the event and the wave 2 interview. Gains in correct knowledge after 3 
days (the minimum value for this variable in the study) for individuals in treatment villages are 
nearly 2 items greater than for comparable individuals in control villages, while this effect falls 
 
20 As in previous analyses, other control variables were also included, and the treatment and control 
villages were balanced initially using the CEM procedures.
-.30
-.15
.00
.15
.30
.45
.60
3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9
Boîtes
Treatment 
Effect
Session Quality
43
to 1.5 items greater for treatment villagers interviewed 10 days after the event, and drops by an 
additional 50% to .76 items greater for treatment villagers interviewed 26 days after the event 
(the maximum value for this variable in the study). This is evidence that the learning taking 
place as a result of the session, and post-session diffusion in the treatment villages, is much 
stronger in the days immediately following the session than it is nearly one month later. While 
the effect is statistically significant regardless of when the post-Boîtes interview took place, its 
magnitude drops by some 60% from the earliest post-Boîtes interviews to the latest. While we 
cannot extrapolate beyond the time period covered in the study, we can say that it does not 
appear that the Boîtes sessions produced as great a longer-term learning effect compared with 
its immediate one. 
Figure III-9 
The Effect of Time Elapsed Since Boîtes Session on Decentralization Knowledge
Figure III-10
The Effect of Time Elapsed Since Boîtes Session on Perceived Knowledge of Decentralization 
.00
.40
.80
1.20
1.60
2.00
2.40
2.80
1 6 11 16 21 26
Boîtes
Treatment 
Effect
Days to Post-BoîtesInterview
-.40
-.10
.20
.50
.80
1 6 11 16 21 26
Boîtes
Treatment 
Effect
Days to Post-BoîtesInterview
44
Figure III-11
The Effect of Time Elapsed Since Boîtes Session on Perceived Knowledge of the Constitution 
Figure III-12
The Effect of Time Elapsed Since Boîtes Session on Political Interest 
Figure III-13
The Effect of Time Elapsed Since Boîtes Session on Political Participation
-.56
-.26
.04
.34
.64
.94
1 6 11 16 21 26
Boîtes
Treatment 
Effect
Days to Post-BoîtesInterview
-.60
-.40
-.20
.00
.20
.40
.60
.80
1 6 11 16 21 26
Boîtes
Treatment 
Effect
Days to Post-BoîtesInterview
-.40
-.20
.00
.20
.40
.60
.80
1 6 11 16 21 26
Boîtes
Treatment 
Effect
Days to Post-BoîtesInterview
45
An even steeper “fade-out” effect is shown for perceptions of knowledge about 
decentralization, and about the DRC constitution in Figures III-10 and III-11. Here the effects 
immediately following the sessions are estimated at approximately .60, more than double their 
“average” values as reported in Table III-3. But this effect falls to .35 by day 10, and becomes 
statistically insignificant and approaching zero by the third week after the session. This pattern 
suggests that the sessions provided an immediate boost in treatment individuals’ perceptions of 
their competence about decentralization and constitutional issues, but that this boost was not 
manifest among individuals interviewed beyond the first few weeks post-treatment. These 
same patterns are shown in Figures III-12 and III-13 for political interest and participation: 
stronger effects in the immediate time period after the Boîtes session – effects that are much 
greater than the average effect seen in Table III-3– but weak and statistically insignificant effects 
by the third week post-treatment. 
It is important to note that we cannot say with certainty that individuals are “forgetting” what 
they initially learned in the sessions, nor that there are initial boosts in efficacy and engagement 
that fade out over time. Since we do not have longitudinal data on the same individuals 
interviewed at different points post-treatment, we do not know what the effects may have been 
for those individuals as time passed after the Boîtes session took place. While the patterns are 
consistent with a “forgetting” process, they may also be consistent with other processes, for 
example, a correlation between session quality and post-session timing producing the “illusion” 
of a fade-out effect. In fact, the correlation between our “quality” and “time of post-Boîtes
interview” among treatment individuals is -.72, indicating that sessions in which wave 2 
interviews were conducted closer to the event were also rated more highly than when they 
were conducted several weeks later. This makes it very difficult to disentangle the independent 
effects of “quality” and “timing” in determining the magnitude of the Boîtes treatment’s impact.
Figure III-14 
The Effect of Session Quality and Timing on Decentralization Knowledge
To the extent that we can estimate independent effects, however, it appears that both factors 
are important. In Figure III-14, we show the estimated effect of the Boîtes events on 
decentralization knowledge for sessions with different levels of quality and different times of 
the post-session interviews. It can be seen that, at every level of quality, individuals interviewed 
at a greater distance from the event show smaller gains, compared to comparable individuals in 
.00
.40
.80
1.20
1.60
2.00
2.40
2.80
1 6 11 16 21 26
Boîte 
Treatment 
Effect
Days to Post-Boite Interview
High Quality
Average Quality
Low Quality
46
control villages, than individuals interviewed closer to the event. It can also be seen that, at 
every time period following the event, sessions of greater quality produce greater gains in 
knowledge. Taken together, the two factors powerfully condition the effects of the treatment: 
when individuals are interviewed very close in time to when high quality sessions took place in 
their village, they show gains of over 2 correct scores on decentralization knowledge compared 
to comparable control villagers; but when individuals are interviewed weeks after a session of 
relatively lower quality, the effects fall by over 2/3 in terms of magnitude. 
F. Effects of Boîtes Sessions on Different Kinds of Individuals 
The final set of analyses tests for differences in the effects of the Boîtes events on different 
kinds of individuals. Previous evaluations suggest somewhat great support for the notion that 
exposure to civic education has greater effects on those individuals in greater “need”, i.e., those 
less well-educated, less socially-connected, and, in the case of Kenya in 2007, those whose 
households had experienced post-election violence (Finkel and Smith 2011; Finkel et al., 2012). 
These effects were generally not substantial, however, and other work has found support for 
civic education having differentially greater effects on political participation among those with 
higher levels of pre-existing political resources as well (Finkel 2003). We examine these 
processes here by interacting the “treatment” indicator with sex, education, and membership in 
secondary organizations to test whether Boîtes effects are larger for men in treatment villages 
than for women, those with more or less education, and those who belong to many versus few 
voluntary associations. We found no consistent evidence that the effects of the Boîtes sessions 
differed across these characteristics of individuals in treatment villages. With few noteworthy 
exceptions, there were similar effects (i.e., non-significant interactions) of the Boîtes session for 
treatment village men and women, for individuals with different educational attainment, and 
for individuals with varying levels of group memberships. We also examined effects for 
individuals who had “lost family members in the recent armed conflict” and found no 
differential impact compared to other treatment village individuals as well. 
Figure III-15
The Effects of Baseline Levels of Decentralization Knowledge on Boîtes Impact
.00
.40
.80
1.20
1.60
2.00
2.40
2.80
3.20
0 1 2 3 4
Boîtes
Treatment 
Effect
Pre-Boîtes Decentralization Knowledge
47
However, we did find suggestive evidence that the Boîtes treatment had greater effects on 
those with lower initial levels of decentralization and perceived knowledge of decentralization 
and the DRC constitution. We constructed interaction terms between the “treatment” indicator 
and the individual’s baseline level of decentralization and perceived constitution knowledge, 
and showed in both cases that the effects of the Boîtes event were greatest among those with 
the lowest initial scores on each of the orientations. These effects are shown in graph form in 
Figures III-15, III-16, and Figure III-17. 
Figure III-16: 
The Effects of Baseline Levels of Perceived Decentralization Knowledge on Boîtes Impact 
Figure III-17 
The Effects of Baseline Levels of Perceived Knowledge of the Constitution on Boîtes Impact 
The figures show that the Boîtes effects are considerable larger among treatment village 
individuals with lower initial scores than higher. Figure III-15 shows that, among individuals with 
no prior decentralization knowledge, the gains among treatment village individuals are over 1.85 
correct answers more than individuals in control villages. This compares to gains of 1.39 for 
individuals with “average” levels of pre-Boîtes decentralization knowledge and gains of only 1 
.00
.20
.40
.60
.80
1.00
1.20
1.40
1 1.3 1.6 1.9 2.2 2.5 2.8 3.1 3.4
Boîtes
Treatment 
Effect
Pre-Boîtes Perceived Knowledge of Decentralization
.00
.20
.40
.60
.80
0 0.2 0.4 0.6 0.8 1
Boîtes
Treatment 
Effect
Pre-Boîtes Perceived Knowledge of the Constitution
48
item more for individuals who answered four of the six questions correctly in the baseline 
interview. While the gains are significant for all individuals, the results do suggest that the 
effects of the treatment are greater among those with lower initial levels of decentralization 
knowledge, with the same pattern being found for perceptions of decentralization and 
constitution knowledge in Figure III-16 and Figure III-17. 
It is important to note that these effects are not simply due to the well-known “regression to the 
mean” effect in longitudinal analyses, whereby individuals with lower levels of a variable tend to 
increase more over time than individuals at higher levels, due to floor and ceiling effects and 
due to random factors pulling unusually high and low scores back to the “average” at a 
subsequent time of measurement. These processes are already taken into account in the 
model, as treatment and control village individuals are matched on their initial levels through 
the CEM process implemented in all models in the analyses. So whatever “regression to the 
mean” processes exist should be occurring with equal prevalence in both treatment and control 
villages, and any additional interaction between the treatment and the individual’s initial level 
should reflect a “true” differential effect.
To this extent, the findings support the notion that Boîtes effects are stronger among those with 
greater “need” for the knowledge imparted in the sessions, though this “need” is not necessarily 
concentrated among individuals in particular demographic groups. Rather, the “need” is based 
on prior levels of decentralization and constitutional knowledge per se, and the sessions appear 
to have been relatively more successful in stimulating knowledge gains among these individuals 
in the treatment villages. 
Chapter IV. Conclusions 
This study was undertaken to determine whether, how, and under what conditions the Voter 
Opinion and Involvement through Civic Education Program (VOICE) in the Democratic Republic 
of the Congo — more specifically, the Boîtes à Images sensitization sessions that were the 
program’s core civic education tool — was successful in changing individuals’ knowledge and 
attitudes about the ongoing decentralization process in the country, and in changing more 
general democratic orientations and political engagement. To accomplish this task, we 
implemented a complex research design with two components: first, a comparison between 
respondents from 8 villages in Bandundu province where Boîtes à Images s sessions took place 
and 8 villages from similar “control villages” in neighboring groupements where sessions did not 
take place; and second, a field-experimental “encouragement design” whereby 100 individuals 
in each of 8 villages were randomly encouraged at the end of a baseline interview to attend the 
upcoming Boîtes session, while 40 randomly-selected individuals in those “treatment villages” 
received no information about the workshop. Follow-up interviews were conducted with all 
respondents between one day and one month after the workshops in the treatment villages 
took place. 
The study is one of the first evaluations of adult civic education programs to make use of 
experimental methods, which provide greater confidence in the attribution of causal effects 
than has been possible in the mostly passive observational designs in previous research in the 
field. To our knowledge, it is the first that implements an “encouragement manipulation”, one 
that preserves the benefits of experimentation in a situation where it is not possible to 
49
randomly assign individuals to experience a treatment, nor to restrict exposure to those 
specifically chosen individuals. 
The study points to a series of important findings about the kinds of effects that the Boîtes à 
Images sessions had, and the conditions under which the sessions had their strongest impacts. 
Some of the findings are unique to the DRC and VOICE experience, but some also echo the 
conclusions that were drawn in the previous evaluations of adult civic education undertaken on 
behalf of USAID in countries such as Kenya, South Africa, the Dominican Republic, and Poland 
(Finkel 2003; Finkel and Smith 2011). In this section we first summarize these key results; we 
then discuss the implications the findings have for the design, implementation, and evaluation 
of future civic education programs in developing democracies. 
A. Major Findings 
There were large differences between individuals in control and treatment villages in terms of 
their changes in factual knowledge about decentralization before and after the Boîtes events 
took place in the treatment villages. While respondents in control villages increased to some 
extent on decentralization-related knowledge (with the percentages of correct responses 
increasing between 5-10% for most questions), respondents in treatment villages increased at 
substantially higher rates (between 12 and 45% increases in correct responses for each 
question). On an overall scale of “correct decentralization knowledge” (ranging from 0 correct 
to 6 correct answers), the average respondent in control villages increased by only .45 correct 
answers over time, while the average respondent in treatment villages increased by 1.5 correct 
answers from the pre- to post-Boîtes interviews. The effect is even more dramatic if one 
examines the proportions of individuals who increased at all in their decentralization knowledge 
over time: in the control villages, just over 1/3 of all individuals (36%) registered some increase; 
this figure nearly doubles to 69%, or more than 2/3 of all individuals in treatment villages. These 
effects remains strong in the context of regression models where the treatment and control 
samples are carefully balanced on possible confounding variables, and in models that control for 
the individuals’ pre-Boîtes levels of the decentralization knowledge. These effects are 
impressive, especially given that they stem from a single civic education event conducted at the 
village level. To the best of our knowledge, these are some of the largest effects registered in 
the several adult civic education evaluation studies previously conducted, and certainly the 
largest registered in these studies on political knowledge (Finkel 2003; Finkel and Smith 2011; 
Finkel et al. 2012). The Boîtes sessions do appear to have stimulated substantial gains in 
knowledge about the decentralization process at the village level from the baseline wave of 
observation to the post-Boîtes interviews. 
We also found village-level effects of the Boîtes sessions on other knowledge-related items, such 
as the individual’s general knowledge about DRC political institutions and actors, and on the 
individual’s perception of their knowledge about decentralization and the DRC constitution. 
These effects were of smaller magnitude than those observed on the general decentralization 
knowledge scale, but they do indicate that the Boîtes sessions were beneficial to some degree in 
stimulating general political learning, and in stimulating greater confidence among individuals in 
their own store of decentralization and constitutional knowledge. These findings reinforce the 
conclusion above that the Boîtes sessions appear to have been successful in their immediate 
goal of transmitting information about decentralization and the DRC political process to their 
target audiences.
50
However, there we found few corresponding effects on individual attitudes about, or support 
for, the decentralization process. There were no differences between the changes in support 
for decentralization among respondents in control villages and respondents in treatment 
villages, no differences between treatment and control respondents in terms of their 
perceptions of the “good” and “bad” features of decentralization, or their satisfaction with the 
current pace of the decentralization process. Individuals in treatment villages, moreover, 
tended to change slightly in the direction of greater, not lesser, national authority in a series of 
policy areas (roads and transportation, schools and education, police and military, taxation), 
compared to comparable individuals in control villages. There were also no effects of the Boîtes 
session on more general orientations about democracy, on political interest, discussion, or 
efficacy, nor on political participation. The effects of the Boîtes events appear to have been 
more or less exclusively concentrated in the realm of decentralization and general political 
knowledge, with no concomitant effects on decentralization attitudes or other political 
orientations or behaviors. These non-effects should be taken into account in the overall 
assessment of the VOICE program’s effectiveness; they also provide cautionary evidence against 
the notion that increasing knowledge about potentially contentious political processes such as 
decentralization will translate readily into support for those processes among ordinary 
individuals (see also Green et al. 2011). 
We exploited the “randomized encouragement” feature of the design in order to determine the 
direct effects of attending the Boîtes event among individuals in treatment villages. Using a 
variety of “intent to treat” and “instrumental variables” models to overcome possible biases 
related to self-selected exposure to the treatment, we found consistent evidence that Boîtes 
attendees registered large and significant gains on decentralization knowledge, relative to non￾attendees, with these effects being even larger than the overall village effects mentioned above. 
Boîtes attendees also were found to have gained significantly on the other variables (general 
knowledge and perceptions of decentralization and the DRC constitution, support for national 
authority in roads, police and military affairs) shown to be significant at the village level as well. 
Further analysis showed minimal “spillover effects” from attendees to others in their family, 
neighborhood or friendship networks. Taken together, the results suggest that virtually all of 
the village-level impacts on the knowledge and decentralization orientations were concentrated 
among those who attended the Boîtes sessions. The sessions appear to have stimulated little, if 
any, secondary learning due to discussion or spillover from attendees to non-attendees in the 
village. These non-effects differ somewhat from previous findings of positive spill-over in civic 
education evaluations conducted in Kenya (Finkel and Smith 2011) and Nigeria (Fafchamps and 
Vicente 2011). It seems reasonable to speculate that the single, relatively brief Boîtes exposure 
was not sufficient to stimulate the kind of post-treatment diffusion via interpersonal discussion 
necessary for spillover effects to occur, especially given the breakdown of the decentralization 
process at the elite level and its relatively low salience in the DRC at the time the study was 
conducted. 
We further examined the conditions under which the Boîtes sessions produced larger impacts. 
Three significant findings emerged. First, the quality of the Boîtes session mattered a great deal. 
Events with higher reported satisfaction of the participants regarding the organization, 
information delivery and competence of the facilitators showed substantially stronger effects on 
decentralization knowledge and many of the other knowledge-related items of lesser perceived 
quality. Importantly, individuals who attended the highest quality events often showed 
significant increases in the cluster of orientations related to political engagement: interest, 
51
efficacy, political discussion and participation, relative to comparable individuals in control 
villages. The findings echo previous research, where a consistent result has been that higher 
quality civic education implementation is strongly associated with individual-level effects of 
greater magnitude. As was found previous research, however, the numbers of individuals in 
DRC treatment villages who were trained at the highest levels of session quality were relatively 
small. Thus, the results indicate both the strong potential for effects from high quality civic 
education, as well as the more limited actual impacts the sessions had on decentralization and 
political engagement outcomes. 
Second, the time since the event mattered as well. Treatment village respondents interviewed 
closer to a month after the Boîtes event showed substantially less improvement in 
decentralization knowledge, relative to comparable individuals in control villages, than 
respondents interviewed in days immediately following the event. Treatment effects decline by 
over 60% from the day after the event to 26 days later on the key variable of decentralization 
knowledge, with even steeper declines on the other knowledge-related political engagement 
orientations. This pattern is consistent with a “fade-out” effect, such that individuals are 
forgetting after several weeks a large portion of what they may have learned in the days 
following the treatment. We cannot show this conclusively with the data at hand, however, 
since individuals interviewed at different times after the sessions had been exposed to different 
treatments. It is also the case that individuals interviewed well after the event report lower 
perceptions of session quality, making it difficult to disentangle the separate effects of each on 
the magnitude of the treatment effects. Nevertheless, this pattern shows strong support for the 
notion — consistent with previous work — that the single-shot Boîtes treatments may not have 
produced lasting impact on the orientations they were designed to affect. 
Finally, we found no consistent evidence that the effects of the Boîtes sessions differed across 
individuals in treatment villages with different demographic, political, or social characteristics. 
There were similar effects of the Boîtes session for men and women, for individuals with 
different educational attainment, for individuals with varying levels of secondary group 
membership, and for individuals who had and had not lost family members in the recent DRC 
armed conflict. However, we did find suggestive evidence that the Boîtes treatment had greater 
effects on those with lower initial levels of decentralization and general political knowledge. In 
every instance, knowledge gains from the Boîtes events were greater among those with the 
greatest “need” for the information. This provides support for a process whereby adult civic 
education has a “compensation effect” in altering the orientations of individuals with fewest 
cognitive resources (see also Finkel and Smith 2011). 
B. Implications for Civic Education Design, Implementation, and Evaluation 
The study’s findings and conclusions have definite implications for how civic education programs 
related to both decentralization and more general democratic orientations should be structured 
in the future. We make the following recommendations for civic education design, 
implementation, and evaluation. 
1. Future civic education programs should continue to use “Boîtes à Images” and other visual 
materials. The use of these kinds of images and visual appears to have been an effective means 
for the transmission of basic civics information related to decentralization and democratic 
political processes in rural populations characterized by relatively low levels of adult literacy. 
52
Moreover, the fact that the effects of the session examined here were greatest among those 
with lowest levels of prior information further validates the image-based approach utilized in 
the VOICE program, given that individuals with a lower store of political information are likely to 
have lower levels of literacy as well. 
2. Future programs must go beyond “one-off” village-level treatments in order to maximize 
individual level impact. It is clear from this study that immediate short-term gains in knowledge 
about the decentralization and general political process are possible through single-shot civic 
education treatments. Indeed, as has been noted throughout the report, the magnitude of the 
knowledge gains on the specific orientation that the VOICE program was designed to influence 
was large and, in comparison to results found in previous research, impressive. At the same 
time, there were negligible effects of the program on virtually all other orientations studied, and 
the effects of the sessions themselves appear to have faded significantly over time as well. 
These patterns conform well to the findings from previous work, where it has been established 
with a high degree of regularity that multiple exposures to civic education treatments are 
necessary for enduring effects to occur, and that multiple exposures are particularly necessary 
in altering orientations, attitudes, and political engagement besides factual knowledge. Thus, 
while we endorse the general form of the civic education delivery in the VOICE, future programs 
should be strongly encouraged to build in multiple exposures in order to reinforce the messages 
to which individuals are exposed and to ward off the “fade out” effects that appear inevitable in 
their absence. 
3. Future programs must give greater emphasis to the training of facilitators and to improving 
the general organization of civic education treatments. Although the Boîtes à Images sessions 
were very popular with most participants, there was nevertheless considerable variation in the 
satisfaction that individuals registered with the organization of the session, with the quality of 
the information that was conveyed, and with the competence of the facilitators. And this 
variation was directly related to the size of the effects observed from the Boîtes sessions, with 
higher quality sessions showing substantially larger impacts than those rated of lower quality. It 
was also the case that high quality workshops were able to influence more general orientations 
related to democratic engagement and not “only” stimulate factual learning among participants. 
But the proportion of individuals exposed to sessions at this level of quality was relatively small, 
thus inhibiting more extensive effects that could have been observed. We recommend that 
significantly more resources be devoted to facilitator training and the organization of workshops 
in the future. “One-off” sessions conducted with inadequately trained and insufficiently 
knowledgeable facilitators are a recipe for limited individual-level impact, yet large numbers of 
individuals continue to be trained in these ways in otherwise well-conceived civic education 
programs. 
4. Future programs should emphasize post-treatment discussions of participants to maximize 
the possibilities of spillover effects. We found little evidence of secondary effects from post￾Boîtes discussions in treatment villages, and this was a “missed opportunity” in terms of 
increasing the overall impact of the sessions in important ways. It also differed considerably 
from the patterns seen elsewhere in Africa, where individuals exposed to civic education went 
on to discuss their experiences with others, with secondary learning effects seen among those 
individuals as a result. Civic education programs should exploit this possibility to maximize both 
the reach and the impact of future activities. Participants in future programs should be 
encouraged explicitly to discuss decentralization and general topics related to democratic 
53
political processes with family, friends, colleagues, and neighbor, and they should be provided 
with instructional materials in how best to do so. The more individuals are taught how to seek 
out and discuss civics materials with others both in and out of their immediate social networks, 
the more future programs will extend their reach and their impact throughout Congolese 
society. 
5. Future evaluations should continue to exploit experimental designs in order to make more 
credible causal inferences about program effects, while recognizing their potential limitations in 
certain settings as well. The use of the randomized “encouragement manipulation” was a key 
component of the research design here, and it allowed us to make stronger statements about 
the effects of civic education exposure, controlling for the self-selection biases that have 
hindered causal inference in previous work. We recommend that future evaluations 
incorporate these and other kinds of experimental manipulations to randomize treatment 
assignment and treatment exposure. At the same time, there were large number of non￾compliers with the encouragement manipulation, and especially large numbers of non￾encouraged individuals who nevertheless attended the Boîtes session. This means that much 
larger samples of treatment village individuals will be necessary in future research in order to 
maximize the statistical power of encouragement designs, and to generate sufficient numbers of 
non-attendees in treatment villages so that more extensive analysis can be conducted on this 
group. It also means that more efforts should be devoted to validating individual attendance at 
civic education events in order to provide more objective measures of individual-level exposure. 
Finally, the success of the experimental manipulations depends to some degree on minimizing 
discussions about the events between encouraged and non-encouraged individuals. To the 
extent that large amounts of interactions between encouraged and non-encouraged individuals, 
and between attendees and non-attendees, it will be that much more difficult to design studies 
that can distinguish the effects from direct and indirect exposure among individuals in 
treatment villages at all. We therefore recommend that future evaluations be certain to 
increase the number of (randomized) treatment and (randomized) control areas so that, 
regardless of the ease with which direct versus indirect spillover effects can be distinguished, 
the casual impacts of civic education interventions can be estimated with greater confidence at 
the overall village level. 
54
References 
Almond, Gabriel, and Sidney Verba. 1963. The Civic Culture: Political Attitudes and Democracy in Five 
Nations. Boston, MA: Little, Brown & Co. 
Bratton, Michael et al. 1999. “The Effects of Civic Education on Political Culture: Evidence from 
Zambia.” World Development 27(5): 807-824. 
Collier, Paul, and Pedro C. Vicente. 2009. “Votes and Violence : Evidence from a Field Experiment in 
Nigeria.” (Working Paper). 
Diamond, Larry. 1999. Developing Democracy: Toward Consolidation. Baltimore: Johns Hopkins 
University Press. 
Dizolele, Mvemba Phezo. 2010. “The Mirage of Democracy in the DRC.” Journal of Democracy 21 
(3):143-157. 
Duflo, Esther, Rachel Glennerster, and Michael Kremer. 2007. “Using Randomization in Development 
Economics Research: A Toolkit.” Handbook of Development Economics 4(6059): 3895–3962. 
Fafchamps, Marcel, and Pedro C. Vicente. 2010. “Political Violence and Social Networks : 
Experimental Evidence from a Nigerian Election.” (Working Paper). 
Finkel, Steven E. 2002. “Civic Education and the Mobilization of Political Participation in Developing 
Democracies.” The Journal of Politics 64(04): 994-1020. 
Finkel, Steven E. 2003. “Can democracy be taught?” Journal of Democracy 14(4): 137–151. 
Finkel, Steven E., Jeremy Horowitz, and Reynaldo T. Rojo-Mendoza. 2012. “Civic Education and 
Democratic Backsliding in the Wake of Kenya’s Post-2007 Election Violence.” The Journal of 
Politics 74(1): 52-65. 
Finkel, Steven E., and Amy Erica Smith. 2011. “Civic Education, Political Discussion, and the Social 
Transmission of Democratic Knowledge and Values in a New Democracy: Kenya 2002.” 
American Journal of Political Science. 
Green, Donald P. et al. 2011. “Does Knowledge of Constitutional Principles Increase Support for Civil 
Liberties? Results from a Randomized Field Experiment.” The Journal of Politics 73(2): 463-476. 
Khandker, Shahidur R., Gayatri B. Koolwal, and Hussain A. Samad. 2010. Handbook on Impact 
Evaluation: Quantitative Methods and Practices. Washington, D.C.: The World Bank. 
Paluck, Elizabeth Levy, and Donald P. Green. 2009. “Deference, dissent, and dispute resolution: An 
experimental intervention using mass media to change norms and behavior in Rwanda.” 
American Political Science Review 103(04): 622–644. 
Rosenbaum, Paul R. 2002. Observational Studies. New York: Springer Verlag.