1
Impact Evaluation of the Niger Participatory & Responsive Governance Project: 
Baseline Report 1
October 2017
Submitted to:
USAID/LAB/HESN
USAID/DCHA/DRG
USAID/Niger
Submitted by:
Ariel BenYishay
Assistant Professor, College of William and Mary
Chief Economist, AidData
Lisa Mueller
Assistant Professor, Macalester College
Member, AidData Research Consortium
Phil Roessler
Associate Professor, College of William and Mary
Member, AidData Research Consortium
Contact Information:
Ariel BenYishay
abenyishay@aiddata.wm.edu
Lisa Mueller
lmueller@macalester.edu
Phil Roessler
Proessler@wm.edu
1 This baseline report is submitted to USAID/USAID Niger, USAID/DCHA/DRG, and USAID/LAB/HESN by AidData 
(AID-OAA-A-12-00096), headquartered at the College of William and Mary. This evaluation is funded jointly by 
USAID/DRG and USAID/HESN through a contract to NORC and a cooperative agreement currently in place between 
USAID/LAB/HESN, AidData, and William and Mary. 
2
Table of Contents
Executive Summary...................................................................................................................................................................4
Introduction..................................................................................................................................................................................6
Impact Evaluation Overview.................................................................................................................................................6
Treatment: PRG-PA Implementation ..........................................................................................................7
Theory of Change........................................................................................................................................................................9
Summary of Monitoring and Evaluation Strategy.....................................................................................................12
Impact Evaluation Design and Implementation .................................................................................. 12
Randomization Design................................................................................................................................... 13
Sampling Design and Power ........................................................................................................................ 14
Household survey sample design ...........................................................................................................................14
Commune Level Official Sample Design .................................................................................................. 16
Data Collection..........................................................................................................................................................................16
Household and Leader Survey Fieldwork Details ............................................................................... 16
Selection and Replacement Protocols for Households...................................................................... 17
Quality Control and Data Processing........................................................................................................ 18
Hypotheses and Measures...................................................................................................................................................19
Household Survey Findings ......................................................................................................................... 19
Summary Statistics: Household Survey ...............................................................................................................19
Balance: Household Survey.......................................................................................................................................21
Baseline conditions: Household Survey...............................................................................................................21
Commune Leader Survey Findings ........................................................................................................... 30
Summary Statistics: Leader Survey .......................................................................................................................30
Baseline Conditions: Leader Survey......................................................................................................................30
Balance: Leader Survey...............................................................................................................................................36
Field challenges................................................................................................................................................ 36
Administration, Access to communities, Logistics, and Military incidence .........................................36
Pre-Analysis Plan.....................................................................................................................................................................37
Timeline.......................................................................................................................................................................................38
Roles and Responsibilities...................................................................................................................................................38
Research Team:................................................................................................................................................ 38
AidData:............................................................................................................................................................... 39
Implementing Partner (Counterpart International): ........................................................................ 39
Survey partner (NORC):................................................................................................................................. 39
USAID/Niger:..................................................................................................................................................... 39
USAID/DRG: ....................................................................................................................................................... 40
USAID/HESN: ..................................................................................................................................................... 40
AidData Staffing and Management Plan ........................................................................................................................40
Principal Investigators .................................................................................................................................. 40
Additional Personnel ..................................................................................................................................... 41
3
List of Figures and Tables:
Figure 1: Treatment and Control Assignments .........................................................................................................14
Figure 2: Simulation Results ..............................................................................................................................................15
Table 1: Summary Statistics ...............................................................................................................................................20
Figure 3: Does the Media Abuse Freedoms by Printing Untrue Things?.........................................................22
Figure 4: Are Leaders Concerned With Serving Their Own Needs or People’s Needs?............................24
Figure 5: How Free Are You To Choose Whom To Vote For?...............................................................................26
Figure 6: How Much of a Democracy is Niger .............................................................................................................27
Figure 7: Voting in the Last Election ...............................................................................................................................28
Figure 8: Free and Fairness of Last Election................................................................................................................29
Figure 9: How Often Do Government Officials Communicate With Residents .............................................31
Figure 10: Have You Ever Heard About Corruption Involving the Current President of Niger? ..........33
Figure 11: Which Describes What Happened at the Last Commune Assembly Session When a 
Decision Had to Be Made on a Commune Issue?........................................................................................................35
Figure 12: Does Any Particular Group Among Poor, Middle, and Rich Residents Dominate Decision 
Making In Assemblies?..........................................................................................................................................................36
4
Executive Summary
Across most development indicators, Niger ranks close to the bottom of global rankings. Over the past 
three years, it has been second to last on the United Nation’s Human Development Index—just above 
Central African Republic. At root of the country’s perennial development crisis are unfavorable structural 
conditions. It is landlocked, arid, susceptible to erratic weather patterns, and heavily reliant on a single 
export—uranium—making it vulnerable to commodity price shocks. These structural problems contribute 
to and are compounded by significant governance challenges. One of the biggest is the absence of an 
effective state—one that can provide citizens with basic public services that are necessary for inclusive 
development. In its place, the state is dominated by a small group of business, military and administrative 
elites who compete over the private distribution of scarce public resources. A second major challenge is 
the militarization, or securitization, of the state, in which military and security institutions represent the 
face of the government for many Nigerien citizens and divert resources from public administration. These 
governance challenges have given rise to a ‘crise de confiance’ or ‘crisis of confidence’ between citizens 
and the state. Low-levels of state legitimacy threaten to continue to erode citizen support for democracy 
and risk strengthening the hand of the military or extremist organizations. 
To help mitigate these governance challenges in Niger and bolster stability in one of the world’s most 
fragile states, USAID is investing in the Participatory and Responsive Governance (PRG) Project. The 
overarching goal of the PRG project is to strengthen the collective responsiveness of the Nigerien 
government to its citizens’ priority public needs. The project aims to strengthen collective responsiveness 
through three channels: 1) political party campaigns; 2) collective engagement and coordination of 
multiple stakeholders (government, non-government, donors) to undertake reforms that address citizen 
priorities; and 3) capacity-building of local think-tanks, media, NGOs, and civil society to promote 
participatory governance.
To better understand how the PRG may affect governance in Niger, a team of researchers from AidData 
at the College of William and Mary have teamed up with the implementing partner, Counterpart 
International (CPI), to undertake a rigorous impact evaluation of the program. The impact evaluation 
focuses on the PRG’s multi-stakeholder dialogues that will bring together community leaders, municipal 
and regional councilors, private sector actors, professionals and citizens to confer upon, design and 
initiate Regional Development Plans (PDRs) and Communal Development Plans (PCDs). The expectation is 
the multi-stakeholder dialogues will catalyze development and government responsiveness both in terms 
of process and outcomes: the dialogues are intended to serve as focal points, or coordination 
mechanisms, for a diverse set of actors to come together to bring multiple perspectives to bear on the 
challenge of community development as well as marshaling the resources and forging the strategic 
partnerships necessary to ensure the development plan is implemented. Multi-stakeholder dialogues 
have strong potential to impact relations between Nigerien leaders and citizens because of the historical 
context in which they will take place. Dialogue is a major pillar of Nigerien democracy dating back to the 
transition to multiparty competition in 1992, which occurred at a National Conference involving 
stakeholders from diverse sectors of society (politics, labor, education, religion, etc.).
5
The evaluation employs a randomized design to test this theory of change, in which the 24 communes 
where targeted activities be held were randomly selected from a broader pool of 48 eligible communes. 
We refer to the 24 communes randomly selected for these activities as the “treatment” group, and the 
remaining 24 communes as the “control” group. This report provides an overview of the design of the 
impact evaluation as well as findings from a baseline household and community leader survey carried out 
in January-March 2017. The household survey entailed a sample of 1,258 households using stratified 
random sampling at the village-level within communes. The leader survey target was 144 surveys across 
the 48 communes (3 interviews per commune) but only 118 surveys were completed due to availability 
and logistical challenges. Follow-up surveys are planned for late 2018, with outcome analysis to follow.
The baseline surveys are valuable to gauge levels of perceived legitimacy prior to the implementation of 
the multi-stakeholder dialogues as well as to evaluate statistical balance between the control and 
treatment communes. Overall, the study communes are well-balanced, with similar demographic profiles, 
religious and ethnic compositions, levels of political participation, and satisfaction with Nigerien
democracy across the treatment and control groups. As expected, there are a small number of 
characteristics that are not completely balanced across the groups due to random chance, including their 
ethnic diversity, political freedom, and voter turnout. We can address these imbalances by controlling for 
these variables in the statistical analysis when evaluating the impact of the PRG program on perceived 
state legitimacy and other related outcomes.
Substantively, as we describe in the report, the baseline survey highlights the governance challenges Niger 
faces and the importance of the PRG program. Overall, participants in the household survey report a 
general frustration with government—from the president to the national assembly down to local 
government—in terms of receptiveness to citizen demands, responsiveness, and putting public interest 
above personal interest. In another sign of low systemic legitimacy, despite high dissatisfaction, there is 
high levels of political disengagement. The majority of respondents have taken very few political actions 
when they have been dissatisfied with their government’s performance. Equally problematically, citizens 
do not see the media or political parties as effective institutions in holding government accountable. This 
crisis of confidence in civilian institutions seems to be filled by higher levels of trust in security institutions. 
Respondents indicate that they are more willing to trust and respect the role of the military and police 
than politicians and political parties.
Despite these significant challenges and important opportunities for strengthening governance, on 
several dimensions citizens’ revealed preferences suggest enduring support for the Nigerien political 
system. For example, when it comes to taxation—the sine qua non of the state-society compact—most 
participants believe the government has the right to collect taxes and citizens have the obligation to pay 
them. Similarly, most see Niger as a democracy and believe it is the best form of government. 
 
Finally, and most relevant for the PRG program, more than 70 percent of survey respondents agree that 
community dialogue among local leaders and citizens is an important vehicle for strengthening 
government responsiveness to citizen priorities. 
6
Whether community dialogues can actually induce the government to be more responsive and in turn 
improve the coverage and quality of priority public goods will be the primary focus of this impact 
evaluation. 
Introduction
This baseline evaluation provides a description of USAID’s Participatory and Responsive Governance 
program being conducted in Niger. The impact evaluation is being implemented by Ariel BenYishay 
(College of William and Mary), Lisa Mueller (Macalester College), and Phillip Roessler (College of Wiliam 
and Mary). The National Opinion Research Center (NORC) was contracted by USAID to conduct this 
baseline data collection in partnership with AidData. NORC in turn contracted the baseline fieldwork to 
Kantar Public. Kantar was responsible for conducting the in-country surveys and managing the field team. 
The survey work began on January 24th and finished on February 12th
, 2017. The final datasets were 
delivered to the research team on March 28th, 2017. 
Impact Evaluation Overview
This document provides an overview of AidData’s monitoring and evaluation (M&E) component of 
USAID’s Participatory and Responsive Governance (PRG) program in Niger. The overarching goal of the 
PRG project is to strengthen the collective responsiveness of the Nigerien government and its citizens to 
priority public needs in order to help mitigate what Nigeriens call a ‘crise de confiance’ or ‘crisis of 
confidence’ between citizens and the state and ultimately bolster stability and governance in one of the 
world’s most fragile states. The project aims to strengthen collective responsiveness through three 
channels: 1) political party campaigns; 2) collective engagement and coordination of multiple stakeholders 
(government, non-government, donors) to undertake reforms that address citizen priorities; and 3) 
capacity-building of local think-tanks, media, NGOs, and civil society to promote participatory governance.
This impact evaluation will focus on evaluating USAID’s Participatory, Responsive, Governance - Principal 
Activity Program (PRG-PA). The PRG is designed to be a principal contributor to the achievement of 
Development Objective 2 under the Niger Operational Framework: Citizen Confidence in the state 
increased among target populations.
Post-electoral activities focused on targeted citizen priorities under Phase 1 will provide initial 
contributions to results under IR 2.2: Equitable access to public sector services increased in target areas
while continuing to contribute to the participatory processes under IR 2.2, including civic engagement in 
governance.
7
Our impact evaluation will focus on the post-electoral activities and IR 2.2. Our objectives for the impact 
evaluation follow those stipulated in the Monitoring and Evaluation Plan provided in the PRG Program 
Description (PD) (p. 28), which states in part that:
“Given the innovative nature of this proposed project, the project design team 
recognizes the value of also developing an external impact evaluation to test the 
fundamental hypotheses and theory of change embedded in the project design, and to 
determine whether changes in outcome measures are directly attributable to the 
project. This aligns with the USAID Evaluation Policy, which requires new and innovative 
or pilot projects to be subject to impact evaluation and thus the counterfactual analysis 
needed to determine causality. We also recognize the particular importance of being 
able to show attributable results in USAID governance programming, and see evaluation 
of this project as an opportunity to try to prove intervention efficacy. Lastly, an impact 
evaluation of aspects of the PRG principal activity in its Phase 1 would allow for findings 
to be applied to improved project design and implementation in the anticipated Phase 2 
scale-up. In this way, the impact evaluation will also play an important formative 
assessment role for the project.”
Treatment: PRG-PA Implementation
USAID selected Counterpart International (CPI) to implement the five-year PRG-PA. CPI’s program will 
seek to increase the capacity of stakeholders – including Civil Social Organizations (CSOs), traditional and 
religious leaders, government and political parties, the media, and private sectors. 
CPI’s plan includes a multi-stage program with three different sets of activities. The overall goal is to 
strengthen electoral accountability and the responsiveness of the Niger government to priority public 
needs. This evaluation will focus on the second of these activities, which involve conducting government 
systems mapping to inform collective action initiatives (activity 2.1); building a set of master dialogue 
facilitators (activity 2.2); brokering commitments around targeted citizen priorities through local partner 
dialogues (activity 2.3); supporting media production of reliable information that supports collective 
action (activity 2.4); initiate three national-level working groups with government, business, academic, 
and civil society representatives focused on health, education, and security (activity 2.6); and train 
government and non-government leaders on the Service Improvement Action Planning (SIAP) tool 
(activity 2.9).
2
CPI plans to accomplish activity 2.1 by identifying individuals who can influence government priorities and 
champion and create advocacy campaigns. These individuals will be pulled together in a systems map 
that will improve the understanding of the different networks of influence within the government. The 
maps will be created based on topic, type of reform, key individuals and institutions, policy decision points, 
2 A small number of other sub activities were not evaluable because their rollout would take place across much of 
the sampled communes, making treatment-control comparisons infeasible.
8
and changes in the system over time. This mapping exercise seeks to increase the overall understanding 
of how to and who to negotiate with in government systems, especially those who might seem like unlikely 
allies. They will be used as points of reference as CPI and partners develop their collection action plans. 
Under activity 2.1, CPI will also be creating maps of media organizations that specify in communications 
on security-related topics. Along with the mapping exercise, CPI will also run a perception survey in 
Agadez, Diffa, Niamey, and Zinder to understand citizen views of the security sector in Niger. The survey 
results will highlight priorities for the planned activities, specifically focusing on improving communication 
between communities and government actors on security issues. 
As part of Activity 2.2, CPI will train up to six sub-grantees from civil society groups and the private sector 
to run multi-stakeholder dialogs that will cover topics that emerge from local election debates and 
conversations. This will include topics such as service delivery problems and gaps in education, health, 
and security. Two individuals (one man and one woman) from each selected sub-grantee will receive 
training on “asset-based community mobilization; cross-cultural communication; socially inclusive, 
participatory facilitation methods and dialogues for social, political, and policy change.”3
 These “Master 
Dialogue Facilitators” will then, over time, be expected to train additional facilitators from their 
organization. 
The trained Master Dialogue Facilitators will then lead local partner dialogues as part of activity 2.3. These 
dialogues are intended to create a forum where different community members will be able to build 
partnerships, utilize resources, and identify common goals. The dialogues will include municipal leaders 
and regional councilors, community groups, private sector actors (such as business based in regions), 
fadas, Cadres de Concertation (CDC), women’s cooperatives, and other local groups. Activity 2.3 will also 
include 10 town hall meetings in specific regions. These town hall meetings will include municipal leaders, 
the media, CSOs and other community groups, religious and traditional leaders, and private sector 
representatives. These meetings will address health, education, and other issues related to security in 
order to identify issues that collective action initiatives can target. 
CPI also intends to train media partners to cover these events as part of activity 2.4. This will include 
holding workshops for media actors to train them on effective interviewing techniques and how to discuss 
important priority issues with stakeholders and citizens. These media actors, which will include radio 
stations, televisions, and print partners (local and national magazines), will then cover the dialogues to 
insure public transparency and encourage action around dialogue outcomes.
Activity 2.6 will work on creating three national–level working groups that will be made up of government, 
business, academic, and civil society members. These groups will focus on discussing health, education, 
and security issues. The members will meet to review current education and health service delivery issues 
and communication problems in the security sector. These groups will produce recommended policy and 
legislation as well as the actors that should be involved in implementing those measures. 
3 Counterpart International, “Participatory Responsive Governance – Principal Activity (PRG-PA) Annual Work 
Plan”, June 8th, 2016.
9
The last activity is activity 2.9, which will introduce monitoring tools for citizens and government officials. 
One of these tools is the Service Improvement Action Planning (SIAP) tool. CPI will train citizens and 
government officials to use the SIAP tool to record the priorities that come from the dialogues. They will 
then seek to use this toolto achieve the dialogue identified service improvements identified by monitoring 
progress on a regular schedule and reporting results and outcomes. Monitoring tools will also include 
citizen surveys results and other quantitative measurements. 
In addition to these activities, CPI added a security-specific modification in late September 2016. As of 
early October, the Nigerien government delayed elections by 2.5 years. Given this delay, CPI has been 
instructed to not run their election related activities. For more information on CPIs work, see CPI’s “PRG￾PA – Y1 Annual Workplan (Feb 2016 – Feb 2017)”.
Theory of Change 
One of the core objectives of the PRG-PA is to improve collective (government and citizen) responsiveness 
to priority public needs by increasing the capacity of Nigeriens to undertake and sustain collective action 
that holds their representatives accountable. Few institutions are purported to be as critical for responsive 
government as political accountability, in which citizens possess the capabilities and capacity to hold the 
government answerable for its policies.4 Extensive research suggests a strong, positive association 
between broad-based political accountability and the effective provision of public services;5 even limited 
accountability in authoritarian regimes is found to improve human development.6
There are two key limitations of existing literature on this subject, however. First, most rely on 
observational data (cross-national datasets that analyze associations between indicators of accountability 
and provision of public services) to draw inferences about the effect of political institutions on 
government responsiveness. The problem with such an approach is isolating precisely how much 
institutions of accountability matter for the provision of public services. It could be that this association 
arises due to reverse causality: the provision of public services (which themselves could be supplied for 
exogenous reasons—for example the threat of external war) leads to political accountability.7 Or it could 
be the case that the association between accountability and public services is spurious; both are a function 
of a third unobserved variable. Thus, accountability does not have as powerful an effect on responsive 
government as we would expect. 
4 Besley, Timothy. 2006. Principled Agents?: The Political Economy of Good government. Oxford: Oxford University 
Press.
5 Bueno de Mesquita, B., et al. 2003. The Logic of Political Survival. Cambridge, MA, MIT Press. Adsera, A., et al. 
(2003). "Are you Being Served? Political Accountability and Quality of Government." Journal of Law, Economics, 
and organization 19(2): 445-490.
6 Miller, M. K. 2015. "Electoral Authoritarianism and Human Development." Comparative Political Studies 48 (12).
7 Fukuyama, Francis. 2011. The Origins of Political Order: From Prehuman Times to the French Revolution. New York 
Farrar, Straus and Giroux.
10
A second limitation is existing cross-national studies tend to focus primarily on the role of structural or 
historical factors in accounting for the emergence of the accountability-responsiveness nexus—such as 
the structure of the economy (the degree to which it is dependent on natural resources), forms of 
colonialism, history of state centralization or societal fractionalization, a culture of meritocracy, or 
geography. While it is critical to understand the long-run processes underpinning institutional variation, 
these studies are less valuable for understanding what policy interventions in the short-term are effective 
at bringing about more accountable and responsive governments.
This impact evaluation aims to help fill this gap in the literature. It will rigorously test the effect of a 
participatory and governance program on improving political accountability and government 
responsiveness to citizen priorities. One of the central components of the program is a series of multi￾stakeholder dialogues that bring together community leaders, municipal and regional councilors, private 
sector actors, professionals and citizens to confer upon, design and initiate Regional Development Plans 
(PDRs) and Communal Development Plans (PCDs). The expectation is the multi-stakeholder dialogues will 
catalyze development and government responsiveness both in terms of process and outcomes: the 
dialogues are intended to serve as focal points, or coordination mechanisms, for a diverse set of actors to 
come together to bring multiple perspectives to bear on the challenge of community development as well 
as marshaling the resources and forging the strategic partnerships necessary to ensure the development 
plan is implemented.
Beyond its effects on coordination and resource mobilization, multi-stakeholder dialogues are theorized 
to have important governance benefits, which are seen as critical for sustainable development.8
In eliciting 
the participation and contributions of a diverse set of actors the dialogues enable an inclusive decision￾making process, which is seen as a key source of legitimacy.9 Moreover, the formalization of collective 
decision-making procedures and a plan of action facilitates common knowledge amongst participants and 
the constituencies they represent, which not only improves transparency but also accountability as the 
stakeholders are seen to commit to implement the plan. Though there are no sanctioning mechanisms 
built into the PRG, there is a strong media component to the program that will publicize the dialogues and 
the commitments coming out of them, which are intended to increase awareness and raise the costs for 
the stakeholders if they fail to follow through. 
Multi-stakeholder dialogues have strong potential to impact relations between Nigerien leaders and 
citizens because of the historical context in which they will take place. Dialogue is a major pillar of Nigerien 
democracy dating back to the transition to multiparty competition in 1992, which occurred at a National 
Conference involving stakeholders from diverse sectors of society (politics, labor, education, religion, 
etc.). Subsequently, the National Commission for Social Dialogue (CNDS) and the National Council for 
Political Dialogue (CNDP) were founded in 2000 and 2003 with the express purpose of facilitating debate 
8 Bäckstrand, Karin. "Multi‐stakeholder Partnerships for Sustainable Development: Rethinking Legitimacy, 
Accountability and Effectiveness." European Environment 16.5 (2006): 290-306. Hemmati, Minu. 2002. Multi￾stakeholder Processes for Governance and Sustainability: Beyond Deadlock and Conflict. New York: Routledge.
9 Hemmati, Minu. 2002. Multi-stakeholder Processes for Governance and Sustainability: Beyond Deadlock and 
Conflict. New York: Routledge.
11
and compromise between political parties and civil society organizations of different partisan leanings. 
However, these institutions have not fulfilled their mission: As of 2017, the space for open dialogue is 
shrinking amid government crackdowns on the press and civil society; political parties are increasingly 
polarized into pro-government and opposition camps. Given this historical background and contemporary 
situation, multi-stakeholder dialogues in the PRG-PA framework have two main advantages as 
experimental treatments: First, they are familiar and salient institutions in Nigerien society, which 
enhances external validity (i.e. the applicability of findings outside the experiment). Second, they address 
several flaws in the CNDS and CNDP. Namely, they are local instead of top-down and they include an 
innovative media component, reducing concerns that the treatments will merely replicate mistakes of the 
past. 
Taken together, the use of multi-stakeholder dialogues is expected to improve local government 
responsiveness through multiple channels—improving the capabilities of community elites to craft and 
implement a development plan that addresses citizen public priorities while ensuring this plan of action 
is seen as legitimate and that it actually gets implemented. 
Most existing social science research on the efficacy of multi-stakeholder dialogues tends to focus on 
international governance and rely primarily on qualitative evidence.10 As far we can tell, there have been 
no experimental studies on the effect of multi-stakeholder dialogues on local development. Thus this 
impact evaluation has the potential to make a broader contribution to development strategies in low￾income countries. 
A second innovation of the impact evaluation is it will also evaluate the efficacy of follow-on informational 
interventions to sustain public discussion and facilitate collective action. There is an emerging literature 
on the effectiveness of informational campaigns on government performance and the quality of services. 
While some of these interventions have been found to be quite effective and cost-efficient—for example, 
the provision of information on the quality of healthcare in one’s community relative to other 
communities and the national average led to significant improvements in health service provision in 
Uganda11—others have found that informational interventions do not always translate into increased civic 
and political engagement, improved accountability, and better services.12
10 See for example, Bäckstrand, Karin. "Democratizing Global Environmental Governance? Stakeholder Democracy 
after the World Summit on Sustainable Development." European Journal of International Relations 12.4 (2006): 
467-498.
11 Björrkman, Martina, and Jakob Svensson. 2009. Power to the People: Evidence from a Randomized Field 
Experiment on Community-Based Monitoring in Uganda. The Quarterly Journal of Economics 124(2): 735-69.
12 Olken, B. A. 2007. "Monitoring Corruption: Evidence from a Field Experiment in Indonesia." Journal of Political 
Economy 115 (2). Banerjee, Abhijit V., et al. "Pitfalls of Participatory Programs: Evidence from a randomized 
evaluation in education in India." American Economic Journal: Economic Policy (2010): 1-30. Humphreys, Macartan, 
and Jeremy Weinstein. "Policing politicians: citizen empowerment and political accountability in Uganda 
preliminary analysis." Columbia University. Unpublished manuscript (2012). Lieberman, Evan S., Daniel N. Posner, 
and Lily L. Tsai. "Does Information Lead to More Active Citizenship? Evidence from an Education Intervention in 
Rural Kenya." World Development 60 (2014): 69-83.
12
In this study, we will analyze the additive effect of messaging citizens to remind them of the commitment 
elites in their communities made in the multi-stakeholder dialogues and to provide a status report on the 
degree to which the stakeholders are making good on their promise to initiate and implement a 
development plan. To do so, the PIs may plan to use SMS to message a randomly selected subset of 
citizens the progress (or lack thereof) that the local government has made in addressing citizen public 
priorities and to emphasize the citizens’ responsibility to continue to monitor progress to ensure its future 
implementation. We propose that this messaging should not only be specialized—directed at individual 
citizens to increase their personal sense of responsibility13—but also provides concrete examples as to 
how their fellow citizens are acting on the information they have received to improve government 
performance. 
Summary of Monitoring and Evaluation Strategy 
Impact Evaluation Design and Implementation
As noted, the impact evaluation (IE) will focus on the second project outcome of the PRG: improved 
collective responsiveness through increased multi-stakeholder contributions to public goods provision. In 
particular, the IE will assess the degree to which increased multi-stakeholder coordination and collective 
action leads to change in perceived government legitimacy and the coverage and quality of public goods 
that meet citizen priority needs. We refer to the subset of PRG-PA activities covered by our evaluation as 
public goods reform (PG reform).
The other project outcomes of the PRG—responsiveness of political parties and local capacity-building to 
promote participatory governance—will not be directly evaluated in this impact evaluation, though the 
interactive effects of party responsiveness and capacity-building with PG reform implementation may be 
assessed.
The IE employs a randomized design that entailed randomly selecting the 24 communes out of 48 to 
receive the PG reform program. This randomized design will allow us to causally estimate the effect of 
the project intervention using panel surveys and changes in objective measures of public goods provision 
and access. 
The overall data collection plan for the evaluation is as follows:
1. Baseline survey at household- and cluster-level on socio-economic and socio-cultural indicators; 
political attitudes and engagement; strength of informal and formal institutions; multi 
stakeholder coordination and contributions; public goods provision and access; citizen 
preferences for public goods. This includes surveys with randomly selected households in 
13 Grossman, G., et al. (2015). Can SMS-Mobilization Increase Citizen Reporting of Public Service Deficiencies to Politicians?, 
Working Paper.
13
treatment and control communes as well as interviews with commune level officials and 
government officials. 
2. Midpoint checks for compliance (e.g., were the multi-stakeholder dialogues held in the
appropriate locations) and change across key indicators for admin and community-level data.
3. Endline survey and qualitative analysis at commune/government level, household, and cluster￾level indicators and administratively measured behavioral data.
Below, we discuss the randomization design, sampling design and power, and data collection efforts.
We include the complete Household Survey Questionnaire and Commune-level Questionnaire 
administered at baseline in the annex folder.
Randomization Design
The goal of the randomized design of the evaluation is to ensure that the treatment and comparison 
groups of communes are otherwise similar in the characteristics targeted by the PRG program. In non￾randomized settings, differences between treated areas and comparison groups may bias estimates of 
the program’s effects. The randomization therefore aims to form groupings of communes whose average 
characteristics are as similar as possible prior to program implementation.
AidData completed the randomization of communes in September 2016. The sample frame for the 
randomization included only communes that were selected by CPI based on criteria that reflected the 
security conditions and existing organizational connections by CPI’s sub-awardees. CPI identified 48 
communes that fit these criteria. Importantly, CPI has project funding and capacity to administer the PG 
reform activities in 24 of these 48 eligible communes. Given this constraint, in order to distribute the 
activities’ benefits fairly among these 48 communes and to better evaluate their impacts, AidData 
randomly selected the 24 “treatment” communes.
Out of the communes provided by CPI, AidData stratified the randomization based on three factors: 
region, urban/rural status, and whether or not they were targeted for CPI’s initial PRG-PA Activity 1 (to 
make elections more responsive to priority public needs). This created 10 sets of communes, each of 
which shared identical values for these factors (i.e. belonged to the same region, were similarly 
urban/rural, and slated for Activity 1/not). The numbers of communes within each set varied: some sets 
contained as few as three communes, while others contained as many as nine. The aim was to select 
approximately half of the communes within each set to be in the treatment group (in cases where the 
number of communes within each set was odd, the number of communes to be selected for treatment 
was randomly rounded up/down).14
 This procedure ensured that equal number of Activity I communes 
14 It is important to note that there need not be exactly equal numbers of communes in the treatment group and 
control groups within each set. What is important is that the assignment of a commune to the group is random.
14
were randomly selected into the PG reform treatment and control groups, and that approximately half of 
each region’s urban communes and approximately half of each region’s rural communes. 
The final randomized assignment was conducted using the STATA statistical software on September 16, 
2016. Figure 2 below maps the treatment and control commune assignments. 
Figure 1: Treatment and Control Assignments
A key aspect of the randomized design is that only the communes assigned to the treatment group will be 
the targets for the PG reform program for the duration of the evaluation, currently expected to last 
approximately two years. CPI’s ongoing partnership in the evaluation has been and will continue to be 
crucial for the evaluation’s success. 
Sampling Design and Power
Household survey sample design
The household survey target sample consisted of 1,200 households selected using stratified random 
sampling. Within each of the 48 communes in our randomization frame, we randomly sampled 3
enumeration areas (Villages), within which either 8 or 9 households were to be sampled.
This sample size was targeted to allow us to detect treatment effects of 0.1 standard deviations or greater
in our primary outcomes. These are quite precise minimum detectable effects. We will use a number of 
Treatment
Control
Niger PRG Impact Evaluation
Randomly Assigned Treatment and
Control Communes
15
techniques to maximize precision available from our sample. First, we will use randomization inference 
approaches to determine exact p-values under the sharp null hypothesis of no treatment effects for any 
unit (as specified Gerber and Green 2015). Second, we will use repeated measures across rounds. Third, 
we will aggregate multiple measures of common concepts into standardized indices, thereby guarding 
against multiple comparisons while maximizing the precision with which each concept is measured (Kling, 
Liebman and Katz 2007).
We used simulations to confirm that our sample design will allow us to detect treatment effects of 0.1 
standard deviations with 95.4% confidence (and 0.2 deviations with >99% confidence). We obtained 
survey responses from the Afrobarometer Round 6 carried out in Niger in 2015 and estimated the intra￾cluster correlation and means of the seven outcomes most closely related to our evaluation (limiting the 
sample to those regions covered by the project). We then simulated 10,000 samples that match these 
correlations and means. For each of these samples, we nextsimulated 10,000 random assignments under 
the sharp null hypothesis of no treatment effects and measure how many of these yield treatment effects 
of each size. The results of this simulation (shown in the figure below) indicate that only 4.6% of placebo 
assignments generate treatment effects as large as 0.1 standard deviations. We thus conclude that our 
design is well powered to detect even moderately sized treatment effects.
The household survey collected information on basic household indicators such as occupation, education 
level, poverty, religion, and connectivity (such as access to phones, internet, radio, and other news 
sources). Survey questions also measured household’s political views by including questions about 
political views and parties, the security situation in their community, elections, government officials, and 
trust and relationships to local and national leaders. In order to measure local views on priority needs, 
the survey measured local individual’s thoughts on how effective local and national governance has been, 
how accessible civil services are, important local and national issues, and how local and national 
government priorities match local household’s needs [see index for full survey].
Figure 2: Simulation Results
0 2 4 6 8
Percent
p=.322 p=.046 p=.002 p=0
-.25 -.2 -.15 -.1 -.05 0 .05 .1 .15 .2 .25
Treatment effect (SD)
16
Commune Level Official Sample Design
CPI outlined in their work plan the goal of building accountability and government capacity. Dialogues are 
designed to advance that goal by inviting commune-level officials to share their perspectives on health, 
education, and security in the context of the postponed 2017 local elections. To measure the impact of 
these interventions, enumerators interviewed commune-level officials at baseline and will conduct follow
up interviews at endline on their knowledge in the following areas: leadership and strategic management, 
information about the citizens and programs of the commune, political opinions, level of communication 
with citizens, and external relations and communications. 
The leader survey target was 144 surveys across the 48 communes (3 interviews per commune) but only 
118 surveys were completed due to availability and logistical challenges. The field team sought to 
interview a commune mayor, an administrator in the health or education field, and a traditional or 
religious leader. The interview targeted to towards the commune mayor could also be directed towards 
the highest-level official in the commune that is available to complete the survey. The mayor could bring 
in officials that work for him/her to complete more of the specific statistical questions. These additional 
people were listed on the survey and it should be clear whether or not these additional people were in 
the survey for the entire time or if they were just part of the interview for the necessary fact based 
information. 
The survey targeted at the commune health or education administrator included individuals outside the 
commune mayor office at the school or health center. This was not necessarily a doctor or a teacher but 
the individual who runs the health center or schools, etc. The final survey reserved for a traditional or 
religious leader survey was targeted at village elders, village leaders, Imams, or other religious leaders. 
Data Collection
Household and Leader Survey Fieldwork Details
The survey work was contracted out to NORC, who in turn contracted Kantar Public to conduct the 
fieldwork. Enumerator training happened in Niamey from 16th to 20th of January 2017 with a team of 45 
enumerators, 12 supervisors and 15 Quality Control staff. Enumerators were recruited and hired by 
Kantar partners in Niger and most recruited enumerators had an average experience of 2 years in survey 
administration. The local team hired enumerators from the various project regions and therefore they 
had experience with the area, local politics, and were very fluent in the local dialects needed for the 
survey. Each survey team was comprised of four enumerators and one team supervisor. 
17
The target for each commune was 25 household interviews in each of the 48 communes across 6 regions 
in Niger. The targeted sample was 1,200 interviews, and Kantar conducted additional interviews in order 
to account for potential replacement of any poor quality surveys. In total, Kantar achieved 1,314 
interviews. Back checks included re-visiting the respondent to confirm the enumerator visited and re-ask 
a series of questions to confirm the interview took place correctly. The survey firm also conducted spot 
checks on surveys, telephone check, audio checks, GPS checks, and forensic checks. After completing the 
back check process, the survey firm discarded 56 surveys delivering a final dataset of 1,258 surveys. 
The leader survey target was 144 surveys across the 48 communes (3 interviews per commune). However, 
due to a variety of reasons including the non-availability of commune mayors and inconsistent work by 
the enumerators, only 118 surveys were completed. The baseline leader survey sought to survey three 
leaders from each commune. The enumerators were instructed to survey the commune mayor, a health 
or education administrator, and a religious leader. The survey selection protocol and the leader survey 
are located in the annex. Given the aforementioned security concerns, some communes were not 
surveyed leaving the total number of surveys at 118. 110 men and 8 women were interviewed between 
the ages of 25 and 95 with the mean age of 49. The survey data proved difficult to collect due to the lack 
of knowledge of commune leaders on commune level statistics such as health and education indicators. 
The survey firm divided their enumerators between the regions of Zinder, Agadez, Niamey, and Diffa. The 
teams were assigned three randomly selected villages from each commune in those regions. The assigned 
villages were pre-selected by AidData via a random lottery from village lists obtained from the National 
Statistics Office. Three replacement villages were also pre-selected in case there were any problems 
surveying the originally chosen village. These three additional villages were comparable in household size 
to the original village to ensure overall survey balance. 
Selection and Replacement Protocols for Households
To select households, the survey firm used a random route walk approach. Kantar, NORC’s in-country 
partner, has used this method successfully in several projects. In the training, the enumerators were 
taught how to canvass the community, identify an entry point, and calculate intervals for selecting 
households. For every selected household, only one eligible respondent was interviewed per household. 
Each enumerator was assigned an area of the village to select respondents from via random walk. Once 
the household was selected, the enumerator asked a household member (often the head of household if 
s/he was present) for information on each member of the household. The survey software would then 
randomly select one respondent from that household list.
Respondents who had travelled out of the village were replaced with an individual with same gender and 
within the same age group in the same household. Otherwise, the household was replaced entirely. 
Enumerators would move on from that household and follow selection criteria to find their next 
household. If respondents were still in the area but were not currently at home the enumerator would 
18
revisit the household at a later time. Interviews usually took approximately 45 min to 1 hour. Overall the 
household survey went smoothly and the data collected was of high quality. 
Quality Control and Data Processing 
The quality control checks included in-field checks and data validation checks. Physical in-field checks 
included a trained quality control team who followed the enumerators and were present during the data 
collection. Their work included performing back-checks, which involved re-visiting the respondent after 
the interview was completed to confirm whether the interview took place, screening the respondent to 
ascertain if he/she is qualified and re-asking some of the questions to see if the responses match what 
was captured during the original interview. 
Quality control personnel also performed spot checks, which involved a follow up on interviews while they 
were being conducted to check if they had been conducted in accordance with the survey guidelines. 
Finally, the quality control team also accompanied an enumerator throughout the entire process (finding 
a respondent, selecting the correct interviewee, and the interview process). They were responsible for 
ensuring the right methodologies were applied and the right questions were asked.
The survey firm also conducted telephone checks, which included follow-up calls made from Kantar 
Public’s Ghanaian office on completed interviews to ascertain whether the respondents had been 
interviewed and whether they were qualified or not. The last quality control check included audio checks. 
These were silent recordings that were programmed on questions of interest. When these questions were 
asked, it triggered an audio recording of the question. These recordings were validated in the office to 
ascertain whether the questions were asked correctly and whether or not responses matched what is in 
the data.
Kantar’s home office data team checked the collected data once it was uploaded to the home database. 
This is a detailed check performed on extracted data to look for the following: 
• Duplicate interviews or entries;
• Omissions; 
• Peculiar interview durations (either far below or above the average duration);
• Dates and time of interviews;
• Straight lining and other inconsistencies.
The data team also performed GPS checks. The team extracted coordinates and plotted them to 
determine whether or not the interviews were conducted in the right locations and whether or not they 
were correctly spaced.
The survey firm reported that all the questions seemed to have been well understood both by the 
interviewers and respondents, and the questionnaires were properly completed. The logic between each 
question was checked more than twice before and after the fieldwork. 
19
Hypotheses and Measures
Household Survey Findings 
Summary Statistics: Household Survey
A total of 677 men and 581 women were surveyed with the household questionnaire. The mean age of 
respondents was 36, the youngest respondent was 18 and the oldest respondent was 90. The sampled 
respondents have an overall low level of education, with the majority of respondents having no education 
(40%), followed by 23% having some sort of informal education, 12% completing some secondary, and 
just a small minority completing more education. Most of the sample is living with their non-polygamous 
spouse (68%) with 17% being single and the rest of the sample split between not living with their non￾polygamous spouse, being in a polygamous marriage, divorced, separated, widowed, and refused to tell. 
Forty-four percent of the respondents were the head of household, with 33% being the spouse, 14% a 
child, and remaining being additional family members or friends. The majority of respondents never had 
a job (22%), are home makers (15%), are in either farming, fishing, or forestry (25%), or are 
vendors/traders (9%). 
The most common preferred language for survey respondents was Hausa (37%), followed by Tamasheq
(19%), Fulfulde (15%), and Kanuri (15%). 89% of respondents are Muslim, with only a handful of 
respondents identifying specifically as Sunni (2%), Tijaniya Brotherhood (2%), or Shia (3%). Most 
respondents were Hausa (34%), Tuareg (21%), Peulh (16%), or Kanuri Manga (16%). Summary statistics 
for both treatment and control groups can be seen in Table 1. 
20
Table 1: Summary Statistics
Variable Control Treatment P-value
Age 36.41 36.37 0.97
Gender Male 0.53 0.55 0.46
Female 0.47 0.45 0.46
Education Level None 0.36 0.43 0.07
Informal 0.26 0.20 0.18
Some Primary 0.09 0.08 0.78
Completed Primary 0.06 0.06 0.88
Some Secondary 0.13 0.11 0.18
Completed Secondary 0.05 0.05 0.92
Post Secondary 0.01 0.02 0.38
Some University 0.01 0.02 0.53
Completed University 0.01 0.01 0.97
Post Graduate 0.01 0.01 0.99
Don’t Know 0.00 0.01 0.14
Marital Status Single 0.17 0.17 0.82
Married, Living With 0.68 0.68 0.97
Married, Not Living With 0.03 0.05 0.15
Polygamous Marriage, Living With 0.03 0.02 0.45
Polygamous Marriage, Not Living 
With
0.01 0.00 0.06
Divorced 0.02 0.03 0.30
Separated 0.01 0.00 0.25
Widowed 0.04 0.05 0.64
Refused 0.00 0.00 0.10
Religion None 0.00 0.00 0.17
Christian 0.00 0.00 0.32
Muslim 0.90 0.89 0.96
Sunni 0.02 0.02 0.96
Ismaeli 0.00 0.00 0.35
Tijaniya Brotherhood 0.02 0.02 0.69
Oadiriya Brotherhood 0.01 0.00 0.22
Shia 0.02 0.03 0.42
Izala 0.00 0.02 0.00
Traditional Ethnic Religion 0.00 0.00 0.99
Hindu 0.00 0.00 0.24
Bahai 0.00 0.00 0.33
Agnostic 0.00 0.00 0.16
Athesist 0.00 0.00 0.24
Refused 0.00 0.00 0.99
21
Balance: Household Survey
A goal of the baseline survey is to ensure that the control and treatment groups have no statistically 
significant differences between them (tested at 5% sig). The household survey was fairly balanced 
between treatment and control characteristics. Because most of our questions of interest asked for 
responses in categorical levels (i.e., “a lot”, “somewhat”, etc.), we conduct balance tests on each of the 
categories for each of the questions. It is important to note that such tests incur more frequent 
differences across treatment and control groups than for continuously measured variables. We include 
the complete balance tables in the annex and discuss the main findings below.
Out of 43 variables examined, 7 had 2-3 answer choices that were unbalanced between the treatment 
and control group. Out of 226 answer choices, only 31 were unbalanced. This rate is somewhat higher 
than would be expected by random chance. Many but not all of these answer choices indicate worse 
initial perceptions and conditions in the treatment group than in the control group (though some variables 
do indicate better perceptions/conditions in the treatment group). The imbalances suggest that the 
treatment group is slightly more ethnically diverse than the control group and more respondents in the 
treatment group feel less free to join political groups or vote for whoever they want without feeling 
pressured. In the same vein, a statistically significant number of respondents in the treatment group also 
didn’t vote compared to the control. The research team plans to address these residual imbalances by 
including these imbalanced baseline conditions as covariates in the statistical analysis when evaluating 
the impact of the PG reform on perceived state legitimacy and other outcomes.
Baseline conditions: Household Survey
Security and the military
Overall, the survey reflects high levels of trust in security institutions in Niger. Consistent with prior 
estimates from the most recent round of the Afrobarometer survey (Round 6 in 2014/2015), respondents 
indicate that they are more willing to trust and respect the role of the military and police than politicians 
and political parties. 
A majority of the sample either agree (57.5%) or strongly agree (15%) that the police always have the right 
to make people obey the law. Fewer people disagreed (9%) or strongly disagreed (5%), while 5% neither 
agreed nor disagreed and 7% responded that they don’t know (Q41). 
When asked, a majority of sampled individuals responded that they trust the police (70%) and the army 
(78%) a lot. The trust in the army and police was higher than respondent’s trust in politicians and 
government officials, specifically the president, the national assembly, the independent National Electoral 
22
Commission, the national tax authority, the local government council, and the ruling party. It was also 
significantly higher than trust in opposition political parties and the ruling party (Q46).
In that same vein, 56% of respondents think the local government has been fairly or very responsive to 
citizens’ needs and security issues, while approximately 26% believe they have been very or fairly 
unresponsive. Overall, respondents indicated that they thought the local government was more 
responsive to demands for security and less responsive to demands for public services and economic 
development (Q49).
Media and Connectivity 
A majority of respondents get their news from the radio (60%) and far less get their news from television, 
newspapers, the internet, or social media with newspapers being the most never used media source. At 
the same time a majority of respondents don’t necessarily believe what the media prints or say things it 
knows are untrue. The survey population was split between thinking the media often (28%) abuses its 
freedoms by printing or saying things it knows are not true and 26% believing that they rarely do. 12% 
say they never do with a small minority (5%) saying they always do. The remaining respondents (28%) 
didn’t know (Q43).
23
Tax Authorities and Courts 
The survey reflects general respect in the authority and legality of the courts and tax authorities. For taxes 
in general, a majority of respondents agree that people should pay their taxes because they are important 
for the country to develop. 
53% agree and 11% strongly agree that tax authorities have the right to make people pay taxes while only
while 9% don’t know and 9% neither agree nor disagree. (41) This view coincides with 20% of respondents 
agreeing and 44% of respondents agreeing very strongly that citizens must pay their taxes to the 
government in order for our country to develop. On the flip side, 10% agree and 17% agree very strongly 
that the government can find enough resources for development from other sources without having to 
tax the people (Q41)
62% of respondents agree or strongly agree that courts have the right to make decisions that people 
always have to abide by. A minority (15%) disagrees with this statement and an even smaller number of 
respondents (6%) strongly disagree. A further 11.5% responded that they don’t know and 7% neither 
agreed nor disagreed (Q41).
Political Parties
Respondents indicate that political parties (both opposition and parties in power) are less trustworthy 
than the military or police and that competition between them can lead to conflict. This conflict could be 
the result of parties being silenced by the government as respondents indicate in the survey.
35% of respondents believe that competition between political parties often leads to violent conflict while 
26% believe it rarely does. Another 13% believes it never causes conflict, 9 % believes it always does and 
18% don’t know (Q43).
32.5% of respondents believe that opposition parties (or their supporters) are often or are alwayssilenced 
by the government while 32.5% don’t know. 25.5% of interviewees think the government either rarely or 
never silences opposition (Q44).
24
Views on the National Government and the President
Overall, respondents were less favorable when asked about politicians and the national government. 
Responses indicate that the sample believes the president often ignores the courts, doesn’t spend time 
listening to local people, and that political leaders are more interested in serving their own interests than 
the interests of the people. It was notable though that respondents often responded with “don’t know” 
when asked questions about the president. 
When asked if they believed the president ignores the courts and laws of Niger, a large margin of 
respondents (30%) said they “don’t know”. 26% said he often does followed by 19% of people saying he 
never does and 18.5% responding that he rarely does. Only a handful of people said he always does 
(5.5%). 
The majority of respondents agree that members of the national assembly (52%) never spend time trying 
their best to listen to what local people have to say. Approximately 20% of respondents believe that 
members of the national assembly often listen and 13% say that this group only sometimes listens to local 
people. The remainder of respondents either thinks they always do (4%) or they don’t know (10%) (Q47).
25
When asked whether or not the president ignores the national assembly and just does what he wants
32% answered “don’t know” while 24% says he never does. 18% said he rarely does and 21% responded 
that he often does. Only 5% said he always does (Q43).
Approximately 40% of respondents agree and 26% strongly agree that leaders of political parties in Niger 
are more concerned with serving their own interests than the interests of the people. The remaining 
respondents either neither agree nor disagree (9%), agree (9.5%) or strongly agree (10%) that they are 
more interested in serving the people that themselves (Q45).
The majority of respondents have the highest level in religious leaders, traditional leaders, the army, and 
the police while respondents exhibit slightly lower levels of trust in the president, the national assembly, 
the Independent National Electoral Commission, the national tax authority, the local government council, 
and the ruling party. The lowest level of trust was seen in opposition political parties and the ruling party
(Q46).
Views on Local Government
Overall respondents are generally displeased with their local government’s ability to maintain roads, 
market places, and create economic development. This goes along with respondents saying that they felt 
their local government did not spend time listening to their constituents. Despite this, they do seem to 
feel somewhat positive about their local governments ability to respond to security concerns 
appropriately. 
Respondents were fairly evenly split between thinking the local government was very or fairly responsive 
(40%) and very or fairly unresponsive (39%) to addressing citizens demands. This is similar to how 
responsive respondents think local government has been to needs and demands of local citizens for public 
services such as healthcare and education. More respondents (around 56%) think the local government 
has been fairly or very responsive to citizens needs and security verses approximately 26% who believe 
they have been very or fairly unresponsive. Overall respondents indicated that they thought the local 
government was more responsive to demands for security and less responsive to demands for public 
services and economic development (Q49).
Overall more respondents felt their local government maintained local roads fairly badly (34.5%) or very 
badly (32%). Approximately 20% felt they were maintained fairly well but very few (4%) thought they 
were maintained very well. Respondents answered more favorably (31.5% answered with fairly well, 4.5% 
answered with very well) with how local governments were maintaining local market places but the 
consensus still leaned towards fairly (30%) or very badly (24%) (Q53).
The majority of respondents agree that the local government councilors (45%) never spend time trying 
their best to listen to what local people have to say. Approximately 20-24% of respondents believe that 
these groups often listen and 13-15% say that those two groups only sometimes listen to local people. 
The remainder of respondents either think they always do (4%) or don’t know (10%) (Q47).
26
Democracy, Freedoms, and Voting 
Overall respondents indicate that they feel mostly free to vote how they want or say what they think. 
They also indicate that they feel able to join political any political organization they identify with. In terms 
of democracy, a majority of the sample believes that while they consider Niger a full democracy or a 
democracy with minor problems, they are torn between being satisfied or not happy with how the 
democracy is currently working. 
Most respondents answered that they feel somewhat free (42%) or completely free (24%) to say what 
they think. The same applies to joining any political organization and choosing whom to vote for without 
feeling pressured. 43% of respondents feel somewhat free to join any political organization while 39% 
feel completely free. 47% of respondents feel completely free to choose whom to vote for without feeling 
pressured while 39% feel somewhat free. Overall respondents indicate that they feel the most free to 
choose who to vote for (Q31).
The survey respondents were evenly split between thinking Niger is a full democracy (28%) and thinking 
it is a democracy with minor problems (27.5%). 24% of respondents think Niger is a democracy with major 
problems while only 6% think it is not a democracy. A final 11% don’t know and 4% do not know what a 
democracy is (Q39).
27
Respondents are fairly evenly split between being very or fairly satisfied (41%) and not at all or not very 
satisfied (42.5%) with the way democracy works in Niger. 2% thought Niger is not a democracy while 14% 
responded with “don’t know” (Q40).
Elections
A very high number of respondents said they voted in the last election and a majority of the sample 
thought the election was fairly free and fair. Despite this, respondents indicate that they don’t think 
elections actually help put representatives in power that will accurately reflect their views. They also 
don’t think elections can help remove representatives that are not doing what local people want. 
75% of interviewees responded that they had voted in the last election with only a minority responding
that they didn’t vote. 5% were not registered while 4% didn’t vote and another 4% didn’t have the time 
to vote (Q32).
28
Overall respondents thought the 2016 election was completely free and fair (44%) while approximately 
18% thought there were minor problems and 15% thought there were major problems. 13% responded 
with “don’t know” and 8% thought the elections were not free at all (Q33).
Approximately 30% of respondents do not think that elections ensure that the representatives to the 
national assembly reflect their views very well while approximately 25% think their representatives reflect 
their views well. Around 18% of respondents felt they didn’t reflect their views at all well while very few 
(approximately 7%) think the representatives represent them very well. 21% of the treatment and control 
group respondents replied with “don’t know” (Q44)
The same follows for respondents’ opinions on how well elections enable voters to remove from office 
leaders who do not do what the people want. Approximately 30% of respondents feel this does not enable 
voters to remove leaders from office who do not do what the people want very well and 9% think it does 
not do it very well at all. Around 25% thinks elections do this well with 15% saying it does it very well. 
22% responded that they don’t know (Q44).
29
Community and Individual Political Actions
The majority of respondents have taken very few political actions when they have been dissatisfied with 
their government’s performance (Q37). Respondents did indicate that they would perform certain actions 
given the chance including meeting or contacting candidates, attending a campaign rally, request action 
from local politicians, or contacting the media. At the same time, actions that the respondents would not 
take included refusing to pay taxes or participating in a demonstration. 
Approximately 75% of respondents did not try to persuade others to vote in a certain way or work for a 
candidate or party while just over 55% of respondents did not attend a meeting with a candidate or 
campaign staff or attend a campaign rally (Q34).
Most respondents said they had never contacted a local leader with more respondents contacting 
religious leaders and traditional leaders than political party officials, government agency officials, 
members of the national assembly, and local government councilors (Q35).
If respondents had visited a local leader the majority of them went with a group (34%) while only 13% 
went alone. 36.5% went for a community problem as opposed to 10% going for a personal problem (Q36).
33% joined others in their community to request action from their government while 63% did not. For 
those that didn’t, approximately 40% would if they had the chance while 26% would never. For those that 
requested action, 15% did it several times, 16% did it often, and 3% contacted them only once or twice 
(Q37).
30
A higher majority of respondents did not contact the media (like a radio program or writing a letter to a 
newspaper) or contact a government official to ask for help or make a complaint. 45-50% of respondents 
would contact the media or a government official given the chance while approximately 40% would never 
take those actions (Q37).
Close to 90% of respondents have not participated in a demonstration or refused to pay taxes with only 
around 12% of respondents saying they would if they had the chance. Close to 87% of respondents say 
they would never act in that way (Q37).
Commune Leader Survey Findings 
Summary Statistics: Leader Survey
118 individuals were interviewed during the leader survey. The youngest respondent was 25 and the 
oldest was 95 with the mean age around 50. The survey respondents were mainly men with only 8 out of 
118 being women. The majority of respondents were Haoussa (43%) with the rest mainly split between 
Kanouri (18%), Touareg (17%), and Foulani (13%). The remaining respondents were Zarma (7%), Toubou 
(1%), and Arabe (1%). The primary language for each respondent mirrors respondent’s ethnicity with 
48% of respondent’s preferred language is Hausa, 19% prefer Kanouri, 13% prefer Tamasheq, and 10% 
speaking Fulfuide. The remaining respondents speak either Zarma (7%) or Tubu (2%). 
The sample is split between respondents being Imam or religious leader (15), the mayor (14), the first 
deputy mayor (10), the SG (11), and a decentralized service agent for the state (12). There were also 
health agents (4), village oracles (6), School director/teacher (9), CSI chief (6), pedagogic advisor (1), 
Inspector (1), Quarter Head (7), Village chief (7), technical service agent (6), municipal S (3), board member 
(3), second deputy mayor (1), and a DDES (1). 
The highest level of education reached by these respondents was post-graduate (2 individuals). The 
majority of respondents finished high school (39%) or finished college (30%). 22% did not complete
elementary school, 7% completed elementary school but nothing further, and 2% completed vocational 
or technical school. 
Baseline Conditions: Leader Survey
Connectivity
According to respondents residents communicate with government officials mainly through phone calls 
and meetings including individual meetings, small group meetings, community meetings. Very few 
residents communicate via community radio, delegation, a petition, a letter, email, and text message 
(GV4). Respondents indicate that 26% of residents communicate with government officials somewhat 
31
infrequently, somewhat infrequently (22%), or infrequently (24.5%). Only 11% responded that they 
communicate frequently (GV4A). 
The sample responses show that when government officials want to communicate with residents they 
use radio, meetings (community, small group, and individual), and phone calls. They rarely or never use 
text messages, email, letters, petition, TV programs or commercials, visits, via the press, through the town 
hall, and sending of courtiers/posters (GV5). The majority of respondents indicate that government 
officials communicate somewhat frequently (36%). 29% say they communicate infrequently while 14% 
say it is somewhat infrequently. Only 10% say they communicate frequently and 11% responded with 
“don’t know” (GV5A). 
When they do communicate, respondents indicate that residents talk about problems with a health 
center, issues with a school, report diseases or other medical problems, reporting violence, conflict, or 
military, complaints about community services (including water, electricity, trash services, etc.), and 
reporting crimes. They more infrequently reach out about complaints about neighbors, complaints about 
voting issues, reporting corruption, questions about government services, questions about political 
parties, property issues, and issues with farm animal breeders (GV6). Government representatives 
communicate the same information on issues as residents but also communicate with more frequency 
information about farm animal breeders, property issues, reporting on community projects, and 
complaints about voting issues than residents (GV7).
61% of respondents had participated in a multi-stakeholder dialog during the past year while 35% 
reported that they had not participated in one (GV8). Respondents reported that dialogs were organized 
32
by the United Nations (10), the town hall council (7), Hydraulics Landlords (4), government authorities 
(10), technical partners (14), the mayor (9), the CARE3N Initiative (8), and NGOs (9) (GV9).
Responsiveness and Accountability of Government Officials
63.5% of leader respondents answered that the President of Niger is either very good or somewhat good 
at carrying out his duties and almost 30% thought he was either somewhat bad or very bad at completing 
his duties. 
The Commune Mayor had overall positive reviews but less positive than the president. Approximately 
45% said he is either very good (19.5%) or somewhat good (25.5%) while around 33% said he is either 
somewhat bad (23%) or very bad (10%). Respondents rated both the president of the commune council 
and the president of the regional council president lower than the Commune Mayor or President of Niger 
indicating that they think they are less good at carrying out their duties (GV1). 
Respondents we asked if they had heard of any corruption involving the current mayor, the president of 
Niger, the president of the commune, and the regional council president. Overall the sample responded 
that they had mainly not heard of corruption with any of the four government officials. More respondents 
answered that they had heard of some corruption with the current mayor and less with the President of 
Niger. “Have not heard” was a popular option for corruption involving the current mayor (19.5%), the 
president of the commune (30%), and the regional council president (16%) (GV2). 
33
Commune leaders lean towards very committed (30.5%) or somewhat committed (24.5%) when asked 
how committed other elites in the commune are in addressing citizen priorities. 19% answered not very 
committed while 14% answered not committed and 12% don’t know (PD9). 
Respondents are also very split when asked how reliable other elites in the commune are in following 
through on promises to addressing citizen priorities but they lean towards answering that they are very 
(27%) or somewhat (29%) committed. 10% answered not very committed while 20% answered not 
committed and 13.5% don’t know (PD10). 
Interacting with Residents
The leaders surveyed are fairly split on how often officials interface with residents. Respondents lean 
more towards “frequently” for how often the mayor’s office connects with commune residents with 22% 
saying they do very frequently and approximately 30% saying they do somewhat frequently. 29% say 
somewhat infrequently and 9% say very infrequently and 8.5% responded with “don’t know”.
Officials of the customary authority were listed as interfacing even more frequently with residents than 
any other official. 30.5% said they interact very frequently and 38% say somewhat frequently. 19.5% said 
somewhat infrequently and 4% respond that they interface very infrequently. On the other hand 
commune council officials interfaced with constituents less frequently than the mayors office and the 
customary authority and the police were listed as the least likely to interface with residents (GV3).
34
Necessary Local Actors for Development Projects: 
In thinking about the development of their commune, the top person that the respondent believes they 
should engage or coordinate with the most to address development issues in your community are chiefs 
(19%) and the deputy mayor/Governor Commissioner (23%). Teachers and presidents of the youth group, 
Secretary general, tax collector, retailer trader, and the SDO were also mentioned as necessary individuals 
to coordinate with to address commune development issues. 76% of respondents said they interacted 
with these individuals a lot with the majority saying they coordinate with them every one to 7 days (PD11). 
The type of people that respondents believe are necessary for the implementation and adoption of 
development projects is different than the individuals that respondents think are necessary to coordinate 
with the address development issues. Housewives, chiefs, the councilor secretary general, the FCC, and 
teachers were the top people that respondents believed were the most important actors who are 
necessary to engage with or coordinate with to ensure development projects are adopted and 
implemented. 75% of respondents say they interact with the top individuals a lot and 43% say they 
interacted with them in the last 2 days (PD12). 
Community Meeting Dynamics
The survey asked a variety of questions to understand meeting dynamics in communes from a community 
leader’s point of view. When asked about how decisions were made in the last commune assembly, a 
majority of leaders (64%) thought that every person decided when a decision had to be made on problems 
or issues in the commune while 18% thought the village leader, other council members, and some other 
non-council members decided. 11% thought the commune mayor decided and the remaining 7% thought 
just the village leader and other council members decided (PD2). 
35
Most respondents (65%) think that they could organize a meeting of important actors to discuss 
development issues in the commune while 9% said maybe, 6% said they did not think so, 6% refused to 
answer, and 14% answered that they didn’t know (PD13). Out of the respondents who thought they either 
could or maybe could put together a meeting, they thought a mean of approximately 46 people would 
show up with a minimum of 2 and a maximum of 1000 attendees (PD14). 
The survey asked a number of questions about the community assembly sessions to help understand the 
dynamics during the meetings. Overall over 50% of respondents thought there was no one group among 
poor, middle, and rich residents dominate decisions about problems or issues in the commune. 28% 
responded that they “don’t know” and 18% thought there was one group that dominated (PD3). Out of 
that 18%, more thought rich residents dominate the discussion while remaining respondents are split 
between poor and middle class residents. While respondents were split between which group dominates, 
they agree that the poor have the least amount of influence in commune discussions (PD5). 
36
When asked how the dynamics work between ethnic groups, the respondents are split evenly. 48% 
believe there is an ethnic group that dominates while an equal number thinks there isn’t a dominant group
(PD6). Of those that believe there is a dominant group, a majority believe that Haoussa (70%) dominates 
the most in commune assemblies during decision-making process while Kanouri (20%) and Foulani (10%)
were also mentioned (PD7). When asked which group has the least influence in commune discussions, 
respondents responded with the Foulani (30%), the Haoussa (20%), and “Others” (20%).
Balance: Leader Survey
Overall, the Leader Survey hypothesis variables were fairly balanced between treatment and control 
groups. Out of 420 possible answer choices across the 46 questions, 11 were statistically different. The 
remaining response categories were balanced across both treatment and control groups. Overall, the 
groups are well balanced. Balance tables are included in the annex. Field challenges
Administration, Access to communities, Logistics, and Military incidence 
The issues encountered during the survey process mainly centered on security concerns given the unrest 
in regions like Diffa. A number of safety precautions were taken before the enumerators left for the field 
to avoid any security issues. Some of the selected villages and communes were determined to be unsafe 
by Kantar Public standards and due to these concerns, the enumerators were not sent to those areas. 
This included two full communes in Diffa (Toumour and Bosso), where enumerators were not allowed to 
37
enter. Seven communes in Diffa and 6 communes in Agadez were determined to only be safe in the 
commune centers and enumerators were not allowed to survey in the surrounding selected villages. 
Additionally there were some selected villages that were deemed unsafe. These villages were replaced 
with the previously randomly selected villages. Some communes only had a limited number of safe 
villages available to conduct surveys in, so those villages replaced the originally selected ones. 
One of the teams did unknowingly trespass into a conflict zone, which led to a prompt response from the 
army. While there was some property damage, no one was hurt. Further villages in this area were not 
visited due to this incident and the enumeration team only worked in this commune’s capital. 
The survey firm also struggled to gain access to significant amounts of the local mayor’s time. Due to this, 
the leader survey was broken up into the subjective questions, which were administered to the mayor, 
and a ‘fact based’ section, which was administered to departments within the commune district office 
that could provide the most accurate information. 
Pre-Analysis Plan
The peer review of the evaluation design was facilitated formally through the Experiments in Governance 
and Politics (EGAP) group at Columbia University, of which USAID/DRG is an institutional member. 
Members of EGAP have signed full non-disclosure agreements to allow them to comment on in-design 
evaluation plans. The peer review will focus both on the technical soundness of the evaluation design as 
well as addressing USAID/DRG’s own strategic criteria regarding impact evaluations. The comments from 
EGAP should be incorporated into or addressed in the final evaluation design document, and will be used 
by USAID/DRG to make final determinations regarding co-funding. This peer review is in addition to the 
review and commenting period by HESN, USAID/DRG, and USAID/Niger described above.
38
Timeline
Fall 
16
Winter 
16-17
Spring 
17
Summer 
17
Fall 
17
Winter 
17-18
Spring 
18
Summer 
18
Fall 18 Winter 
18
Task Y2 Y3 Y4 Deliverables
Impact 
Evaluation 
Plan 
Finalized
Finalized 
Evaluation 
Plan
Pre￾analysis 
plan 
finalized 
and 
registered 
Finalized 
household 
survey firm 
Baseline 
survey
Baseline 
surveys 
(household, 
leader survey)
Baseline 
report 
Baseline report
Endline 
survey
Endline 
surveys 
Endline 
report
Endline report 
Roles and Responsibilities
Research Team:
Ariel BenYishay, Lisa Mueller, and Phil Roessler will serve as co-principal investigators for the project. 
BenYishay, Mueller and Roessler will be responsible for the planning and execution of all phases of the 
impact evaluation. They will develop the evaluation design and coordinate with a survey firm to carry out 
the baseline and endline evaluations.
39
AidData:
AidData will be responsible for the reporting requirements of HESN, USAID/DRG, and USAID/Niger. In 
regards to the HESN award, AidData will be required to meet all obligations as previously outlined in its 
award, including but not limited to: financial reporting, HESN M&E indicators, annual reports, etc. In 
regards to USAID/Niger, AidData is responsible for coordinating with BenYishay, Roessler, and Mueller on 
the evaluation design report and analysis report at the completion of each phase of the evaluation, as 
well as for sharing with the Mission the reports required by HESN. AidData will review any reporting 
documents or any modifications to the scope of work prior to approval.
Implementing Partner (Counterpart International):
AidData will work closely with Counterpart International (CPI) to ensure the success of the evaluation. At 
least one of the PIs will meet with CPI to ensure the roll out of the program will coincide with the 
evaluation design. AidData will also hold weekly calls with CPI to ensure close coordination between 
treatment, AidData’s evaluation, and CPIs program evaluation. As the evaluation design is carried out, 
the PIs will keep the implementing partner informed of the progress.
Survey partner (NORC):
NORC has been contracted by USAID to perform the baseline and endline survey for this project. AidData 
worked closely with NORC on the development and implementation of the baseline survey as well as on 
the review of the collection methodology and the baseline data. AidData staff will continue to work 
closely with NORC during the implementation of the endline survey to maintain a high level of data 
collection.
USAID/Niger:
The primary point of contact (POC) for AidData and for USAID/HESN will be Garret Harries. A secondary 
POC for AidData will be Abdourahamane Hassane, primarily for project-related technical concerns. The 
primary POC will be responsible for ensuring that communication is maintained between the Mission, 
HESN, DRG, and AidData, including through the required reporting listed above. Regarding reporting 
requirements, the primary POC will have final approval of the three required evaluation reports, and will 
review for information purposes, the reports required by HESN. The technical POC will be available to 
serve as liaison with the implementing partner and with any technical/field-based issues that may arise. 
The technical POC will be included in the Mission’s approval of the three evaluation reports, and will also 
view the interim reports required by HESN. The primary POC, however, has the responsibility of sharing 
40
these documents and soliciting the feedback and approval of others in the Mission, including the technical 
POC, as well as soliciting feedback from the primary POC in USAID/DRG.
USAID/DRG:
The primary POC for AidData and for USAID/HESN will be Morgan Holmes, Evaluation Specialist for 
USAID/DRG. The primary POC will be responsible for ensuring communication is maintained between the 
Mission, DRG, HESN, and AidData. The POC will also facilitate the peer review of the draft evaluation plan. 
The POC will be included in the Mission’s approval of the three evaluation reports, and will also view the 
interim reports required by HESN. NORC will also have the opportunity to review the three evaluation 
reports. Final approval of all reports is held by the primary POC for USAID/DRG. The primary POC will also 
be responsible for liaising through NORC that will provide the sub-contracting and management for data 
collection activities.
USAID/HESN:
USAID/HESN will be responsible for the day-to-day project management for this research project until the 
end of the HESN cooperative agreement to AidData. This means that the AOR for the HESN award to 
AidData will comply with reporting requirements under the award. He/she will also conduct periodic M&E 
on the progress of AidData’s PRG work to ensure that AidData is meeting its delivery timeframe on 
schedule. As necessary, the AOR will periodically communicate to USAID/Niger the status of AidData’s 
progress, as well as immediately notify the Mission if any problems arise during the course of this impact 
evaluation project.
AidData Staffing and Management Plan 
Principal Investigators
Dr. Ariel BenYishay—Dr. BenYishay is AidData’s Chief Economist and Assistant Professor of Economics at 
the College of William and Mary. He previously served as Lecturer in Economics at the University of New 
South Wales in Sydney. He also served as Associate Director of Economic Analysis and Evaluation at the 
Millennium Challenge Corporation. He has served as the principal investigator on a variety of large-scale 
experiments in developing countries, including Malawi, the Philippines, and the Solomon Islands. 
Dr. Lisa Mueller—Dr. Mueller is an Assistant Professor of Political Science and African Studies at 
Macalester College. Her research focuses on civic engagement and political economy of development in 
Niger. She has directed surveys in Niger on protest participation and citizen-politician linkages and has 
41
conducted additional fieldwork in Guinea, Mali, and Senegal. During summer 2015 she will be a Visiting 
Scholar at the West African Research Center in Dakar.
Dr. Philip Roessler—Dr. Roessler is an Assistant Professor in the Department of Government and Co￾Director of the Center for African Development at the College of William and Mary. He is an expert on 
African politics and has conducted qualitative, quantitative and experimental studies in a range of African 
countries, including Sudan, Chad, Democratic Republic of Congo, Rwanda, Liberia, Tanzania, and 
Zimbabwe. 
Additional Personnel
Bachirou Ayouba Tinni, Field Coordinator— Full-time staff member based in Niamey during baseline data 
collection, initial program roll-out, and potentially endline data collection. Staff member will be 
responsible for coordinating training of data collection team, monitoring during collection, and tracking 
program status and randomization compliance during initial roll-out. The coordinator would also be 
responsible for obtaining administrative data from relevant national sources. Depending on funding, 
coordinator may return to complete endline data collection. This individual will likely have a graduate 
degree or studies and have previous research experience in a similar setting.
Katherine Nolan, AidData Project Manager- AidData staff member who will assist with compliance with 
USAID/ HESN and USAID/Niger reporting requirements and coordinating among team members. 
GIS Analyst—AidData staff member who will assist in merging existing georeferenced datasets on 
population, economy, agriculture, ecology and other factors to ensure matched pairs are most precisely 
formed for the randomization.
Policy & Communications Analyst—AidData staff member who will assist with drafting policy briefs and 
communications materials to promote learning from the evaluation.