Constituency Dialogues and Citizen 
Engagement in Cambodia: 
Findings from a mixed methods impact evaluation
February 2013

The National Democratic Institute is a nonprofit, nonpartisan, nongovernmental 
organization working to support and strengthen democratic institutions worldwide 
through citizen participation, openness and accountability in government. 
Since its founding in 1983, NDI has worked with local partners in 125 countries and 
territories, bringing together individuals and groups to share ideas, knowledge, 
experiences and expertise. Partners receive broad exposure to best practices in 
international democratic development that can be adapted to the needs of their own 
countries. NDI’s multinational approach reinforces the message that while there is no 
single democratic model, certain core principles are shared by all democracies. 
The Institute’s work upholds the principles enshrined in the Universal Declaration of 
Human Rights. It also promotes the development of institutionalized channels of 
communications among citizens, political institutions and elected officials, and strengthens 
their ability to improve the quality of life for all citizens. For more information about NDI, 
please visit www.ndi.org. 
Copyright © National Democratic Institute 2013. All rights reserved. Portions of this work 
may be reproduced and/or translated for noncommercial purposes provided NDI or other 
contributing authors are acknowledged as the source of the material and is sent copies of 
any translation.
455 Massachusetts Ave, NW
8th Floor
Washington, DC 20001
Telephone: 202-728-5500
Fax: 888-875-2887
This report is made possible by the generous support of the 
American people through the United States Agency for International 
Development (USAID) under Award No. 442-A-09-00001. The 
opinions expressed herein are those of the author(s) and do not 
necessarily reflect the views of USAID or the United States 
Government.
CEPPS NDI Cambodia Impact Evaluation Report Page 1 
Table of Contents 
Forward........................................................................................................................................................ 2
Executive Summary..................................................................................................................................... 3
I. Background ...................................................................................................................................... 9
Political Context....................................................................................................................................... 9
NDI Programs in Cambodia .................................................................................................................... 9
Evaluation Context................................................................................................................................10
Evaluation Team Composition .............................................................................................................11
II. Evaluation Methodology ..............................................................................................................12
Development Hypothesis......................................................................................................................12
Evaluation Questions ............................................................................................................................14
Methodological Approach.....................................................................................................................14
III. Summary of Key Findings ............................................................................................................21
Constituency Dialogues.........................................................................................................................21
Deliberative Sessions ............................................................................................................................24
Deliberative Session Gender Dynamics ...............................................................................................25
IV. Conclusions.....................................................................................................................................27
Program Design and Implementation ................................................................................................. 27
Evaluation Design and Implementation ..............................................................................................29
V. Annexes……………………………………………………………………………………………………………………………….I-1 
 Annex I -Evaluation Report: The Constituency Dialogue Program in Cambodia (Susan 
Hyde)…………………………………………………………………………………………………….……………..….……………….I-1 
 Annex II- NDI Cambodia: Qualitative Findings from a Randomized Control Trial (Linda Stern) 
……………………………..…………………………………………………………………...…………………………………………………......II-1
CEPPS NDI Cambodia Impact Evaluation Report Page 2 
Forward
In 2008, the National Academies of Science completed a study recommending that USAID 
and its partners use randomized controlled trials (RCTs) to better understand the impacts 
of democracy assistance overseas. To that end, in 2009 USAID requested that the 
Consortium for Electoral and Political Processes (CEPPS) partners each include 
a randomized controlled trial (RCT) in their proposals to support USAID’s five-year 
Accountability in Governance and Politics (A-GAP) program in Cambodia. 
At the time few democracy promotion agencies, or their implementing partners, had the in￾house capacity to design or implement an RCT. Nor were the practical implications of such 
an endeavor fully understood. NDI completed the impact evaluation of its Multi-party 
Constituency Dialogue program in Cambodia in 2013, and the experience greatly 
strengthened the Institute’s internal capacity to monitor, evaluate and learn from its 
programing. 
While USAID funded the design and implementation of NDI’s RCT, to successfully complete 
the RCT the Institute mobilized additional resources, both internal and external, and owes 
a debt of gratitude to many people. NDI would like to thank the academic reviewers from 
the EGAP network - Experiments in Governance And Politic -- who provided their feedback 
and insights on NDI’s draft RCT design. NDI owes a particular debt of gratitude to Dr. 
Susan Hyde, Professor of Political Science and International Affairs at Yale University. 
Without her tireless support, creativity and guidance as the Principal Investigator the study 
would not have been completed. 
Linda Stern
NDI Director
Monitoring, Evaluation & Learning
CEPPS NDI Cambodia Impact Evaluation Report Page 3 
Executive Summary 
Background. The National Democratic Institute (NDI or the Institute) was chosen to 
participate in a U.S. Agency for International Development (USAID) pilot program to 
evaluate the impact of democracy and governance programs in Cambodia. An impact 
evaluation measures change caused by a program and also measures what would have 
happened if the program had not taken place. In such a study, participants are randomly 
assigned to participate in all, some, or none of the program activities, and data are collected 
before and after these activities to compare any changes across these groups. This report 
summarizes the impact evaluation conducted to measure the effect of the Institute’s 
program on Cambodians’ knowledge of, attitudes toward, and participation in political and 
civic life. Two companion reports present the different data sets, analyses and findings in 
greater detail, both of which are summarized in annexes to this report.
Political context. While Cambodia has made some progress in its transition to democracy, 
the country remains under semi-authoritarian rule. The ruling party gained a 
supermajority in the 2008 National Assembly elections, and opposition parties are 
marginalized from legislative and other political activities. There is little interaction 
between Members of the National Assembly (MNAs) and citizens. Government officials 
have publicly threatened not to serve citizens who support opposition parties. 
Program context. Since 2004, NDI and its local civil society partners have organized and 
facilitated constituency dialogues (CDs) in rural villages across twelve provinces that bring 
together MNAs from different parties to hear constituents’ concerns in a town-hall setting. 
This evaluation focused on three provinces to study the existing CD program as well as a 
citizen deliberation program component introduced for this evaluation. 
Deliberative sessions1 were comprised of small groups of citizens, similar to focus group 
discussions, convened the day prior to the CD. Deliberative sessions are different from 
focus groups in that participants used a standardized matrix-ranking grid to discuss and 
vote on priority community problems and proposed solutions to these problems.2 NDI 
trained facilitators to help participants use the matrix-ranking grid and to document the 
group’s discussion. Adding the deliberative sessions allowed NDI to study an innovative 
activity and to answer additional evaluation questions related to the study’s focus on 
citizen engagement. 
Evaluation context. The Institute incorporated an impact evaluation into its existing 
program design and budget. The evaluation design underwent various revisions in 
consultation with USAID. NDI received preliminary technical assistance from USAID during 
the evaluation design process and then managed the conclusion of the design process, 
implementation and analysis with its own evaluation team. Using a practitioner-academic 
 
1
 This program component was inspired by existing Participatory Action Research approaches and was 
designed for low-literacy populations to reflect, learn and plan action together.
2
 For each problem deliberative session participants identified two solutions – one internal solution that 
required community action, and one external solution that required action by an MNA.
CEPPS NDI Cambodia Impact Evaluation Report Page 4 
research model, the evaluation team was comprised of NDI staff, a Yale University 
professor, and a Cambodian research firm. 
Development hypothesis. The CD program is based on the theory of change that when 
citizens are provided a forum to engage with and voice their concerns to their elected 
representatives, these representatives will become more responsive and accountable to 
constituents. The development hypothesis is that participation in the CD positively affects 
both the supply (MNA) and demand (citizen) sides of this accountability relationship. This 
study focused on the citizen demand side of that hypothesis. NDI expected that the 
deliberative sessions would have an additional positive effect on citizens. Specifically, NDI 
examined two steps in this results chain: that a CD (and a deliberative session) increases 
participants’ understanding of their roles in political and civic life and that multiple 
political perspectives exist (step 1), which in turn leads citizens to demand that MNAs 
address their concerns (step 2). 
Mixed methods design. The evaluation design placed individuals into one of three categories 
for comparison: those who were not invited to any program activity; those who were 
invited to attend a CD; and those who were invited to participate in a deliberative session 
and a CD. Deliberative session participants were then randomly assigned to all-male, all￾female or mixed gender groups. In many mixed methods evaluations, qualitative data (such 
as focus groups) are used only to describe programmatic context as a supplement to survey 
or other quantitative findings. A unique feature of this mixed methods design was that both 
survey and deliberative session data were used to study effects caused by the program; 
other data describe the programmatic context. 
Evaluation questions and key findings. The Institute posed four main questions to answer 
through the impact study. Experimental findings describe changes or effects caused by the 
CD or the deliberative session. Descriptive findings illustrate the context in which the 
program occurred or describe the content of program activities. 
1. Does the constituency dialogue change individual attitudes, knowledge and behavior?
Experimental findings. A CD caused increases in knowledge of the political process and 
self-reported engagement in civic and political behaviors among individuals invited to 
attend a CD.3 Respondents were significantly more likely to report a personal interest in 
politics after a CD was conducted in their village. These individuals were also more 
likely to report that their MNAs communicated with and conducted activities on behalf 
of their communities. On average, the CD was not found to have positive or negative 
effects on surveyed individuals’ confidence about politics in Cambodia or their role in 
the political process. However, women in CD villages were more likely than women in 
control villages to report positive attitudes about the political process and their role as 
voters after a CD was held in their village. 
 
3
 Women in CD villages were more likely than men to self-report some civic and political behaviors, such as 
filing a case in court or supporting a citizen petition.
CEPPS NDI Cambodia Impact Evaluation Report Page 5 
Descriptive findings. Participating MNAs generally adhered to the standard code of 
conduct implemented by NDI staff at all CDs that establishes protocols for civil conduct 
toward other MNAs and citizens. A balance of ruling and opposition party supporters 
was observed in CD audiences. Heavy police presence was observed at one third of the 
CDs within the study, and in some cases, CD attendees appeared to be nervous or 
intimidated by the police.
2. Does participation in a deliberative session before the constituency dialogue change 
individual attitudes, knowledge, behavior, or priority issues?
Experimental findings. The deliberative session had few detectable effects on 
individuals’ responses to survey questions about political and civic knowledge, 
attitudes, reported behavior, or priority issues. These findings are limited by the fact 
that fewer individuals than expected acted on the deliberative session invitation, which 
greatly diminished the sample size available for this analysis. 
Descriptive findings. Small groups’ discussions demonstrated that priority problems 
varied by province and commune; however, groups across communes identified 
corruption as a multi-faceted problem affecting their individual, family, and community 
experiences. Land seizure was acutely noted in one third of the communes, where 
participants associated this problem with detrimental environmental and health effects 
and instances of violent retribution or imprisonment for citizens who mobilized in 
protest. 
Additional descriptive findings showed that deliberative session participants were 
more active at the CD than other CD attendees. Deliberative session participants raised 
their hands and spoke at a CD more than twice as often as CD participants who had not 
attended a deliberative session.
3. Does the gender of the facilitator or gender composition of deliberative sessions 
differentially impact participant attitudes, knowledge, or behavior?
Experimental findings. The facilitator’s gender and the gender composition of the group 
had few detectable effects on participant responses to survey questions about their 
knowledge of, attitudes toward, and participation in political and civic life. These 
findings should be interpreted cautiously due to the small percentage of surveyed 
individuals who accepted the deliberative session invitation and were considered in 
this analysis. 
Additional experimental findings demonstrate that the facilitator’s gender affected 
participant voting patterns for priority solutions. Male-facilitated deliberative groups 
were significantly more likely than female-facilitated groups to vote in consensus, 
“clustering” their votes for single solutions as well as among their top-ranked 
solutions.4 These effects are statistically significant but based on measures piloted in 
 
4
Deliberative session facilitators recorded participant vote totals for the priority solutions on each group’s 
matrix-ranking grid. Several measures were used to analyze the degree to which participants distributed 
CEPPS NDI Cambodia Impact Evaluation Report Page 6 
this study. Understanding the meaning of these findings for measuring deliberation –
whether positive or negative – requires further study.
4. Does the facilitator gender or gender composition of deliberative sessions differentially 
impact participant priority issues?
Experimental findings. Both the facilitator’s gender and the group’s gender composition 
influenced how groups ranked their priority problems. The facilitator’s gender also 
affected groups’ vote preference for priority solutions. On average, all groups tended to 
prioritize external solutions (those that required MNA action) over internal solutions 
(those that required citizen action) to resolve their identified problems. However, male￾facilitated groups were significantly more likely than all other group types to prioritize 
external solutions, whereas female-facilitated groups were significantly more likely to 
prioritize internal solutions compared to all other group types. 
Conclusions. These conclusions are intended to contribute to NDI’s current and future work, 
as well as to the ongoing program design and evaluation dialogue within the international 
democracy assistance community of practitioners, donor agencies, and academics. 
Programmatic conclusions: 
 This study showed that a CD caused positive change in individual knowledge levels and 
self-reported engagement in some civic and political behaviors, but did not detect 
evidence of the program causing citizens to demand that MNAs address their concerns. 
Lack of evidence does not necessarily mean that there is not a link between the CD program 
and citizen demand in Cambodia, and certain factors beyond the scope of this study could 
help to better understand this linkage. Data could be collected over a longer period of time 
to capture medium- or long-term changes, beyond the period immediately after a CD. The 
study relied on measures of self-reported political and civic behavior, but measures of 
observed behavior could show different outcomes in actual practice (see evaluation 
conclusion on behavior indicators below). Data was collected after a single CD (or 
deliberative session), but participation in multiple program activities could have a positive 
effect on citizen demand. Finally, the additional program support could be provided to 
include more explicit civic education or citizen engagement content than the CD events 
alone. 
 The deliberative session pilot effort generated reliable qualitative data within a 
randomized impact evaluation and demonstrated the differences in men’s and women’s 
voices as well as the risks Cambodians face in speaking out about such issues as 
corruption. 
A single deliberative session is unlikely to support sustainable solutions to the problems 
discussed and could even put participants at risk by encouraging the discussion of such 
 
their votes, or not, across all possible solutions and between internal (requiring citizen action) or external 
(requiring MNA action) solutions.
CEPPS NDI Cambodia Impact Evaluation Report Page 7 
topics. Rather than a stand-alone program activity, deliberative sessions should be 
considered part of a broader programmatic approach to support citizen engagement and 
demand for greater responsiveness and accountability of their elected officials. 
 The mixed methods approach of this impact study highlighted positive changes in 
individuals’ knowledge caused by the program as well as the contextual challenges and 
risks of greater citizen engagement in Cambodia. 
While the CD caused some positive changes for individuals, data from the deliberative 
sessions captured participants’ experiences – beyond the context of the CD – with 
corruption, land seizure, intimidation and sometimes violence. Together, the different data 
sets provide a fuller picture of citizen engagement in the Cambodian context than any of the 
data sets do alone. These findings point to the need for further study of the conditions 
related to and consequences of greater citizen demand in a closed political system. 
 The scope of this evaluation did not study the effects caused by this program among 
MNAs, and many questions remain regarding changes in their accountability to citizens. 
Since the CD program started in 2004, MNAs have been the target program audience prior 
to the impact study. In the Cambodian context, it would not be feasible to randomize MNA 
participation in the program. Similar to the questions noted above about citizen demand, 
more systematic information is needed to understand program effects for MNAs. Program 
staff has collected information that demonstrates anecdotes of both citizen demands for 
action and MNA initiatives to address citizen demands, and this non-experimental data 
could contribute to evaluation efforts around MNAs. 
Evaluation conclusions: 
 The academic-practitioner partnership strengthened the evaluation team and the 
quality of the impact study, and this model should be considered when feasible for 
future impact studies in the democracy and governance sector. 
This partnership merged practitioner and academic expertise on equal footing. It was 
critical in exploring evaluation design options, troubleshooting implementation challenges 
and analyzing the complex body of data while preserving the study’s rigor.
 The evaluation team considered various indicators to measure change in individual’s 
behavior in the evaluation design, but few of these measures proved viable in practice. 
The evaluation team attempted to incorporate measures of observed citizen behavior 
outside of the program activities, but ultimately relied on survey respondents’ self￾reported behavior and observation of participants’ behavior at the CD. The social and 
political context and practices rendered other measures of citizen behavior unfeasible (e.g. 
commune council meeting attendance records are not consistently maintained) or 
potentially harmful for participants (e.g. citizen petitions could draw negative attention to 
organizers). These measurement challenges were shaped by the Cambodian political 
CEPPS NDI Cambodia Impact Evaluation Report Page 8 
context, but similar issues are likely to be encountered across democracy and governance 
programs in other countries. 
 The deliberative session matrix ranking measures were effective in capturing reliable 
qualitative data within a randomized impact evaluation but may be cost prohibitive for 
many programs. 
As noted above in the programmatic conclusions, deliberative sessions should not be 
considered as stand-alone or one-time program activities. Deliberative session data 
collection produced a large volume of data, and both the collection and analysis process 
required significant staff and program resources. This study was limited by the fact that 
fewer than expected individuals acted on the deliberative session invitation, which resulted 
in a smaller sample size for analysis. Future work with deliberative sessions should take 
these factors into account for program design and cost planning. 
CEPPS NDI Cambodia Impact Evaluation Report Page 9 
I. Background
Political Context 
Since the Paris Peace Accords of 1991 and the administration of the United Nations 
Transitional Authority, Cambodia has made progress in its shift to democracy, particularly 
in developing an increasingly robust civil society and a decline in election violence. 
However, the overwhelming dominance of the Cambodian People’s Party (CPP) in all facets 
of government has kept the country under semi-authoritarian rule marked by patronage, 
rampant corruption and lack of accountability among government and elected officials.
While Cambodia has little tradition of political rivals participating in a constructive contest 
of ideas, the role of the opposition was further limited after the National Assembly 
elections in 2008. The CPP gained 17 additional seats and, with 90 of the 123 seats in the 
assembly, a two-thirds supermajority that can be used to monopolize legislative debate. 
Opposition parties such as the Sam Rainsy Party (SRP) and the Human Rights Party (HRP) 
hold 26 and 3 seats respectively, but these seats are distributed in only 12 of the country’s 
24 provinces. The opposition does not hold any seats in the nine parliamentary 
commissions. The two other parties represented in the National Assembly – the National 
United Front for an Independent, Neutral, Peaceful and Cooperative Cambodia (known by 
its acronym in French, FUNCINPEC) and the Norodom Ranariddh Party (briefly known as 
the National Party), once a strong royalist party – now hold only two seats each. Ruling 
party leaders have openly questioned the right of opposition members to comment on the 
government's legislative agenda, and government officials have threatened not to serve 
those citizens who support the opposition.
A centralized decision-making process, concentrated almost exclusively within the 
executive branch, has left Cambodia’s legislature a weak and opaque institution. The 
National Assembly does not conduct legislative hearings nor does it have a formalized 
public consultation process. Bill drafting is conducted by the executive, and information on 
the process is carefully guarded; the public cannot access transcripts of policy 
deliberations. Assembly members do not exercise any oversight function of the 
government. In addition, little interaction exists between Members of the National 
Assembly (MNAs) and their constituents. Members are elected by party-list system with a 
proportional distribution of seats by province, allowing an MNA to be elected without 
interacting with constituents before or after an election. This system provides limited 
channels for citizens to be included in public policy development or to hold elected officials 
accountable for use of public resources. Few MNAs have made sustained efforts to conduct 
independent outreach activities through their political parties, and some lack a clear 
understanding of their official role as legislators. Based on prior NDI focus groups with 
citizens, the public also has little understanding of the legislative function of the National 
Assembly and tends to look to MNAs as providers of material goods, services and money.
NDI Programs in Cambodia
NDI has aided democratic activists in Cambodia since 1992 to attain more open and 
inclusive political processes. NDI's programming seeks to promote a more level political 
CEPPS NDI Cambodia Impact Evaluation Report Page 10
playing field by improving the access of all political parties and civil society to the media, 
promoting women’s political engagement, and enhancing the capabilities of political 
parties to participate more equitably in elections. To this end, NDI has organized public 
debates among candidates from different political parties contesting seats in commune 
councils and the National Assembly. The Institute has supported local election monitoring 
organizations and citizen volunteers to increase transparency in the electoral process and 
to identify needed reforms to the electoral framework through systematic voter 
registration and election observation efforts and voter registry audits. At the grassroots 
level, the Institute has assisted civil society groups across Cambodia in engaging their 
communities and elected representatives on issues of common concern.
Constituency dialogues. Since 2004, NDI and its local civil society partners have conducted 
more than 200 constituency dialogues across 12 provinces that bring together MNAs from 
different parties to hear constituents’ concerns in a town-hall setting.5 The dialogues are 
moderated by NDI staff according to a standard agenda and code of conduct. Members of 
the audience can pose questions to the MNAs during two open microphone periods. MNAs 
are each allotted equal time increments to make comments and respond to citizen 
questions. Citizen efforts to hold MNAs to account have included expressing concerns to 
MNAs directly about conflicts over land, public services that were not delivered and cases 
of local authorities abusing their power. There have been instances of MNA or other 
government official action take after some dialogues, including: the construction of canals 
and pathways; the reinstatement of compensation for rural school teachers; the return of 
land to its rightful owners; and the investigation of local officials.6
Evaluation Context
Evaluation purpose 
Following the release of the USAID-commissioned 2008 National Academy of Sciences 
study, Improving Democracy Assistance: Building Knowledge through Evaluations and 
Research,
7 USAID sought to strengthen the quantity and quality of its evaluations. As part of 
this effort, USAID selected NDI’s program in Cambodia to participate in a 2009 pilot effort 
to evaluate the impact of democracy and governance programs. As part of USAID’s program 
description requirements, the Institute incorporated an impact evaluation into its overall 
program design and budget proposal.
The initial focus of the evaluation was to conduct an impact evaluation of a democracy and 
governance program. NDI managed the evaluation design process, including the definition 
of the evaluation questions, with some preliminary technical assistance from USAID. 
 
5 This program is implemented with USAID funding through the Cambodia: Accountability in Governance and 
Politics (AGAP) program. 
6 NDI program information and past CD reports can be found at http://www.ndi.org/content/cambodia. 
7 National Research Council 2008. 
CEPPS NDI Cambodia Impact Evaluation Report Page 11
Evaluation design development 
NDI originally submitted to USAID a proposed impact evaluation design to include all CDs 
to be implemented over the course of the program’s five year timeline to study the effect of 
the program on both citizens and MNAs. The scope of this evaluation design would have 
required more resources than the funding available; therefore, NDI narrowed its scope. 
USAID provided short-term technical assistance8 to support NDI’s development of this 
revised design, which focused on the impact of the program on citizens. NDI faced the 
implementation challenge of integrating the evaluation design into a long-standing 
program. Since 2004, CDs had been conducted in villages in many of the communes, or 
villages in surrounding communes, that were being considered for the study. As a result, 
NDI mapped out a smaller number of provinces with a sufficient number of villages where 
CDs had not been conducted previously. NDI further refined the evaluation design with 
feedback from USAID evaluation experts and academics affiliated with the Experiments in 
Governance and Politics (EGAP) network.9
Evaluation Team Composition 
NDI found the EGAP network to be a valuable space for peer review of the evaluation 
design and for the formation of an academic-practitioner partnership with the Principal 
Investigator for the duration of the evaluation. The evaluation team was built around this 
partnership and was comprised of NDI staff in the Cambodia and Washington, DC offices, a 
political science professor from Yale University, a field researcher, and a local research 
firm. Evaluation team members and their roles included: 
 Principal Investigator. Dr. Susan D Hyde10 contributed to the EGAP peer review of 
the evaluation design and then served as the principal investigator for the 
remainder of the evaluation. She provided technical assistance during the transition 
from design to implementation and led the quantitative analysis and drafting of 
those findings.
 NDI Director of Monitoring, Evaluation and Learning. Linda Stern designed the 
evaluation and served as the technical advisor on the deliberative session methods, 
including the design and piloting of the discussion guide and protocol for facilitation 
and data collection. She led the qualitative analysis and drafting of those findings.
 NDI Evaluation Manager. Alison Miranda contributed to the evaluation design 
process and managed team resources and communication through the analysis and 
reporting phase, including authoring this report.
 
8 The funding mechanism that supported this technical assistance expired prior to the implementation phase 
of the evaluation. 
9 EGAP is a network of academic researchers and development practitioners working on experimental 
analyses of governance and politics. Both NDI and USAID are members of the network. Among other 
resources, EGAP offers a peer review process for member-submitted impact evaluation designs (http://e￾gap.org/).
10 Hyde is an Associate Professor of Political Science and International Affairs at Yale University. She served as 
a pro bono advisor on this project.
CEPPS NDI Cambodia Impact Evaluation Report Page 12
 NDI Program Managers. Marjan Ehsassi and Christina Costello managed evaluation 
team consultant agreements, the overall program budget, including impact 
evaluation resources, and contributed to this report.
 NDI Senior Resident Director. Laura Thornton is based in Cambodia and manages 
all aspects of NDI’s programmatic activities in the country, including oversight of 
program implementation and supervision of office operations and personnel. She 
provided management support of field-based evaluation team members. 
 Field Researcher. Emily Lamb and Molly Watts were each based in Cambodia to 
manage the overall data flow from collection through translation, and ensure the 
integrity and quality of the quantitative and qualitative data sets. 
 NDI Senior Program Officer. Kimsrun Chhiv coordinated the deliberative session 
facilitators, provided them support in the field, and liaised between the Field 
Researcher and facilitators, as needed.
 Research Firm. Cambodian research organization the Centre for Advanced Study 
(CAS) supported survey pilot efforts, and managed survey field administration and 
enumeration.
 NDI Qualitative Analyst. Ornanong Maneerattana managed the qualitative data 
inventory and contributed to final data analysis with the NDI Director of Monitoring, 
Evaluation and Learning. 
 NDI Quantitative Analyst. Transitioning from the Field Researcher role, Molly 
Watts managed the quantitative data inventory and contributed to final data 
analysis with the Principal Investigator. 
II. Evaluation Methodology 
Development Hypothesis 
Constituency dialogues bring together MNAs from different parties to hear constituents’ 
concerns in a town-hall setting. Such a public forum can help to level the playing field 
between governing and opposition parties and raise MNA’s awareness of their political 
competition and of constituents’ needs. For citizens, this is a forum to publicly voice their 
concerns, better understand elected officials’ roles and responsibilities, and hear from 
multiple political perspectives. NDI’s hypothesis is that, over time, these factors will lead 
citizens to demand more responsiveness and accountability from their MNAs, and 
ultimately, for MNAs to be more responsive to their constituents (see Figure 1 below). 
CEPPS NDI Cambodia Impact Evaluation Report Page 13
Figure 1 CD Program Development Hypothesis 
NDI implemented a field experiment – that is, a randomized evaluation conducted in the 
field rather than through desktop research – in three provinces, Kampong Cham, Kampong 
Chhnang and Kratie, to evaluate the impact of the program on Cambodians’ knowledge of, 
attitudes toward, and participation in political and civic life. The evaluation design placed 
individuals into one of three categories for comparison: those who received no program 
activities at all; those who were invited to attend a CD held in their village of residence; and 
those who were invited to participate in a deliberative session and a CD, both of which took 
place in their village of residence. 
NDI varied the CD program by introducing a deliberative session conducted one day prior 
to the CD. The deliberative sessions were not an existing component of the CD program but 
Objective: MNAs are more responsive/accountable to 
their constituents
MNAs' 
perceptions of 
political 
competition are 
increased
MNAs' 
understanding of 
their 
constituents' 
needs is 
increased
Constituents 
create pressure 
on MNAs to 
address their 
concerns
Constituents 
better 
understand their 
role and options 
in the democratic 
process
Program activity: Constituency dialogues are held
Program variation: Deliberative sessions are held
CEPPS NDI Cambodia Impact Evaluation Report Page 14
did contribute to the study’s focus on the impact of the program on citizens. NDI’s 
experience in other countries and regions suggested that a smaller forum for citizens to 
discuss issues – especially in politically uncertain environments – is a pre-condition for 
change in individual and/or collective behavior such as pressuring representatives to 
address concerns. NDI introduced this activity after narrowing the evaluation design to 
include three provinces. Adding the deliberative sessions allowed NDI to study an 
innovative activity and to answer additional evaluation questions. 
Comprised of three to five individuals, the deliberative sessions were small group meetings 
designed for low-literacy populations to reflect, learn and plan action together. This 
program component was inspired by, and utilized tools adapted from the existing body of 
participatory action research approaches. Trained facilitators oversaw the deliberative 
sessions, but the sessions were structured for group participants to largely self-facilitate 
their identification of three local problems and six potential solutions (represented by 
pictures) using a matrix-ranking tool (a large paper grid). For each of the three priority 
problems identified, participants brainstormed and agreed on two solutions – one internal 
solution that required community action, and one external solution that required action by 
an MNA. Finally, participants voted for the priority solutions and closed the session with a 
brief discussion of the outcomes. 
Building on the above results chain outlined by a dashed line in Figure 1, NDI further 
assumed that participation in a deliberative session and a CD – rather than participation in 
a CD alone – would increase the likelihood of a positive effect on citizen attitudes, 
knowledge and behavior. In addition, NDI postulated that the CD and the deliberative 
session would have different effects on men and women. 
Evaluation Questions
The Institute posed four main evaluation questions to answer through the field experiment:
1. Does the constituency dialogue change individual attitudes, knowledge and 
behavior?
2. Does participation in a deliberative session before the constituency dialogue change 
individual attitudes, knowledge, behavior, or priority issues?
3. Does the facilitator gender or gender composition of deliberative sessions 
differentially impact participant attitudes, knowledge, or behavior? 
4. Does the facilitator gender or gender composition of deliberative sessions 
differentially impact participant priority issues?
These questions were broken down into eight more specific research questions for 
analysis. The companion reports summarized in annexes I and II reference and analyze the 
eight questions.
Methodological Approach 
The evaluation team applied three levels of randomization to form the following groups: 
CEPPS NDI Cambodia Impact Evaluation Report Page 15
 Type 1 (in villages where no CD was conducted): individuals not invited to any 
program activity; 
 Type 2 (in villages where a CD was conducted): individuals invited to attend a CD; 
and
 Type 3 (in villages where a CD was conducted): individuals invited to attend both a 
deliberative session and a CD. 
Random assignment 
1: Village assignment. Since 2004, NDI has collaborated with local partner organizations to 
identify and select CD village locations, handle local logistics for the CD event, and 
contribute to the management of relations with local authorities in a given province. In an 
effort to minimize modifications to the standard program implementation practice, NDI 
continued to engage local partner organizations in the selection of rural villages where CDs 
were to be conducted during the field experiment. At least one month prior to the target CD 
implementation date, local partner organizations identified potential pairs of villages in 
their respective provinces in which to conduct a CD. Selected villages had to be rural but 
accessible by road and not have been the site of a CD before. Based on local partner 
organizations’ knowledge, villages with ongoing contentious community conflicts or high 
levels of political tension were not selected. Flipping a coin, NDI assigned one village to 
receive no program activity (Type 1, control village) and one village to have a CD 
(treatment village). 
NDI conducted one CD in each of nine treatment villages during the field experiment, along 
with pre- and post-CD surveys (see Table 1 below). Among the nine villages that did not 
receive any program activity, only six villages were surveyed. Thus, for the survey data 
analyses of the effects of the CD, the sample includes only six pairs of treatment and control 
villages. 
Table 1 Village Assignment for CD 
Province District Commune Village CD / No CD Survey / No 
Survey
Kampong 
Cham
Memot Chaom Ta Mau Bos Ta Oem CD Survey
Kravean Satum No CD Survey
Chamkar 
Leu
Ta Prok Rumcheck CD Survey
Cheyyou Cheyyou No CD Survey
Kaoh Soutin Kampong Reab Kampong Sdei 
Kraom
CD Survey
Pongro Pongro Kaeut No CD No survey
Kampong Baribour Pech Chanvar Tang Trapeang CD Survey
CEPPS NDI Cambodia Impact Evaluation Report Page 16
Province District Commune Village CD / No CD Survey / No 
Survey
Chhnang Trapeang Chan Trapeang Chan No CD Survey
Kampong 
Tralach
Ta Ches La Peang CD Survey
Longveaek Boeng Kak No CD Survey
Rolea B'ier Krang Leav Krang Leav CD Survey
Tuek Hout Kouk Sdau No CD No survey
Kratie Sambo Koh Khnhaer Svay Chek CD Survey
Ou Krieng Ou Krieng No CD Survey
Chit Borey Thmar 
Andaeuk
Chuor Krouch CD Survey
Thmei Chronorl No CD Survey
Kratie Town Sangkat 
Orussey
Sre Sdao CD Survey
Sangkat Kaoh 
Trong
Dem Koh No CD No survey
2: Individual assignment. Within each village assigned to have a CD, NDI used the pre-CD 
survey to randomize invitation to the deliberative session. CAS survey enumerators 
administered the baseline survey one week prior to the CD. They randomly selected a 
household for the first survey using a map of village households. The survey enumerators 
then interviewed one adult in each household, flipping a coin to determine whether they 
interviewed a man or woman. At the conclusion of every baseline survey in villages 
assigned to have a CD, the respondent was invited to attend the upcoming CD (Type 2). A 
subset of randomly selected survey respondents living in CD villages were also issued a 
non-transferable invitation to participate in a deliberative session the day before the CD 
(Type 3). Survey respondents in control villages were not invited to attend any event. A 
follow up survey was conducted with the same respondents one week after the CD.11 For 
the survey data analyses of the effects of the deliberative sessions, the sample included the 
nine villages where a CD was conducted. 
3: Deliberative session assignment. To examine the effect of gender on the deliberative 
sessions, NDI created four different deliberative group types among participants:
 
11 Control village baseline and follow-up surveys were administered on a timeline comparable to the survey 
timeline of the CD treatment village in the same district. 
CEPPS NDI Cambodia Impact Evaluation Report Page 17
Table 2 Deliberative Session Gender Variations 
Facilitator
Group composition
Female facilitators Male facilitators
All female participants Type A NONE
All male participants NONE Type B
Mixed participants Type C Type D
Prior to each deliberative session, the field researcher randomly assigned ten trained 
facilitators to work with a specific group type, taking into account the facilitators’ gender 
and the desired group gender composition. When participants arrived for the session, the 
field researcher randomly assigned individuals to a group composition type that 
corresponded to the participant’s gender. 
All groups used a matrix-ranking tool to record their priority problems and solutions. The 
pictures representing their top three problems were placed vertically on the left hand side 
of the matrix grid. The six solutions – one internal and one external solution for each 
problem – were placed horizontally across the top of the matrix grid. Participants used 
buttons to cast votes for their priority problems and solutions, and the facilitator recorded 
the vote totals on the matrix during the session. Facilitators monitored the discussion 
around the identification of priority issues and solutions and the voting outcomes for each 
deliberative session and recorded the information on forms for later analysis. The 
facilitator notes and the deliberative groups’ matrix voting patterns were used as data sets 
as noted below. 
Mixed methods approach 
This study used a variety of quantitative and qualitative methods to understand the 
program context and answer the four main impact evaluation questions about the effect of 
CDs and deliberative sessions on citizen attitudes, knowledge, behavior, and priority issues. 
NDI used a mixed methods approach to 1) collect experimental data (data that can answer 
questions about causal effects) about the program components; 2) collect descriptive data 
(data that does not answer questions of causality) about the program components and the 
broader context in which they occurred; and 3) use multiple data sets to examine specific 
elements of the program (triangulation), such as individuals’ priority issues. In many mixed 
methods evaluations, qualitative data (such as focus groups) are used only to describe the 
context in which the program occurs as a supplement to survey or other experimental 
findings. A unique feature of this mixed methods design was that survey and deliberative 
session data sets were used to study effects caused by the program; additional data from 
deliberative sessions and the CDs were used to describe the programmatic context as well. 
Quantitative and qualitative data collection included: 
CEPPS NDI Cambodia Impact Evaluation Report Page 18
 CAS fielded pre- and post-surveys to collect baseline and outcome data in CD and 
control villages. 
 Deliberative session participants generated and ranked priority problems and then 
used a matrix-ranking grid to vote on their priority community problems and 
solutions; the evaluation team analyzed the groups’ matrix priorities and matrix 
voting patterns. 
 The facilitators used a form to record notes from the deliberative group’s discussion 
of the problems and solutions identified for each session they facilitated. 
 The facilitators used a CD observation form to record their deliberative session 
participants’ attendance and behavior at the CD and the behavior of police, local 
authorizes and MNAs at the CD event.
 Facilitators also recorded their personal reflections after each session they 
facilitated to assess perceived changes on a series of standardized questions, 
including their facilitation ability and the group dynamics, among other factors. 
 Finally, NDI staff and representatives of its local partner organizations monitored 
follow-up by MNAs and local authorities on the issues raised at each CD. The issue 
monitoring forms were used in both the CD villages and the control villages.12
For the purposes of this report, the evaluation team prioritized the analysis of some of the 
available data sets to identify key findings. 
Samples for analysis 
This study drew on several of the data sources noted above to conduct various levels of 
analyses. The findings presented consider the following comparison groups: 
 Type 1: individuals not invited to any program activities; 
 Type 2: individuals invited to attend a CD; 
 Type 3: individuals invited to attend both a deliberative session and a CD; and 
o Among deliberative session participants:
 Group type A: Female facilitator with all female participants
 Group type B: Male facilitator with all male participants; and
 Group type C/D: Mixed participants with either a female or a male 
facilitator.13
A more detailed discussion of the survey analyses and findings is available in the report 
referenced in annex I, and the matrix voting pattern analyses are discussed in the reports 
referenced in annexes I and II. 
CD analysis. The CD analysis compared experimental survey data among individuals in 
control villages (Type 1) and individuals in CD villages (Type 2) (see Table 3 below). At 
baseline, individuals in villages selected for CDs were slightly more likely to have attended 
 
12 The issue monitoring forms were not analyzed for this report or the findings presented in the qualitative 
and quantitative companion reports. 
13 For analysis purposes, mixed participant deliberative session groups were considered one type, whether 
the group was facilitated by a male or female, and compared to the other single-gender group types. 
CEPPS NDI Cambodia Impact Evaluation Report Page 19
a prior CD and much more likely to report having heard a CD broadcast on the radio before, 
compared to individuals in control villages. It is unlikely but unclear whether prior CD 
attendance or radio exposure caused any substantive differences in the CD effects reported 
below. Among those individuals who completed the endline survey across all treatment 
villages, 56% of respondents reported having attended a CD. No respondents in the control 
villages reported having attended a CD. 
Table 3 Sample for CD Analysis 
Control villages CD villages
Villages 6 villages (2 villages x 3 provinces) 6 villages (2 villages x 3 provinces)
Baseline survey 
respondents
720 baseline respondents
(120 respondents x 6 villages) 720 baseline respondents
(120 respondents x 6 villages)
Endline survey 
respondents
661 endline respondents
(92% of baseline respondents) 655 endline respondents
(91% of baseline respondents)
Deliberative session analysis. The deliberative session analysis examined experimental 
survey data within a CD village, comparing individuals who were invited to attend both a 
CD and a deliberative session (Type 3) with those individuals who were invited to attend a 
CD only (Type 2) (see Table 4 below). Of the survey respondents invited to attend a 
deliberative session, 46% reported that they actually attended. Among survey respondents 
invited to attend a CD only, 7% reported that they attended a deliberative session. 
The evaluation team also conducted an extensive analysis of citizen priorities across 
provinces and communes using descriptive qualitative data. These findings are based on 
deliberative group matrix-identified priority problems and solutions, facilitator 
documentation of both the deliberative session discussions and the CD environment, and 
facilitator CD observational data. 
Table 4 Sample for Deliberative Session Analysis 
CD villages
Villages 9 villages (3 villages x 3 provinces)
Baseline survey 
respondents 1,080 total baseline respondents (120 respondents x 9 villages)
Treatment CD only Deliberative session + CD
Baseline survey 
respondents
540 respondents invited 
(60 respondents x 9 villages)
540 respondents invited 
(60 respondents x 9 villages)
Endline survey 
respondents
498 endline respondents 
(92% of baseline respondents)
499 endline respondents 
(92% of baseline respondents)
Deliberative session gender analysis. As a sub-set of the deliberative session analyses, the 
evaluation team explored whether the gender composition of the group or facilitator 
gender had an impact on deliberative session participants’ attitudes, knowledge, behavior, 
or priority issues. The total number of groups formed during the study is presented in 
CEPPS NDI Cambodia Impact Evaluation Report Page 20
Table 5 below. These analyses compare each group’s gender composition to the other 
possible group composition types. Similarly, the effect of having a male facilitator is 
compared to that of having a female facilitator. 
These questions were analyzed using survey data among individuals who were invited to 
attend a deliberative session. However, as noted above, fewer than expected individuals 
completed both surveys and attended a deliberative session. The limited findings 
presented should be interpreted with caution. Additional experimental findings presented 
were drawn from the analysis of the matrix voting patterns generated by the deliberative 
groups. These experimental findings are discussed in both companion reports, and the 
specific measures are described in detail in the qualitative findings report. 
Table 5 Sample for Deliberative Session Gender Analysis 
CD villages
Villages 9 villages (3 villages x 3 provinces)
Treatment Deliberative session
Deliberative 
session participants 322 total participants
Group type A. All female 
group
B. All male 
group
C. Mixed group
Female 
facilitator
D. Mixed group
Male facilitator
24 groups 24 groups 16 groups 16 groups
Data limitations
There are several known limitations to the data collected, which should be taken into 
consideration in interpreting the findings presented and in exploring the possibility of 
future data collection related to this study. 
Survey sample. The survey sample is stratified on gender, which means that a sufficient 
number of male and female survey respondents were chosen to allow for comparison of 
effects between these two sectors of the population. In consultation with CAS, NDI 
determined that it would not stratify the survey sample by age, in part because of budget 
considerations (more surveys would have been required to compare effects by age group). 
The findings cannot be generalized to the broader village or country population. Within 
each gender, the survey respondents are representative of the village population that is 
most likely to be exposed to the CD. 
Survey timeline. The two week period between the pre- and post-survey is a relatively short 
time horizon. Surveys were administered on a rolling basis, depending on the date of the 
CD. This plan allowed for the same data collection timeline relative to each CD; however, 
CDs were conducted over a nine month period, spanning seasons that require the 
population to travel for agricultural work. With input from local research firm CAS, NDI 
chose this data collection timeline to minimize the challenge anticipated in locating the 
same individuals for the second survey. While this data does not address the sustainability 
of any changes caused by the program activities, a subsequent survey in the target villages 
CEPPS NDI Cambodia Impact Evaluation Report Page 21
would reveal whether these findings persist at the village level – rather than the individual 
level. 
Compliance rates. Compliance rate refers to an individual’s completion of the steps related 
to the treatment group to which s/he was assigned. For example, an individual assigned to 
receive no treatment (Type 1) was to complete a pre- and post-survey without attending 
any program events between these surveys. An individual invited to attend a deliberative 
session (Type 3) was to complete a baseline survey, attend a deliberative session, then 
attend a CD, and finally complete an endline survey. There are many factors that could have 
prevented individuals from complying with each of the desired steps in their assigned 
treatment group. In anticipation of this, the evaluation team invited more individuals to 
attend the deliberative session than were anticipated to act on the invitation. In this study, 
not all survey respondents in CD villages actually attended the CD. Similarly, far fewer 
individuals invited to the deliberative sessions actually attended, and other un-invited 
individuals attended the sessions. The findings presented almost certainly underestimate 
the effects – positive or negative – that would have been measured if all survey 
respondents had participated in their assigned program treatment(s). 
New measures. During the deliberative session, facilitators recorded participants’ priority 
problems and vote totals for their priority solutions on each group’s matrix-ranking grid. 
For this study, the evaluation team piloted new measures based on these matrix voting 
patterns. Several variables recorded how closely (or not) participants’ votes were 
distributed across all solutions or on a specific solution. Another variable recorded the 
proportion of votes cast for internal solutions (requiring citizen action) versus external 
solutions (requiring action by an MP or other official). These measures were used to 
understand how facilitator gender and group gender composition affected participant 
behavior and voting patterns during the deliberative session. The analysis of statistical 
significance was valuable in identifying gender as a factor in deliberative session outcomes; 
however, the meaning of these voting measures as indicators of deliberation is not entirely 
clear. These data confirm that creating a separate space for women to deliberate with other 
women leads to different outcomes than mixed-gender or all-male groups. It is less clear 
whether certain voting patterns are indicative of “better” deliberative outcomes. 
III. Summary of Key Findings 
A summary of key findings based on descriptive and experimental data collected during the 
impact study of constituency dialogues and deliberative sessions is presented below. 
Detailed descriptions of the analyses and discussions of findings for the qualitative and 
quantitative data sets are included in the companion reports referenced in annexes I and II. 
Constituency Dialogues 
Experimental findings 
Holding a CD had statistically significant positive effects for individuals in treatment 
villages on indicators of knowledge of the political process and self-reported engagement 
in civic and political behaviors. The effect of a CD on individual attitudes towards politics, 
CEPPS NDI Cambodia Impact Evaluation Report Page 22
the political process and citizens’ role in that process is not consistently positive or 
negative or statistically significant for most of the relevant survey indicators. There are 
some differences in how men and women responded to the CD across knowledge, attitudes 
and reported behavior indicators. 
CD effect on knowledge. The survey included open-ended and closed questions (that offered 
fixed or multiple choice options) about the roles and responsibilities of the National 
Assembly and MNAs and individuals’ right to talk to MNAs about their concerns. 
 The baseline data alone demonstrated the stark difference between men and 
women in political knowledge. Women had significantly lower survey scores than 
men, which would indicate that women in villages served by the CD program have, 
on average, much less political knowledge than men. 
 After a CD was held, respondents in treatment villages were significantly more likely 
than respondents in control villages to provide any response to various open-ended 
questions and to answer closed questions correctly. On several questions, the 
positive effect is either greater or exclusively among men. 
CD effect on attitudes. To assess individuals’ confidence in and attitudes towards the 
political process, the survey included a series of questions about respondents’ general 
interest in politics, their individual role in solving problems at the community and country 
levels, the role of opposition parties in a democracy, the meaning of voting in elections, and 
perceived MNA responsiveness to respondents’ communities. 
 Compared to individuals in control villages, respondents in CD villages were 5.6%
more likely to report being “very interested” in politics and 8.7% less likely to 
report being “not interested” in politics after the CD was conducted. 
 Across other attitudinal questions, there were not consistent and statistically 
significant effects of the CD, either positive or negative, on confidence in the political 
process or citizens’ role in that process. 
 Although respondents’ attitudes did not change overall, women in CD villages were 
more likely (than women in control villages) to agree with the following statements 
because of the CD: “It is good for democracy in Cambodia to have opposition 
parties,” and “I can choose who represents me in government.” 
 The CD caused survey respondents in treatment villages to be 27% more likely to 
agree that “MNAs communicate with [my] community about what they are doing in 
parliament”, and 15% more likely to report being aware of specific activities MNAs 
had conducted on behalf of their communities. 
CD effect on behavior. To assess the effect of a CD on behavior, the survey included a series 
of questions on self-reported membership in community associations and groups, 
CEPPS NDI Cambodia Impact Evaluation Report Page 23
engagement or willingness to engage in specific [legal] civic or political actions, and 
engagement or willingness to engage specific political actors or officials. 
 The CD had significant positive effects on treatment village respondents’ self￾reported membership to two out of the nine types of groups or neighborhood 
associations listed. Individuals living in CD villages were 6.3% more likely to report 
belonging to a political party and 9.5% more likely to report belonging to a “self￾help” group after the CD took place, compared to individuals in control villages. The 
latter effect was more pronounced for women (10.9%) than for men (8.3%). 
 A series of 12 civic and political actions were combined into an index for survey 
analysis. The index included actions other than voting, such as attending a political 
party event or commune council meeting, filing a case in court, or distributing 
information on political issues. On this index of citizen engagement, the CD caused 
an average of a 6.3% increase in the probability that respondents in CD treatment 
villages said they “already had” or “would” take some kind of action. The CD effect 
was significant and positive for 10 of the 12 specific actions, ranging from a 4% to 
15% increase in the likelihood that respondents said they “already had” or “would” 
engage in those behaviors. Results for some of the specific actions on this index are 
noted below.
 Among the commune-level actions, the CD caused a 4% increase in respondents 
reporting that they “already have” or “would” attend a commune council meeting. 
The findings are not significant for the related actions of speaking at a commune 
council meeting or filing a complaint with the commune council. The CD caused 
women, relative to men, to be more likely to report specific actions, including filing a 
case in court, participating in a peaceful demonstration, participating in a strike in 
the workplace, signing or thumb-printing a petition, and writing a letter or 
contacting an MNA. 
 Finally, the CD caused an increase in the frequency with which respondents 
reported talking about politics with other people, such as friends or neighbors. The 
CD did not have a significant effect, positive or negative, on whether respondents 
would have a hard time discussing controversial issues with friends or neighbors. 
Descriptive findings 
Opposition and ruling party supporters were both present at all nine CDs in generally 
balanced numbers. Participating MNAs respected the CD code of conduct most of the time. 
The CD environment varied by commune with a strong police presence observed at three 
of the nine CDs. At some of these CDs, police were perceived to be attempting to intimidate 
CD participants who raised concerns. 
CEPPS NDI Cambodia Impact Evaluation Report Page 24
Deliberative Sessions 
Experimental findings 
The deliberative session had few detectable effects on participant knowledge, attitudes, 
behavior, or priority issues as measured by the same survey questions summarized in the 
CD findings section. As noted in the data limitations discussion above, and in the 
companion quantitative report, these findings should be interpreted cautiously as fewer 
survey respondents acted on their deliberative session invitations than anticipated. There 
is some survey evidence that the deliberative sessions affected men and women differently, 
and sometimes in opposite ways. 
Deliberative session effect on knowledge. The survey did not detect any significant effects –
positive or negative – of the deliberative sessions on participant knowledge of the political 
process. There is one exception among women. 
 The deliberative sessions caused a 6.6% increase in the probability that women 
correctly responded that “giving gifts,” one of the options for a survey question, was 
not a responsibility of MNAs. 
Deliberative session effect on attitudes. The deliberative sessions had no effect on survey 
respondents’ reported interest in politics and no average effect across men and women on 
questions about confidence in the political process. However, there is some evidence that 
deliberative sessions affected attitudes among men and women differently. 
 Among men invited to attend a deliberative session, significantly fewer agreed with 
the statement, “None of the MNAs from my province are aware of issues facing the 
local people.” Among women, significantly fewer agreed with the statement, “It does 
not matter how I vote, nothing will change.” 
Deliberative session effect on behavior. The deliberative sessions had no average effects 
detected on the survey across men and women on self-reported civic and political 
behavior. There are some differences between men and women around specific behaviors. 
 Contrary to the evaluation team’s expectations, the deliberative sessions did not 
affect the frequency with which respondents reported talking about politics with 
other people, such as friends or neighbors, or whether respondents would have a 
hard time discussing controversial issues with friends or neighbors. 
 The deliberative sessions caused male respondents to be 2.3% more likely to join an 
NGO and 1.1% more likely to join a fisher’s association. These changes are 
statistically significant but are small shifts, and less meaningful, in actual practice.
 The deliberative sessions caused women to be 6.5% more likely to say they would 
write a letter or complaint to a government authority. While this change is not 
statistically significant, it could be a meaningful change in actual citizen practice.
CEPPS NDI Cambodia Impact Evaluation Report Page 25
Deliberative session effect on priority issues. The survey included a list of priority issues 
from which respondents could choose the top three issues they personally considered most 
important. The deliberative session had no significant effects on individuals’ priority issues, 
as detected in the survey. 
Descriptive findings 
Systematic observation at the CD showed that deliberative session participants were more 
active at the CD than other CD attendees. This observational data compares deliberative 
session participants to other CD-only participants. The deliberative session participants 
included some individuals who were not randomly selected to be invited to these sessions, 
and this data includes many self-selected individuals; therefore, these are descriptive 
findings. Across all deliberative sessions, 65.7% of participants attended a CD the following 
day. Among these individuals, deliberative session participants raised their hands and 
spoke at a CD more than twice as often as CD participants who had not attended a 
deliberative session. This was true for both male and female deliberative session 
participants: women spoke at a CD one and a half times more often than CD-only female 
participants; and men spoke two and a half times more often than CD-only male 
participants. 
Further descriptive findings provide context to the priority issues identified by deliberative 
session groups – many of which are similar to, but ranked somewhat differently than those 
listed on the survey. These findings are based on descriptive qualitative data from 80 
deliberative sessions and nine CDs conducted during this study. 
 The issues identified through the deliberative session group ranking process 
highlighted the interrelated nature of problems faced by the participants. The 
groups’ discussions demonstrated that the problems raised had wide-reaching 
effects on their families, community politics, the local economy, and the natural 
environment. 
 The priority issues identified through the deliberative sessions varied by province 
and commune; however, corruption was a common theme across communes. The 
corruption experienced by participants took many forms, affecting service delivery, 
distribution of humanitarian aid, and land concessions. Land seizure was acutely 
noted in a third of the communes. Participants associated this issue with other 
problems, including dispossession of land and infringement of legal rights, 
detrimental environmental and health effects, and violent retribution or 
imprisonment for citizens who mobilized in protest. 
Deliberative Session Gender Dynamics
Experimental findings 
Experimental survey data were used to examine the effects of deliberative session gender 
variations on participant attitudes, knowledge and behavior. Experimental matrix-ranking 
CEPPS NDI Cambodia Impact Evaluation Report Page 26
data were used to examine the effects of facilitator gender and group composition on 
priority problems and solution ranking. 
Deliberative session participant attitudes and knowledge. The deliberative session 
facilitator’s gender and the gender composition of the group had few detectable effects on 
participant responses to survey questions about their knowledge of, attitudes toward, and 
participation in political and civic life. This analysis is limited to the participants who 
attended the deliberative session and could be matched to a pre- and post-survey. The 
significant findings could be due to chance rather than an effect caused by the program. 
Deliberative session participant behavior. There are few survey findings on deliberative 
session effect on respondents’ reported behavior. For example, participants in all-female 
groups were more likely (than participants in other group gender compositions) to report a 
willingness to vote in the 2012 commune council elections. Again, these findings should be 
interpreted cautiously due to the small percentage of surveyed individuals who accepted 
the deliberative session invitation and were considered in this analysis. 
The evaluation team considered the deliberative session voting results recorded on the 
group matrices as an observed behavior among deliberative session participants. These 
effects are based on measures piloted in this study, which suggest that the facilitator 
gender and group composition affected participant behavior. While the findings are 
statistically significant for gender, understanding their meaning for measuring deliberation 
– whether positive or negative – requires further study.
 Facilitator gender affected how deliberative session groups distributed votes across 
their matrices. Male-facilitated groups were significantly more likely than female￾facilitated groups to vote in consensus, “clustering” their votes for single solutions 
as well as among their top three out of six priority solutions. 
Deliberative session participant priority issues. As noted above, the deliberative session had 
no significant effects on individuals’ priority issues, as detected in the survey. However, 
some differences were detected in the analysis of deliberative session qualitative data and 
matrix voting patterns. 
 The gender composition influenced how groups ranked their priority problems. All￾male groups placed a higher priority on the lack of infrastructure and were less 
likely (compared to other group types) to rank corruption as one of the top three 
priorities. Mixed-gender groups were more likely than other group types to identify 
a lack of health clinics among the top three priorities. 
 The facilitator gender also caused a difference in how deliberative groups ranked 
their priority problems. Male-facilitated groups were more likely than female￾facilitated groups to vote for lack of infrastructure. They were less likely than 
female-facilitated groups to vote for high cost of living and access to markets among 
the top priority issues. 
CEPPS NDI Cambodia Impact Evaluation Report Page 27
 Facilitator gender also affected the types of solutions for which groups voted. All 
groups tended to identify external solutions – those that required MNA action – over 
internal solutions – those that required citizen action. However, male-facilitated 
groups were significantly more likely to prioritize external solutions compared to 
the other group composition types. Female-facilitated groups were significantly 
more likely to prioritize internal solutions compared to other group types. 
IV. Conclusions
The findings from NDI’s mixed-methods impact evaluation will contribute to the current 
CD program and to similar future programming in Cambodia and other countries. The 
findings of this study are also relevant to ongoing program design and evaluation dialogue 
within the broader community of international democracy assistance practitioners, donor 
agencies and academics. Conclusions are presented below, first those related to the 
program followed by considerations for future evaluation initiatives. 
Program Design and Implementation
 This study showed that a CD caused positive change at the individual knowledge levels 
and self-reported engagement in some civic and political behaviors, but did not detect 
evidence of the program causing greater citizen demand of their MNAs. 
The survey data demonstrated that a CD caused an increase in basic civic and political 
knowledge among individuals, which is the first step in the results chain examined by this 
study. However, the evaluation team did not detect evidence that individuals were more 
likely to put pressure on MNAs to address their concerns, which is the subsequent step in 
the results chain. Lack of evidence does not necessarily mean that there is not a link 
between the CD program and citizen demand in Cambodia. Among others, certain factors 
merit further study to understand the linkage between citizens’ increased political 
knowledge and awareness and more active citizen demand for MNA responsiveness, 
including: 
a. expanding the data collection timeline beyond the period immediately following a 
CD to allow time for medium- or longer-term changes in citizen behavior to take 
place and to assess the sustainability of effects; 
b. identifying measures of observed, rather than self-reported behavior change, to 
study outcomes in actual practice (see the evaluation conclusion on behavior 
indicators below); 
c. determining the “dosage” – or the quantity of program events– needed to affect 
citizen demand beyond a single CD or deliberative session; and
d. exploring additional program support for citizen engagement, such as the addition 
of an explicit civic education or citizen participation component to the CD. 
 The deliberative session pilot effort generated reliable qualitative data within a 
randomized impact evaluation and demonstrated the differences in men and women’s 
voices as well as the risks Cambodians face in speaking out about issues like corruption. 
CEPPS NDI Cambodia Impact Evaluation Report Page 28
While the experimental survey data did not detect any changes caused by the deliberative 
sessions, the experimental matrix voting pattern data did show that the facilitator’s gender 
and group gender composition caused groups to reach different outcomes. These findings 
underscore the differences in men and women’s voices. These discussions also 
demonstrated the risks that Cambodians face in speaking out about issues like corruption. 
A single deliberative session is unlikely to support sustainable solutions to the problems 
discussed and could even put participants at risk by encouraging the discussion of such 
topics. Rather than a stand-alone program activity, deliberative sessions should be 
considered part of a broader programmatic approach to support citizen engagement and 
demand for greater responsiveness and accountability of their elected officials. 
 The mixed methods approach of this impact study highlighted positive changes in 
individuals’ knowledge caused by the program as well as contextual challenges and 
risks of greater citizen engagement in Cambodia. 
Together, the different data sets provide a fuller picture of citizen engagement in the 
context of this program than any of the data sets do alone. Descriptive observational data 
from the nine CDs showed that the participating MNAs generally adhered to the code of 
conduct, and that the CDs were inclusive of both the ruling and opposition party MNAs and 
their supporters in the audience. Experimental survey data showed that the CD caused a 
positive effect on individuals’ knowledge; however, this data did not show a clear or 
statistically significant effect of the CD or the deliberative session on individuals’ 
confidence in the political process or system. The descriptive deliberative session data 
highlighted participants’ experiences – beyond the context of the CD – with intimidation, 
corruption, land seizure, and sometimes violent retribution against individuals who spoke 
out against local problems. These findings point to the need for further study of the 
conditions related to and consequences of greater citizen demand for individuals in a 
closed political system. 
 The scope of this evaluation did not include the “supply” side of the program theory of 
change, and many questions remain regarding how the program may change MNA 
behavior. 
Since the CD program began in 2004, most of the programmatic inputs (prior to the 
introduction of the deliberative sessions) have focused on MNAs and the supply side of the 
theory of change. The program has collected descriptive qualitative data that demonstrate 
anecdotes of both citizen demands for action and MNA initiatives to address citizen 
demands. Similar to the questions noted above about citizen demand, more systematic 
information is needed to understand the extent to which the program contributes to or 
causes positive or negative change in MNAs’ perception of political competition and their 
understanding of constituents’ needs. There are challenges to studying the program effect 
on MNAs. In the Cambodian context, it would not be feasible to randomize MNA 
participation in the program or even to randomly assign MNAs to different phases of a 
delayed program roll-out. Non-experimental data could help to explore programmatic 
contribution to MNA’s understanding of and action on constituent needs. 
CEPPS NDI Cambodia Impact Evaluation Report Page 29
Evaluation Design and Implementation 
 The academic-practitioner partnership strengthened the evaluation team and the 
quality of the impact study, and this model should be considered when feasible for 
future impact studies in the democracy and governance sector. 
This partnership merged practitioner and academic expertise on equal footing, which 
enabled the evaluation team to mitigate anticipated risks and effectively troubleshoot 
challenges that arose during implementation and to collect and analyze a range of 
quantitative and qualitative data sets. The practitioners’ relationships and country 
experience proved critical for understanding the political and programmatic contexts. This 
perspective served to represent the original program purpose in the midst of evolving 
evaluation design options and to inform the development of data collection tools and their 
translation into Khmer. The academic perspective was invaluable in exploring evaluation 
design options and preserving the rigor of the impact evaluation design as challenges arose 
during implementation. This expertise was essential for exploring analysis options for the 
complex body of quantitative and qualitative data sets. Finally, the extension of the 
academic-practitioner partnership into the analysis phase of the evaluation ensured that 
evaluative findings have utility for a broader audience, including program implementers, 
policymakers as well as the academic community. 
 The evaluation team considered various indicators to measure change in individual’s 
behavior, but few of these measures proved viable in practice. 
This evaluation included several measures of individual behavior, including survey 
questions on participant-reported action or intent of future action, and systematic 
observation of deliberative session participants’ behavior at the CD. The social and political 
context and practices rendered other measures unfeasible or potentially harmful to 
participants. The evaluation team explored the use of commune council meeting records 
and other means of tracking individual actions (letter campaigns, citizen petitions, SMS 
messages, citizen visits to an MNA’s office) in the months following the deliberative 
sessions and CDs. Commune council meeting records are not regularly maintained nor 
made publicly available. MNA records of constituent contact are not systematically 
maintained, if they exist at all. Citizen action through written formats would exclude large 
parts of the target program population, which has high illiteracy rates. The more visible 
measures of citizen behavior expose individuals to scrutiny and possible retaliation from 
government or other state officials. These measurement challenges were shaped by the 
Cambodian political context and are likely to be encountered across most democracy and 
governance programs. 
 The deliberative session matrix-ranking measures were effective in capturing reliable 
qualitative data within a randomized impact evaluation. However, collecting and 
analyzing this qualitative data may be cost prohibitive for many programs. 
The introduction of the deliberative session was a valuable pilot effort both for this impact 
evaluation and to assess a potential citizen engagement program activity. As noted above in 
CEPPS NDI Cambodia Impact Evaluation Report Page 30
the programmatic conclusions, deliberative sessions should not be considered as stand￾alone or one-time program activities. The volume of qualitative data collected and analyzed 
through the 80 deliberative sessions conducted for this study required significant staff and 
program resources. These costs should be considered against the broader program design 
and available resources and may be prohibitive for many programs to absorb. Future 
program design and evaluation work around this program component should also take into 
consideration challenges faced with high levels of participant attrition. The study was 
limited by invited participants not acting on the deliberative session invitation as 
frequently as anticipated. The participant attrition resulted in a diminished sample size for 
analysis, which may not have detected actual effects. 
ANNEX I to CEPPS NDI 
Cambodia Impact 
Evaluation Report
Evaluation Report: The Constituency Dialogue 
Program in Cambodia
Susan Hyde, Associate Professor, Department of Political Science, Yale University
Feburary 2013

ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-1 
Evaluation Report: The Constituency Dialogue Program 
in Cambodia1
February 11, 2013 
Susan Hyde 
Associate Professor 
Department of Political Science 
Yale University 
Susan.hyde@yale.edu
 
1 The views expressed represent only the views of the author, and do not reflect the views of NDI or its staff. 
The author served as a pro-bono consultant on the project, acting as the principal investigator during the 
implementation and analysis phases in exchange for access to the data. Travel to Cambodia, lodging, and per 
diem were paid for the researcher by NDI. Molly Watts and Emily Lamb served as field researchers and 
provided excellent support in implementation and analysis of this study. All errors and omissions are the sole 
responsibility of the author. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-2 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-3 
Table of Contents 
I. Executive Summary .......................................................................................................................................... 4
II. Project Description .......................................................................................................................................... 5
III. Existing Studies of Democracy Promotion and Deliberative Democracy ................................. 7
IV. Experimental Design(s) ............................................................................................................................... 8
V. Quantitative Data .......................................................................................................................................... 14
VI. Quantitative Results: Effects of CD and Deliberative Sessions on Knowledge, Attitudes 
and Behavior ........................................................................................................................................................ 19
VII. Quantitative Results: Effects of CD and Deliberative Sessions on Priority Issues ............ 23
VIII. Quantitative Results: Effects of Deliberative Session Group Gender Composition and 
Facilitator Gender on Knowledge, Attitudes, and Behavior .............................................................. 24
IX. Discussion of Findings ............................................................................................................................... 25
Works Cited .......................................................................................................................................................... 38
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-4 
I. Executive Summary 
Cambodia has held periodic multiparty elections since 1993 but is still considered 
an electoral authoritarian regime. The Cambodian People’s Party (CPP), led by Prime 
Minister Hun Sen, dominates Cambodian political life.2 Cambodia is one of the poorest 
countries in the Asian region, with a per capita gross national income of USD $2,230 in 
2011. The effects of the civil war, Cambodian genocide (1975-1979) and Vietnamese 
occupation are still evident. 
Within this context, the constituency dialogue (CD) program in Cambodia is a multi￾year democracy promotion effort implemented by the National Democratic Institute (NDI) 
which brings elected members of the National Assembly (MNAs) to rural villages in their 
province in order to participate in a multi-party town hall meeting. The evaluation team 
conducted a randomized evaluation of the program to better understand some of the 
effects of participation in a CD, as well as the effects of small group meetings, or 
deliberative sessions, held before the CD,. This document provides a detailed description of 
the study design and the quantitative results, primarily based on survey evidence but also 
including quantitative evidence collected during the deliberative sessions. In a companion 
document authored by Linda Stern of NDI, the qualitative results from the same study are 
presented. 
The central questions in the study focus on how rural Cambodians were influenced 
by attending a CD, as well as how participants were influenced by an additional 
deliberative session held before the CD on problems facing their community and possible 
solutions to those problems. Before this evaluation, program staff had little systematic 
knowledge about the population of citizens that are typically served by the program, nor 
strong predictions about the effects the program has on citizens.3 This study focused on the 
hypothesis that if rural citizens attended a CD, their political knowledge, attitudes toward 
democracy, and reported political behavior would change in a manner that would make 
them more aware of democratic institutions, more willing to discuss issues facing their 
community, more aware of their rights as citizens, and by extension, more likely to hold 
their elected officials accountable. 
The second component of this study was the addition of deliberative sessions held 
the day before the CD took place, where a small group of participants discussed problems 
facing their community, proposed solutions to those problems, and voted on their 
preferred solution to each of three priority problems within their group. These groups 
were varied by gender composition of participants and facilitator gender. This component 
of the study explored the general expectations that the deliberative sessions would also 
influence attitudes, knowledge, and behavior, and have effects on what citizens outline as 
priority issues for their community. NDI also was interested in whether the effects of the 
program varied by gender. 
In interpreting these results, it is important to note that the evaluation focused on 
change among citizens and did not test the effect of the program on MNA behavior. In this 
 
2 McCargo 2005. 
3 Prior to the evaluation, program staff had stronger beliefs about the likely effect of the CD program on 
participating MNAs. In planning the evaluation design, program staff articulated a theory of change that 
encompassed this belief as well as the “bottom up” effect of changing citizen demands on MNAs. This is 
described in greater detail in the companion qualitative report and summary report. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-5 
case, MNAs were not randomly assigned to participate in a CD event for a variety of 
logistical reasons; therefore, the evaluation focused primarily on citizen-level or “bottom￾up” change.4 Citizens in treatment and control villages were surveyed before and after the 
CD. Quantitative data collected during the deliberative session voting processes were also 
used to test for facilitator and group composition gender effects on the outcomes of the 
deliberative process. Overall the study found: 
1. Participation in a CD had positive and significant effects on citizen knowledge of the 
political process, self-reported engagement with the political process, familiarity 
with politics, and perceptions of MNA responsiveness. 
2. Citizens exposed to the CD were also more likely to have reported voting in the last 
election, more likely to say they would vote in the next election, and more likely to 
say they belonged to a political party or “self-help” group. 
3. Contrary to expectations, the CD does not consistently change citizen confidence in 
the political process. 
4. The deliberative sessions overall had few detectable effects on knowledge, 
attitudes, and behavior, as measured in the average effects on survey responses. 
5. There is some evidence that men and women were affected differently by the 
additional deliberative sessions. 
6. The gender compositions of the deliberative group’s participants and the 
facilitator’s gender have consistent effects on deliberative session voting behavior. 
Note that null effects should be interpreted with care, as they indicate that the 
study, as designed and implemented, did not detect consistent effects of the intervention. 
Null results can occur because there is no effect of the intervention; however they can also 
occur for a variety of other reasons, such as problems with the implementation of the 
study, or if the effects of the intervention were not captured by measures used. 
II. Project Description 
This document summarizes and presents the results of a study of the constituency 
dialogue program in Cambodia, a multi-year democracy promotion effort implemented by 
NDI that brings elected members of the National Assembly to rural villages in their 
province for a multi-party town hall meeting. The CD events are open to the public and are 
intended to provide an opportunity for citizens to learn more about multi-party democracy, 
voice their concerns to their MNAs, and request that actions be taken to resolve specific 
problems. Each event is attended by MNAs from two or more political parties and is 
moderated by an NDI staff member. The events take place in rural villages that are rarely 
visited by national level officials. CDs are a major event in the community and are typically 
attended by 400-1,000 residents. 
At each CD, an NDI staff member gives a short introduction in which they make brief 
remarks about multiparty democracy, the importance of open discourse, and the roles and 
 
4 Note that as part of the evaluation, NDI’s local partner non-governmental organizations (NGOs) did collect 
data on whether MNAs had followed up on any of their promises in both the treatment and control villages. 
But, because MNAs were not randomly assigned to participate in the CD event, the causal effect of the CD on 
MNA behavior is difficult to tease out. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-6 
responsibilities of the MNAs. Each MNA is given a set speaking time at the beginning and 
end of the CD. After the MNAs’ opening remarks, the floor is open to questions from anyone 
in the community. The forum thus provides a rare opportunity for policy debate and 
showcases the distinctions between legislators from multiple political parties and their 
approaches to both local and national concerns. In the context of Cambodian politics, it is 
an extremely rare opportunity for opposition parties to appear on more or less equal 
footing with the dominant Cambodian People’s Party. MNAs often use the opportunity to 
update citizens on the activities of the legislature and government and provide other 
information relevant to their constituents. The dialogues encourage two-way 
communication and are unscripted, allowing for sometimes challenging questions and 
demands from citizens. 
The study, as a whole, planned to address eight specific evaluation questions: 
(1) Does exposure to the CD change individual attitudes, knowledge, and reported or 
observed behavior? Are men and women affected differently participation in the CD? 
(2) Does participation in a deliberative session before the CD change individual attitudes, 
knowledge, and reported or observed behavior? 
(3) How does participation in the deliberative sessions before the CD differentially impact 
individuals’ priority issues? 
(4) Does the gender composition of the deliberative sessions differentially impact 
participant attitudes, knowledge, and reported or observed behavior? 
(5) Does the gender composition of the deliberative group differentially impact the priority 
issues of participants? 
(6) Does the gender of the deliberative session facilitator differentially impact participant 
attitudes, knowledge, and reported or observed behavior? 
(7) Does the gender of the deliberative session facilitator differentially influence the 
priority issues of participants? 
(8) Do individual deliberative session facilitators differentially impact the priorities of 
participants? 
The evaluation of this program consists of several components, and the study relied 
on three separate randomizations and two datasets which are described in greater detail 
below. Overall, the study can be divided into three parts based on the randomized 
treatments: 
Part A: Constituency Dialogue: Through a random selection process, one village received 
the CD treatment out of a pair of villages from the same district. Data include pre￾and post-treatment survey data for six village pairs (twelve total villages), as well as 
qualitative observation of actions in the treatment and control villages. This 
randomization addresses question (1) above. 
Part B: Pre-CD Deliberative Sessions: Within the CD treatment villages, half of pre￾treatment survey participants were randomly invited to participate in an additional 
deliberative session the day before the CD event. Using an invitation with a unique 
identifying number that respondents were asked to bring with them to the 
deliberative session, the evaluation team attempted to match whether an individual 
participated in the deliberative session with that individual’s survey responses. This 
component of the study addresses questions (2)-(3); although issues with 
implementation, including difficulty in matching survey participants with the 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-7 
invitation numbers and lower than expected acceptance rates of the invitation, 
present challenges. 
Part C: Deliberative Group Composition and Facilitators: The evaluation team 
randomly assigned individuals at the deliberative sessions to small groups of three 
to five people such that groups consisted of only women, only men, or both. Women￾only groups were facilitated by female facilitators, men-only groups were facilitated 
by male facilitators, and mixed groups were facilitated by either a male or female. 
This final level of randomization addresses questions (4)-(8). Deliberative session 
facilitators recorded information on the substance of the group discussion, including 
lists generated by the groups of priority issues facing the community, solutions 
posed by the group to the top three problems, and participant voting data taken 
from a 3x6 matrix of problems and solutions intended to capture the deliberation. 
Deliberative session facilitators discretely observed their group participants at the 
CD event the next day to determine whether these individuals were more or less 
likely than the average CD participant to raise their hand or to ask a question of the 
MNAs. 
The next section of this report provides a very brief overview of academic work 
relevant to this study. The remainder of the document summarizes the quantitative 
findings from the survey about the effects of the CD and the effects of the deliberative 
session on citizen knowledge, attitudes and behavior; priority issues identified by citizens 
on the survey; and voting behavior during the deliberative sessions. 
III. Existing Studies of Democracy Promotion and Deliberative Democracy 
Although this report evaluates only one part of a broader democracy promotion 
program, it is relevant to several larger questions of interest to academics in international 
relations and comparative politics, as well as to policymakers engaged in democracy 
promotion. Can local-level pro-democracy change be spurred through the intervention of 
outside actors? Can meaningful change in political attitudes be induced in the short run? 
How malleable are the aspects of political culture associated with democratization? What 
are the effects of deliberation in a non-democratic context? This study does not attempt to 
answer these questions definitively, but it does provide rigorous field experimental 
evidence that speaks to these questions. 
This study joins a growing body of research that relies on field experimental 
methods to study policy relevant questions.5 Although the project is motivated by an 
interest in understanding the effects of democracy promotion efforts, there are no known 
studies that have used field experimental methods to evaluate an analogous type of 
international democracy promotion program, particularly in an electoral authoritarian 
context such as Cambodia.6 Existing studies on democracy promotion are diverse, and 
 
5 Humphreys, Masters, and Sandbu 2006; Humphreys and Weinstein 2009; Wantchekon 2003; Olken 2007; 
Hyde 2010; Paluck and Green 2009; Fearon, Humphreys, and Weinstein 2009; Beath, Christia, and 
Enikolopov 2012. 
6 Leonard Wantchekon’s study of town hall meetings introduced as a campaign tactic to political campaigns in 
Benin is the closest, but is different in that his study is not of a foreign democracy promotion program and the 
town hall meetings are campaign-related (Wantchekon 2011). The CD town hall meetings are not campaign 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-8 
many rely on cross-national and case-study evidence.7 Finkel et al.’s study of the effects of 
US foreign assistance on democratization found that foreign aid can have a positive effect 
on levels of democratization, but the cross-national nature of the study makes it very 
difficult to eliminate the possibility that selection concerns or omitted variable bias can be 
fully addressed. Scholars including Thomas Carothers and Michal McFaul provide 
numerous anecdotal examples of successes in democracy promotion, while also 
highlighting examples of the limits of democracy promotion efforts.8
The National Research Council (NRC) report on Improving Democracy Assistance
highlighted the shortcomings of existing research and tools for evaluation of democracy 
promotion programs and called for organizations that engage in democracy promotion to 
use more rigorous methods of evaluation, as many democracy promotion programs should, 
in theory, have short-term micro-level consequences that can be measured.9 This study 
joins a handful of studies that are a response to the call made in the NRC report and 
represents one of the first pilot impact evaluations initiated because of the report.10
Much of the existing field experimental research relevant to democracy and 
governance interventions does not study democracy promotion specifically.11 Rather, as 
Devra Mohler summarizes, academics have examined the political consequences of a 
number of interventions that are relevant to good governance or democratization – and
could therefore be linked to a democracy promotion agenda – but that are not evaluations 
of existing democracy promotion programs. There are important exceptions to this 
generalization, including several studies that are not yet published, but there are still only a 
small number of studies that can identify the causal effects of international democracy 
promotion efforts. 
IV. Experimental Design(s) 
In order to test the proposed program evaluation questions, the evaluation team 
designed and implemented several randomized components of the study and collected 
several types of quantitative data. In the following sections, this report presents detailed 
evidence relevant to many of the research questions outlined above. Overall, the report 
documents a number of positive effects on citizen knowledge, attitudes, and behavior as a 
result of exposure to the CD event (question 1), some of which vary by gender. The effect of 
the CD on citizen confidence in the political process, however, is weak. 
Consistent effects of the deliberative sessions (questions 2-3) were not detected in 
the survey data, with several exceptions detailed below. There are also not consistent 
effects of the deliberative session’s group composition or facilitator gender on reported 
 
events and are attended by members of multiple political parties. For the deliberative session intervention, an 
important motivating study is Humphreys, Masters, and Sandbu (2006). 
7 Brown 2005; Bush 2011; Carothers 1997a; Carothers 2004; Carothers 2006; Gillespie and Youngs 2002; 
Goldsmith 2008; Knack 2004; Ottaway and Carothers 2000; Schraeder 2003; Youngs 2001; Crawford 2001; 
Finkel, Pérez Liñan, and Seligson 2007; Finkel et al. 2006; National Research Council 2008; and Pevehouse 
2002.
8 Carothers 1997a; Carothers 2004; Carothers 2006; Ottaway and Carothers 2000; Carothers 1997b; McFaul 
2004; and McFaul 2009. 
9 National Research Council 2008. 
10 For an excellent summary of relevant work, see Moehler 2010. 
11 Ibid. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-9 
priority issues for MNAs, as measured in the survey (questions 5, 7, and 8).12 Using data 
collected from the deliberative sessions, there are significant differences found in voting 
patterns based on group gender composition, facilitator gender, and individual facilitator 
(questions 4 and 6). 
The absence of evidence regarding effects of the deliberative sessions on priority 
issues and on knowledge, attitudes, and behavior as reflected in the survey data could 
indicate that the deliberative sessions have no effect on knowledge, attitudes, behavior, or 
priority issues. However, it could also be due to lower than expected compliance with the 
deliberative session treatment and difficulty in matching the survey data with participation 
in the deliberative sessions. Therefore, it is important to keep in mind that the survey data 
represents only a subset of evidence on the deliberative sessions, and the deliberative 
sessions should also be evaluated using the qualitative data collected during the 
deliberative sessions (see companion report). 
Constituency Dialogue Randomization 
First, for analysis of the CD, randomization was introduced to the process by which 
villages were selected to receive a CD event. For a variety of reasons, NDI (in consultation 
with USAID) determined that national or regional random assignment of the CD to villages 
was not feasible. During the normal course of the CD program, and outside of the context of 
this study, villages are selected for the CD event by a local Cambodian NGO that is already 
active in the relevant province. This decision is delegated to NDI’s local partners, who 
consider a number of factors based on their localized knowledge of villages within the 
province. Typical criteria for a town hall meeting event include finding villages that are 
small but accessible by road (rather than boat or footpath), as the goal is to serve rural 
areas; and that have some political tension but neither too much or too little political 
tension in the month before the CD. They also look for villages that have local police or local 
officials who are not hostile to the program, primarily because local officials have the 
ability to prevent a CD from occurring or pressuring villagers not to attend. NDI views the 
cooperation of its local partners as essential to the program. 
As a result, NDI’s field staff determined that this delegated selection process was 
essential to the program, and abandoning it in favor of NDI-mandated (randomized) 
selection of villages would compromise the program. The evaluation team instead came up 
with a method to introduce randomization into the process of assigning the CD events to 
villages while staying true to the delegated village selection process and the timeline in 
which villages are only vetted and suggested a minimum of one month before the CD event. 
The evaluation team requested that the local NGO choose two candidate villages for each 
potential CD. We also asked the local NGO to ensure that the two villages selected as 
candidates for the CD were sufficiently far apart as to make travel between them on the day 
of the town hall meeting unlikely, but that otherwise met all of their criteria for selecting 
villages. Thus, for each planned CD event, the local NGO used their normal criteria to select 
two villages (rather than the usual one village) within each target district that would be 
otherwise equal candidates for a town hall event and similar on variables of interest.13
 
12 One future line of inquiry is the comparison between individual survey priorities, group-level priorities 
after the deliberative sessions, and CD level priorities, as discussed during the deliberative sessions. 
13 This design is also partly a result of a challenge that arose during implementation, which led to a 
modification of the original plan for randomization at the village level. The original plan was for five villages 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-10
After the two village names were forwarded to NDI’s offices in Phnom Penh, a field 
staff member (with one witness) flipped a coin to determine which village would receive 
the CD “treatment” and which village would go untreated. NDI and its local partners 
selected 18 villages in this manner, creating nine pairs of villages, each containing one CD 
treatment village and one control village. From these 18 villages, the CD analysis drew on 
pre- and post-test survey data from only 12 villages, or six village pairs. This is due to the 
fact that the final three control villages selected did not receive the survey.14
Figure 1: Sample Selection Process for CD Analysis 
 
to be proposed simultaneously in each province, three of which would then be randomly selected to receive 
treatment. It was eventually discovered that the local partners could not propose all five villages in a province 
simultaneously, and the village-pair procedure was adopted instead. 
14 Prior to establishing the coin toss protocol to randomize village selection within village pairs, this study 
planned to randomly select three treatment villages out of five villages proposed by the local NGO partners. 
This initial plan would have used all fifteen villages surveyed in the analysis of the CD and deliberative 
session programs. When it was discovered in the field that the local NGOs could not choose all five villages 
simultaneously, the plan had to be altered. Logistically, the survey plan could not be adjusted to include all 
village pairs. All 18 villages received the qualitative follow up monitoring on activities in the village. 
Three Cambodian provinces selected for study: Kampong Cham, Kratie, and Kampong Chhnang Remaining districts excluded from study Two districts within each province (21 districts) selected to receive CD event (6 districts) Two villages selected as candidates to receive CD event
in each district (12 villages) All other villages excluded from study 6 villages assigned to treatment (one within each pair) 6 villages assigned to control (one within each pair) Baseline survey conducted for 120 respondents in each village (720 total respondents) Follow-up survey conducted for 661/720 (92%) baseline survey respondents Follow-up survey conducted for 655/720 (91%) baseline survey respondents
Baseline survey conducted for 120 respondents in each village (720 total respondents)
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-11
Figure 2: Sample Selection Process for Deliberative Session Analysis 
Deliberative Session Randomization and Quantitative Data 
The second component of this study was the addition of small-group deliberative 
sessions held the day before the CD took place. At each session, participants were divided 
into groups of 3-6 individuals and one facilitator, and discussed problems facing their 
community and their proposed solutions to those problems. Each group voted on their 
preferred solutions to each of three priority problems. 
The evaluation team used the surveys in CD treatment villages to randomize 
deliberative session participation. Within CD villages, half of the pre-treatment survey 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-12
respondents were randomly invited to participate in the deliberative sessions while the 
rest of the respondents were invited to participate in the CD only. About half of all invited 
participants actually attended the deliberative sessions, which was a lower compliance rate 
than anticipated and made it less likely that effects of the deliberative session could be 
detected in the survey data. Evaluation of the deliberative sessions included data from nine 
villages in which CDs were held, six of which overlap with the CD analysis. The deliberative 
session analysis compares individuals who were invited to participate in both the 
deliberative sessions and the CD to those who were invited to participate in the CD only; all 
individuals reside in the same village, and the comparison should be thought of as within￾villages. 
In contrast, the CD analysis compares survey respondents in villages that were 
selected to receive the CD with survey respondents in villages that were not selected to 
receive the CD. Thus, in the CD analysis the individuals compared are from two different 
villages in the same district. Thus, the populations for the CD and the deliberative session + 
CD survey analyses are not identical. 
Randomization of Group Composition and Facilitators at Deliberative Sessions 
The third level of randomization took place once participants arrived at the 
deliberative session, when the evaluation team randomly assigned participants to small 
groups (3-6 individuals) of all men, and all women, or mixed gender. Each group was then 
randomly assigned a facilitator conditional on the group type (all male groups could only 
have a male facilitator, and all female groups could only have a female facilitator). There 
are several types of quantitative data used in this part of the analysis. 
First, using the deliberative session invitation numbers, the information about 
group gender composition and facilitator gender was matched to the survey data. However, 
there are two important details to note. First, this matching could only be completed for 
surveyed individuals who actually participated in the deliberative session. Second, 
individuals who participated in the deliberative session and took the pre- and post-test 
survey could only be matched to their survey responses if they brought their invitation 
number with them to the deliberative session. There were 540 individuals invited to 
participate in the deliberative sessions, and 499 of these individuals completed the post￾test survey. Using invitation numbers collected at the deliberative sessions, 241 individuals 
were successfully (and anonymously) matched to the survey data. We suspect that a small 
portion of these “matches” are likely errors resulting from respondents giving away their 
invitations and/or lying about their participation in the deliberative sessions and the 
survey. 
After participants were randomly assigned to their small groups, the facilitator led 
them through a scripted process to brainstorm problems or challenges facing their 
community. To accommodate illiterate participants, the facilitators used pictures to 
represent each problem from the brainstorming session. Each participant then used 15 
chips (buttons) to distribute across any combination of up to three problems, represented 
by the pictures. Using this voting process, each deliberative group identified the top three 
problems among those discussed. Additional quantitative data were collected during the 
voting at the deliberative sessions, as described below. Whereas the survey data could only 
be matched to a subset of deliberative session participants, voting data were collected for 
all deliberative group participants. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-13
Figure 3: Sample Selection Process among Deliberative Session Attendees15
Once each group identified their top three problems, the facilitator placed the 
corresponding pictures on a large 3x6 paper matrix, visible to all participants, and asked 
participants to brainstorm, deliberate, and agree on one “internal” solution to each 
problem (what citizens could do to help solve that problem) and one “external” solution to 
each problem (what an MNA or other outsider could do to solve the problem). Finally, 
citizens were asked to vote for their favored solution, casting votes for only one problem at 
a time. For problem A, for example, citizens could vote for any of the six solutions, and 
could distribute their 15 chips (in any combination) across all six possible solutions, even 
those that were not originally suggested as solutions to Problem A. The same process was 
then repeated for the other two problems. These voting data were recorded by the 
facilitator. 
Table 1: Deliberative Session Matrix 
Solution 1 
internal
Solution 1 
external
Solution 2 
internal
Solution 2 
external
Solution 3 
internal
Solution 3 
external
Problem A
Problem B
Problem C
 
Thus, in theory, for each deliberative group containing n participants, there were 
15n votes cast for each of three problems. In practice, there were some slight variations in 
the total number of votes recorded. These errors could be the result of participants leaving 
early, failing to cast all their votes, or a data entry error by the facilitator. To evaluate 
whether the group type, facilitator gender, or bias from specific facilitators influenced the 
behavior of deliberative session participants, four calculations were made for each 
deliberative group based on the matrix voting data. These measures were developed after 
 
15 Note that two of the all-female groups were excluded from the analysis because on one, too many 
participants left during the deliberative session, and in the second, the facilitator made errors that invalidated 
the exercise. 
Attended deliberative session
(322 total participants) Assigned to female group w/
female faclitator (24 groups) Assigned to male group w/ male facilitator (24 groups) Assigned to mixed group
(32 groups) Assigned to female facilitator (16 groups) Assigned to male facilitator (16 groups)
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-14
the data were collected, and the evaluation team did not have a clear ex ante hypothesis 
about how these measures would vary across deliberation session group composition and 
facilitator gender. However, these measures are expected to indicate whether group 
composition type and/or facilitator gender had any measurable effects on group voting 
patterns. Please refer to the companion qualitative report for additional discussion of these 
data. 
The first aims to measure the degree to which participants voted for the solutions 
originally proposed for specific problems. This variable is called On-Diagonal Voting and 
records the percentage of all votes cast in the shaded cells in Table 1. A high score would 
mean that participants generally voted for one of the two originally proposed solutions. A 
low score would mean that participants voted for solutions other than the two that the 
group originally proposed for each solution. 
The second variable created from the matrix voting data, the External Solution Vote 
Share, aims to measure the proportion of votes cast for internal solutions versus external 
solutions. This measure is an indicator of the degree to which participants tended to vote 
for external solutions instead of internal solutions across all of the problems. The percent 
of votes cast for external solutions is the inverse of the percent of votes cast for internal 
solutions. 
One concern in public voting sessions is that one group participant will dominate 
the choices by voting first, and all other participants will cast their votes for the same 
choice, or that citizens will feel pressure to vote the same as others. If this happened, it 
could be reflected in clustering on one solution, although it could also reflect actual 
consensus resulting from deliberation. To capture this potential pattern, we created a 
measure of the degree to which voting is not evenly distributed across all possible vote 
choices, called Vote Clustering. We first calculate the votes that would be received in each 
cell if votes were distributed evenly for all six possible choices, and then calculate the 
average distance from this number for each group. 
Finally, to capture the proportion of votes cast for any one solution, the evaluation 
team created a measure of the maximum vote share received by any one solution across all 
three problems. This measure is called Max Solution Vote Share and should be higher when 
the groups’ votes are clustered on one particular solution. 
V. Quantitative Data 
Survey Data 
A Cambodian research firm, Center for Advanced Study, implemented the survey, 
hiring and training Cambodian enumerators who conducted face-to-face interviews with 
respondents. In each village selected for the study, 120 adults were surveyed one week 
prior to the CD, and the same individuals were contacted again approximately one week 
after the survey; comparable baseline and outcome data were collected from individuals in 
six villages that did not receive the CD treatment. As much as possible, the surveys in the 
paired villages were fielded at about the same time.16 The relatively short time horizon was 
 
16 A precise timeline is available upon request. Flooding disrupted one of the surveys and delayed its 
implementation. One of the CDs was canceled at the last minute, which meant that the pre-test survey and the 
deliberative session were held as originally scheduled, and the CD and post-test survey were delayed by three 
weeks. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-15
used primarily because of concerns about individuals who took the pre-test being relocated 
for the post-test, particularly during seasons when significant portions of the population 
travel for agricultural work. However, the short time horizon means that any effects 
detected after one week could dissipate over longer periods of time. 
The survey captain picked a random starting point within each target village, and 
then enumerators surveyed one adult from each household; the gender of the target 
respondent in each household was selected by coin flip. If 120 adults were not available in 
the village selected, the survey continued into the next closest village. All groups of 120 
adults are from the same commune. No other stratification occurred within each commune, 
and the survey is not intended to be representative of the larger Cambodian population nor 
necessarily representative of the entire population of the village. Many young people work 
outside of their home villages for significant portions of the year. The population is
stratified on gender, and within each gender is representative of the village population that 
is most likely to be exposed to the CD. 
For the CD only analysis, because the randomization took place at the village level, 
we compare outcomes within six village-pairs by whether the village was treated with the 
CD event (six treatment villages) or not (six control villages). Although there are several 
plausible methods for analysis, we have elected to present only one here for ease of 
interpretation. Because we are primarily interested in individual level change we continue 
to rely on individual level data but cluster the standard errors at the village level, as the 
assignment of the CD and the randomization occurred at the village level. All CD analysis 
should account for the village-pair randomization, and therefore all CD models include 
village-pair fixed effects. This means that respondents in a treatment village are compared 
to respondents in the control village in the same district. Results for the CD analysis are 
presented from the following model: 
 
Where represents each outcome variable of interest for individual i at time t. Time t 
indicates the follow up survey, whereas time t-1 indicates the baseline survey. 
represents the treatment status, and is equal to 1 if individual i lived in a village that
received the CD treatment. The variables through indicate village-pair dummies 
(with one village-pair serving as the residual category), and 
is the error term. 
Turning to the deliberative session analysis, we use a similar comparison but 
account for the randomization at the individual level. For all results on the effects of the 
deliberative sessions, we compare survey responses between those who were invited to 
the deliberative session with those who were not, within each of the nine villages that 
received the CD. As noted above, the samples for the CD only and the deliberative session 
analysis are not the same in this study. The deliberative session analysis includes 9 villages: 
6 villages from the CD only analysis, as well as the three additional treatment villages.17
Whereas the CD randomization took place at the village level, the deliberative session 
 
17 As noted above, the original plan was for five villages to be proposed simultaneously in each province, 
three of which would then be randomly selected to receive treatment. If this method were used, then the 
analysis could include village and province-level dummies. Under the village-pair procedure actually adopted, 
survey data from the final three CD villages are, therefore, primarily used for the evaluation of the 
deliberative sessions in the analysis presented here. It would be possible to use the control village data in the 
deliberative session analysis if several assumptions were relaxed. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-16
treatment was assigned at the individual level. The control villages are not used in the 
deliberative session analysis presented here because it is methodologically complicated to 
combine the two randomizations, one at the village level, and one at the individual level. 
Within each CD treatment village, we compare survey responses based on whether the 
individual was invited to the deliberative session. Results are presented from the following 
model: 
 
Where represents each outcome variable of interest for individual i at time t. Time t 
indicates the follow up survey, whereas time t-1 indicates the baseline survey. 
represents treatment status and is equal to 1 if individual i was invited to attend the 
deliberative session. The variables through indicate village dummies (with one 
village serving as the residual category), and 
is the error term. The central difference 
between these two models is that the CD analysis compares individuals across village-pairs, 
and the deliberative session analysis makes comparisons within villages. 
The results from the survey presented in Tables 3-10 (below) include four pieces of 
information: the baseline average in the 12 villages included in the CD only study, the 
baseline average in the 9 villages included in the deliberative session study, the estimated 
effect of the CD on survey responses and the estimated effect of the deliberative session on 
survey responses. 
Note that many of the outcome variables are binary, and results for these variables 
are presented as linear probability models. When the outcome variables are ordinal 
(including Likert scales), the results are also presented from a linear model (OLS). Results 
are also available with each option on the scale considered separately as a dummy variable. 
Survey Attrition and Balance Tests 
In using survey data as the outcome measure for a field experiment, one potential 
concern is that respondents became more likely to drop out of the survey in the post-test if 
they were assigned to either of the treatment groups. For the CD analysis, in the six 
treatment villages, 91 percent of pre-test survey respondents completed the follow-up 
survey. In the six control villages, 92 percent of respondents completed the follow up 
survey. For the deliberative session analysis, across the nine villages included, completion 
of the follow up survey is 92 percent in both the treatment and control conditions. The test 
shown in Table 2 confirms that the probability of taking the follow-up survey is not 
determined by assignment to either of the treatment conditions. Assignment to the 
treatment condition has no statistically significant effect on whether the respondent 
completed the follow up survey, suggesting that survey attrition is not a problem for the 
analysis. 
In experimental designs, it is also important to test for balance among pre-treatment 
covariates. Because of the randomization, all pre-treatment measurements are equal in 
expectation. In testing for balance on pre-treatment covariates, we use the following model, 
similar to that discussed above: 
 
Where represents each outcome variable of interest for individual i at time t-1. 
Time t indicates the follow up survey, whereas time t-1 indicates the baseline survey. 
represents the treatment status, and is equal to 1 if individual i lived in a village that
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-17
received the CD treatment. The variables through indicate village-pair dummies for 
the CD analysis and village dummies for the deliberative session analysis, and 
is the error 
term. Note that all comparisons are within village-pairs. Figure 4 presents the baseline 
tests for balance between villages assigned to the treatment of the CD event and villages 
assigned to the control. 
Table 2: Evaluation of Survey Attrition 
(1) (2)
CD--Was Follow-Up Survey 
Completed?
Deliberative Session --Was 
Follow-Up Survey Completed? 
Assignment to CD 
Treatment
0.008
(0.007)
Assignment to 
Deliberative 
Session Treatment
0.002
(0.16)
Constant 0.913
(0.005)
0.916
N 1440 1080
F 8.85 1.33
Prob > F 0.0011 0.22
R-squared 0.0075 0.01
Root MSE 0.2816 0.27
Notes: Robust standard errors in parentheses. In Model 1, coefficients for village-pair dummy variables are 
not reported, and in Model 2, coefficients for village dummy variables are not reported. 
 The expectation is that all pre-treatment covariates will be the same across 
treatment and control comparisons. As shown in Figure 4, many of the demographic 
variables are balanced (i.e. they are not significantly different) between individuals in 
villages receiving the CD villages and individuals in control villages. However, respondents 
in the treatment villages are about two years older, on average, and have lived in the village 
about two years longer. These differences are not substantively meaningful; it is not clear 
why individuals who are two years older, or who have lived in the village two years longer, 
would necessarily respond to the CD differently. 
In addition to demographic variables, however, citizens were also asked whether 
they had ever attended a prior CD meeting, or whether they had ever listened to one of 
these programs on the radio. Contrary to expectations, citizens in treatment villages were 
slightly more likely to report having ever attended a CD meeting in the baseline survey, and 
were much more likely to report having listened to a CD on the radio in the baseline survey. 
The baseline surveys were conducted about one week before the CD took place, and these 
results are puzzling and suggest that news of the CD leaked in the treatment villages during 
the baseline survey. 
Advertising for the event did not start until after the surveys were completed, 
however some advance work was conducted by the local NGO partners. There are several 
possible explanations for this difference in what should be a pre-treatment covariate that is 
equal between treatment and control. First, the surveys took place over 3-4 days, and each 
participant was invited to attend the CD at the end of the survey. It is possible that news of 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-18
the upcoming CD reached subsequent survey respondents, or that village officials 
communicated to residents about the upcoming CD event before advertising officially 
began. 
Whatever the explanation for the difference, all analyses have been done including 
the pre-treatment covariates that are not balanced between treatment and control groups 
(age, years in commune, previous CD, and CD radio) and are available upon request. For 
simplicity and for reasons of space, we present the models without these variables in this 
document. 
Figure 5 presents the same comparisons for the deliberative session treatment, but 
using the appropriate within-village comparison for the deliberative session analysis. 
There are no significant differences between the group assigned to the deliberative session 
treatment and the group assigned to the deliberative session control group within the CD 
villages. 
Figure 4: Baseline CD Balance Tests 
a.Gender (male=1)
b.Age
c. Years in Commune
d. Married
e. Single
f. Divorced
g. Widowed
h. Years Schooling
i. Reading Level
j. Household Asset Index
k. Khmer Ethnicity
l. Dwelling Type
m. Electricity
n. Prior CD?
o. CD on Radio?
0 1 2 3 4 Estimated Difference & 95% Confidence Interval
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-19
Figure 5: Baseline Balance Tests for Deliberative Session Analysis 
VI. Quantitative Results: Effects of CD and Deliberative Sessions on Knowledge, 
Attitudes and Behavior 
All reported effects are “intent to treat” estimates. This means that all estimates 
compare the individuals that were randomly assigned to receive the treatment with those 
who were not. Because not all survey respondents actually attended the CD event, these are 
likely underestimates of the effect that would have been measured if all survey 
respondents had attended the CD. Each set of results is presented in a table, with combined 
indices also presented where appropriate. Across all six treatment villages, about 56 
percent reported that they had attended the CD event in the follow-up survey. Zero 
respondents in the control villages reported having attended the CD event. 
Of survey respondents invited to attend the deliberative session, 46 percent 
reported that they had actually attended the deliberative session. An additional 7 percent 
of survey respondents who were not invited to the deliberative session also reported 
having attended. These compliance rates were difficult to predict in advance of the 
evaluation, and make it less likely that any effects of the deliberative session + CD can be 
detected in the survey data. The same concerns do not apply to the analysis of the matrix 
voting data collected during the deliberative sessions. 
In analyzing the effects of the CD only and the deliberative session + CD, it is 
possible to account for compliance issues and the deliberative session spillover (that some 
“control” individuals attended these sessions). In this report the more conservative 
estimates are reported, in part because they are more easily explained to a diverse 
audience, and in part because of time and space constraints. The companion academic 
a.Gender (male=1)
b.Age
c. Years in Commune
d. Married
e. Single
f. Divorced
g. Widowed
h. Years Schooling
i. Reading Level
j. Household Asset Index
k. Khmer Ethnicity
l. Dwelling Type
m. Electricity
n. Prior CD?
o. CD on Radio?
-2 -1 0 1 2 Estimated Difference & 95% Confidence Interval
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-20
article(s) will consider these issues in greater detail, but the more sophisticated analyses 
are unlikely to change the core findings. 
There are six substantive sections of the survey, all of which were intended to 
evaluate whether the CD and the deliberative sessions influenced citizen knowledge, 
attitudes, or political behavior. The six sections are: 
A. Knowledge of the Political Process
B. Confidence in the Political Process
C. Familiarity with Politics 
D. Engagement in the Political Process
E. MNA Responsiveness 
F. Priority Issues (discussed in Section VII) 
Exposure to the CD event caused statistically significant changes in the responses to nearly 
all sections of the survey, consistent with our expectations, with the exception of 
Confidence in the Political Process. There are some notable differences in how men and 
women respond to the CD and the deliberative session + CD treatments, all of which are 
noted in Tables 3-10 and shown in detail in the data appendix. For example, the effect of 
the CD on several of the political knowledge questions is greater for men, and the effect of 
the CD is only significant among women for several questions about confidence in the 
political process. 
Several results were unanticipated, including the largely null effects of the CD and 
the deliberative session on confidence in the political process. Another small but surprising 
result is that exposure to the CD made respondents 5 percentage points more likely to say 
they had voted in the last election, even though no election occurred between the pre- and 
post-test surveys. This may be interpreted as evidence that respondents in the treatment 
villages were more likely to give their perception of the “right” answer. This phenomenon 
should be less of an issue on the other sections of the survey. For example, if a citizen 
responds more accurately to a question about their knowledge of the political process 
because of social desirability bias, it is still the case that exposure to the CD made them 
more likely to increase their knowledge about politics, even if the increase could be for the 
“wrong” reasons. Additionally, misreporting one’s own voting behavior is a common 
problem in survey data. 
Exposure to the CD also made individuals more likely to report belonging to a 
political party. This is surprising in part because it is not clear that the CD should have had 
such an effect. Upon reflection, however, it is plausible that local officials from the political 
parties used the CD as an opportunity to recruit new members, or that citizens felt 
motivated to join a political party after the CD event. 
For each part of the survey, the following sections of this report present the baseline 
data from the pre-treatment survey, the estimated effect of the CD and the CD plus 
deliberative session treatment on responses to each question, and information on any 
gendered or otherwise notable differences. More detailed statistical tables are included in 
the data appendix to this document. 
A. Knowledge of the Political Process 
As shown in Table 3, exposure to the CD caused consistent and statistically 
significant increases in how citizens responded to questions about their knowledge of the 
political process. For example, respondents in CD villages became 18 percentage points 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-21
more likely to be able to answer the question, “What does a Member of the National 
Assembly Do?”, and 15 percentage points more likely to say they have a right to talk to 
their MNAs about their concerns. 
To summarize the changes across all knowledge survey questions, two indices were 
created. The first combines five separate yes/no knowledge questions (KPP1-KPP5), and 
the second combines the six possible responses to the KPP6 about roles and 
responsibilities of the National Assembly. Respondents in villages exposed to the CD scored 
11 percentage points higher on the Knowledge Index I. To put this change in perspective, 
an 11 percentage point increase would occur if half of all respondents in CD villages 
answered one additional knowledge question. On the Knowledge Index II, the CD caused a 
6.9 percentage point increase. 
There are some differences between men and women in the effects of the CD. The 
CD caused an average of a 3.6 percentage point increase in the probability that the 
respondent would give an answer when asked “What does the National Assembly Do?”. 
This effect is entirely among men, and the CD increases the probability that men will 
answer this question by 8.3 percentage points. The baseline responses also differ 
significantly between men and women. The average score on the two knowledge indices for 
men are 49% and 57%. The average score on the knowledge indices for women are 30% 
and 46%, indicating that women in villages served by the CD program have, on average, 
much less baseline political knowledge than men. 
Turning to the deliberative session survey results, exposure to the deliberative 
session did not lead to any statistically significant changes in citizen knowledge of the 
political process. The (statistically insignificant) point estimates are also small compared 
with the effects of the CD, which is consistent with a null effect. The one exception is that 
among women, the deliberative sessions had a 6.6 percentage point increase in the 
probability that women will say that “giving gifts” is not a responsibility of MNAs. In 
retrospect, these largely null findings are unsurprising, as the deliberative sessions did not 
have an explicit educational component. 
B. Confidence in the Political Process and Options 
In contrast to the knowledge questions in which the effect of the CD was uniformly 
positive, there is mixed evidence about whether exposure to the CD leads to increased 
confidence in the political process. Respondents were asked about a series of statements, 
and responded on a scale ranging from Strongly Disagree to Strongly Agree. “Don’t Know,” 
“Can’t Choose” and “Decline to Answer” were coded as indifference, and placed in the 
middle of the five point scale.18
As shown in the CP1 & CP2 Index in Table 4, when asked about agreement or 
disagreement with various statements about politics, there is not a consistent and 
statistically significant effect of the CD on confidence in the political process or confidence 
in political options. The CD treatment is significant for three individual questions: there is 
an increase of 0.1 points in agreement in with “If I tell my MNAs about issues in my 
community, they will take action to help solve the problem;” a .16 point increase in 
 
18 Results are available upon request with these responses dropped from the analysis, and are largely 
consistent. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-22
agreement with “It is good for democracy in Cambodia to have opposition parties;” and a 
0.09 point increase in agreement with “I can choose who represents me in government.” 
There are also several differences in how men and women are influenced by the CD. 
For example, the results show that the average effect of the CD is to cause a .16 point 
increase (on a five point scale) in the degree to which respondents agree with the 
statement that “It is good for democracy in Cambodia to have opposition parties.” However, 
this effect is only statistically significant among women (+0.23). Exposure to the CD also 
makes women more likely to agree (+0.11) with the statement “I can choose who 
represents me in government.” 
The average effect of the deliberative session is not statistically significant for any 
question in this section. However, there are several statistically significant results that are 
clearer in the analysis by group gender composition. Among men, the deliberative sessions 
decrease agreement with the statement “None of the MNAs from my province are aware of 
issues facing the local people” by about a quarter of a point. Among women, the 
deliberative sessions decrease agreement with the statement “It does not matter how I 
vote, nothing will change,” also by about a quarter of a point on the five point scale. It is 
also notable that there are null results of the deliberative session for two of the questions 
on which the evaluation team anticipated the deliberative sessions to have had a significant 
effect: “I have a role to play in solving problems in my community” and “I have a role to 
play in solving problems in my country.” These null results are surprising because the 
effect of the deliberative session is negative (counter to expectations) and insignificant. 
Overall, because these results are not consistent across all questions that deal with 
confidence in the political process, it is possible that the statistically significant findings are 
due to chance, so they should be interpreted with greater caution and less certainty. 
C. Familiarity with Politics 
Questions in this section focused on whether respondents are willing to talk about 
politics, the frequency with which they talk about politics, and difficulty in talking about 
politics. Because the possible responses for each question are not on the same scale, we did 
not create an index for this section. However, for two of the three questions, exposure to 
the CD led to significant improvements in familiarity with politics. Somewhat surprisingly, 
neither the CD treatment nor the deliberative session + CD treatment affected whether 
respondents would have a hard time discussing controversial issues with friends or 
neighbors. 
This null finding would be consistent with respondent awareness that the 
Cambodian political environment is repressive. Nevertheless, the evaluation team expected 
a positive and significant effect of the deliberative sessions and potentially also the CD, on 
the ease of discussing politics with friends or neighbors. Keeping this null finding in mind 
could be useful in considering future programing decisions. 
D. Engagement in the Political Process 
This section of the survey included a number of questions about respondent’s self￾reported political behavior and their willingness to engage in activities like filing a case in 
court, signing a petition, or participating in a rally. Overall, there are consistent and 
significant increases in engagement in the political process caused by the CD. For example, 
the CD treatment causes a 12 percentage point increase in the probability that a 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-23
respondent “would” or “has already” signed a petition, a 9 percentage point increase in 
willingness to write a letter to a government authority, and a 15 percentage point increase 
in willingness to attend a political party event. 
Exposure to the CD makes citizens more likely to report belonging to a political 
party (perhaps because parties recruited at CD events) or a self-help group. Respondents 
are not more likely to join any other groups because of the CD. Respondents were also 
asked about whether they are willing to engage in a number of “actions people sometimes 
take as citizens.” When all possible actions are combined into an index, the CD causes a 6.3 
percentage point increase in the probability that respondents will say they “already have” 
or “would” take actions like attending a political party event, signing a petition, or 
contacting an MNA. Many of these effects of the CD are more pronounced among women. 
The deliberative session + CD effects are not statistically significant on average in 
this section, but there is some evidence that men and women reacted differently (and 
sometimes oppositely) to the deliberative session. Any differences by gender are 
referenced in Table 6, Table 7, and Table 8 below. 
E. Responsiveness of Members of the National Assembly 
Respondents were also asked to reflect on the degree to which they thought MNAs 
were responsive to their communities. Note that these questions followed what was likely 
the first ever visit to each community by an MNA, and almost certainly the first 
simultaneous visit by MNAs from multiple political parties. Respondents exposed to the CD 
were much more likely to think that MNAs were communicating with their community, and 
more likely to name infrastructure or public goods provided by the MNA to the community 
when asked if their MNAs had done anything for their community recently. It is possible 
that much of these effects are simply the result of the fact that several MNAs actually did 
visit the village and met with local leaders during the CD process. The deliberative session 
+ CD treatment has no detectable effects on perceptions of MNA responsiveness, which is 
consistent with expectations. 
VII. Quantitative Results: Effects of CD and Deliberative Sessions on Priority Issues 
 This section turns to the quantitative results about the effects of participation in a 
deliberative session on priority issues. We rely exclusively on survey data to examine the 
effect of the deliberative sessions as a whole on priority issues (question 3). For the effect 
of the group type and the facilitator gender on priority issues, we rely on the qualitative 
data from the deliberative sessions. 
The evaluation team expected that the deliberative session + CD treatment would 
have significant effects on what respondents listed as priority issues, in part because 
engaging in deliberation with members of their community and their elected officials 
should cause some change in which issues respondents would like their MNA to take action 
on. For reference, the effects of the CD alone on priority issues for MNAs to take action on 
are also presented. We did not have specific expectations about which issues would 
become more popular. 
As summarized in Table 10, the CD had several significant effects on which issues 
were reported as priorities, making some issues less likely to be mentioned in the top 
three, and some issues much more likely to be mentioned in the top three. Most notably, 
the lack of infrastructure became 22 percentage points more likely to be mentioned among 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-24
respondents exposed to the CD. The deliberative session + CD had no significant effects on 
priority issues, at least as detected in the survey. 
 
VIII. Quantitative Results: Effects of Deliberative Session Group Gender Composition 
and Facilitator Gender on Knowledge, Attitudes, and Behavior 
As introduced above, there are two types of quantitative data relevant to questions 
4 and 6. The first type is the survey data, but is limited to the participants who attended the 
deliberative session who could be matched to the survey, and should be interpreted with 
greater caution. The second type of data is drawn from the matrix voting that took place 
within the deliberative sessions, which can be considered a measure of individual behavior 
as influenced by deliberative session group type or the facilitator. Questions 4 and 6 focus 
on the effect of group type, facilitator gender, and facilitator bias on knowledge, attitudes 
and behavior. 
Across the survey, there are not consistent and statistically significant effects of 
group type or facilitator gender, as shown in Table 11. The exceptions, which may be due to 
chance, are that participants in female groups are more likely than the other two types of 
groups to report a willingness to vote in the next election, score about 0.47 points higher 
(five point scale) on the Confidence in the Political Process Index, and experience a 4.7 
percentage point increase in one of the three indices that measure Engagement with the 
Political Process. Compared to male and female groups, mixed groups are 2.8 percentage 
points less likely to say they will vote in the next election. The final significant finding is 
that participants in male groups are 6 percentage points more likely to be able to list one or 
more activities undertaken by their MNA in the past six months. 
Turning to the effects of facilitator gender, they are almost all indistinguishable from 
zero, with the two exceptions being that male facilitators increase scores on the first 
Knowledge of the Political Process index by 5.4 percent, and slightly decrease scores on the 
Confidence in the Political Process index. However, the effect is only .05 on a 5 point scale, 
which is not substantively meaningful. Note that these comparisons are to groups 
facilitated by females rather than a full control group. 
 The matrix voting data suggests several differences in voting patterns that are 
attributable to group type, facilitator gender, and/or specific facilitators. Note that these 
results make clear that there are some differences in voting patterns based on group type 
and facilitator, but the precise meaning of these differences is still subject to debate. None 
of the matrix voting patterns are straightforward measures of “good” or “bad” deliberative 
processes, but they are suggestive of changes in behavior. The results for all four measures 
of patterns in the matrix voting data, including External Solution Vote Share, On-Diagonal 
Voting, Vote Clustering, and Max Solution Vote Share, are presented in Table 12 and Table 
13. 
When compared to the two other group types (mixed gender and all-female), male 
groups receive higher average scores on all four measures, although the difference is only 
statistically significant for External Solution Vote Share. Male-only groups receive 9.6 
percentage points more External Solution Vote Share than all other groups, indicating that 
male-only groups are much more likely to choose external solutions, rather than internal 
solutions, for problems facing their community. Female groups are 8 percentage points less 
likely to vote for External Solutions (or 8 percentage points more likely to vote for internal 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-25
solutions). In terms of the gender composition of the groups, none of the other matrix 
voting patterns demonstrate statistically significant differences. 
There are clearer differences by gender of facilitator. Keep in mind that female 
facilitators can never be assigned to an all-male group and male facilitators can never be 
assigned to an all-female group. Therefore, some of the differences attributable to the 
gender of facilitator may be caused by the group type. Nevertheless, relative to female 
facilitators, male facilitators are associated with higher scores on all four measures, and 
these differences are statistically significant for three the four measures. Groups with male 
facilitators receive 16.1 percentage points more External Solution Vote Share, 2.29 more 
votes on Vote Clustering, and 5.5 percentage points greater Max Solution Vote Share. 
There are also some differences attributable to individual facilitators, as shown in 
Table 13. For example, compared to all other facilitators, the groups led by Facilitators 1 
and 9 are significantly more likely to vote for external solutions, whereas the groups led by 
Facilitators 1 and 9 are significantly more likely to vote for internal solutions. Facilitator 8 
is the only one who is significantly different from the other facilitators on all four matrix 
voting patterns: her groups are more likely to vote for internal solutions, less likely to vote 
On Diagonal, have less Vote Clustering than average, and are less likely to pool on one 
solution to all problems (Maximum Solution Proportion). 
 
IX. Discussion of Findings 
Overall, there is clear evidence that the CD influenced citizen knowledge of the 
political process, self-reported political behavior, perceptions of Members of the National 
Assembly, and their priority issues in a manner that is consistent with our expectations and 
evaluation hypotheses (question 1). There is far less evidence from the survey data that the 
deliberative session + CD treatment had any meaningful effect on citizen knowledge, 
attitudes or behavior in a manner consistent with our hypotheses (questions 2-3). Part of 
this could be due to the lower-than-expected compliance with invitations to the 
deliberative session. It is also clear, in retrospect, that the survey may not have been the 
ideal instrument to capture changes caused by the deliberative session, in part because 
these sessions did not involve an explicit civic education component. Finally, there is little 
evidence that deliberative session group composition or facilitator gender have an effect on 
citizen knowledge, attitudes and behavior (question 4), but there is some evidence of group 
composition and facilitator gender affecting the voting patterns of deliberative session 
groups (questions 6 and 7). 
Even if the effects of the deliberative sessions detected with the survey were 
doubled or even tripled (which would account for the low compliance rate), they are still 
small for most sections of the survey, and sometimes go in the opposite direction than 
expected. The deliberative session + CD may be an interesting tool for a number of reasons, 
but based on the survey data, the deliberative session did not have a detectable effect on 
citizen knowledge of the political process, attitudes toward democracy, self-reported 
political behavior, or self-reported priority issues, as detected in the survey data. There is 
some evidence that group type and facilitator gender influenced behavior within the 
deliberative sessions, but in the quantitative data alone, it is unclear whether these changes 
are consistent with the objective of promoting democracy. 
Moving forward, it is clear that the levels of knowledge among typical recipients of 
this program are much lower than anticipated, particularly among women. The survey 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-26
suggests that the typical CD participant has very little knowledge about the political 
process or his/her government. The very high percentage of individuals in the pre-test who 
do not know the Khmer words for “National Assembly,” or have heard the words but 
cannot say anything about what the National Assembly does, suggests that holding 
Members of the National Assembly accountable for their actions may be hamstrung by a 
lack of knowledge about what MNAs are supposed to be doing at all. The effects of the CD 
and/or the deliberative sessions might be enhanced by including more explicit civic 
education into the program. 
The one exception to the consistent positive effects of the CD are the largely null 
results for most questions about confidence in the political process. The fact that the CD 
had little effect on citizen confidence in the political process was not what we expected 
when designing this evaluation. The effects on citizens could be sufficiently heterogeneous 
(both positive and negative) that a consistent average effect cannot be detected. Some of 
this was apparent in the analysis of CD effect by gender, where it is clear that men and 
women were affected differently by the CD. The baseline perceptions regarding confidence 
in the political process are more positive than negative, but still not very high. It is worth 
debating what it means to try to increase citizen confidence in the political process in an 
electoral authoritarian regime, like Cambodia, where the process is clearly dominated by
and biased in favor of the Cambodian People’s Party. Perhaps increasing political 
knowledge in Cambodia should be expected to decrease citizen confidence in the political 
process, but this remains an open question and should be subject to further study. 
Although the effects of the CD were largely in line with expectations of the 
evaluation team (with a few exceptions, as noted), it is not clear that the effects are lasting. 
The surveys were conducted about a week after the CD event, which is a long time 
compared to many laboratory or survey experiments. However, it remains unclear whether 
the one day exposure to the CD created permanent changes in citizens’ knowledge, 
attitudes, and behavior. It would be interesting (and still feasible from a research design 
perspective) to return to the 12 villages included in the CD study and conduct a shorter 
follow-up survey to measure whether the effects of the CD are lasting. Although the survey 
could not include the same individuals again, such data would allow for examination of any 
longer-term village-level – rather than individual-level - effects resulting from the CD. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-27
Table 3: Knowledge of the Political Process, Summary of CD and Deliberative Session + CD Results 
Survey Question Outcome Variable Baseline CD/
Delib.
Session
Estimated Effect 
of CD 
Estimated 
Effect of 
Deliberative 
Session + CD
Other Relevant 
Findings
KPP1: What does the National 
Assembly do?
Prob. that respondent can 
answer the question (0/1)
23%/ 24% Increase of 3.6% - 2.4% (insig.) CD Effect only among men 
(increase of 8.3%)
KPP2: What does a “Member of the 
National Assembly” do?
Probability that respondent 
can answer the question 
(0/1)
35%/37% Increase of 18% +2.9% (insig.)
KPP3: Can you name one or more 
Members of the National Assembly 
from your province?
Probability that respondent 
can answer the question 
(0/1)
26% /26% Increase of 9.3% -2.3% (insig.) CD effect greater among men 
(12%) and respondents’ 
accuracy in identifying an 
MNA correctly increased by 
26%.
KPP4: What are three things you 
believe your National Assembly 
Members are supposed to do for you 
and your community?
Probability that respondent 
can answer the question 
(0/1)
68%/83% Increase of 10% +2.2 % (insig.) Responses included “Builds 
Infrastructure” (58%) and 
“Helps People” (9%)
KPP5: Do you have the right to talk 
to your National Assembly Members 
about your concerns?
Probability of unqualified 
“yes” (0/1)
55%/ 49% Increase of 15% +0.7 % (insig.) CD Effect greater among men 
(17%). 
Knowledge Index 1 (KPP1-KPP5) Percent “yes” of KPP1-KPP5 
(0-1)
39%/ 41% Increase of 11% + 0.7% (insig.)
KPP6: Are the following all roles and 
responsibilities of the National 
Assembly? 
Percent correct of KPP6_1 
to KPP6_6 (0-1)
58%/ 57% Increase of 6.9% + 1.1% (insig.)
Make/ Approve Laws Probability of “yes” 63%/ 73% Increase of 8.9% + 0.4% (insig.)
Decide Court Cases (inverted) Probability of “no” 46%/ 62% + 4% (insig.) + 0.2% (insig.)
Represent the people Probability of “yes” 64%/ 75% Increase of 8.9% + 3.1% (insig.)
Give gifts (inverted) Probability of “no” 50%/ 56% Increase of 9.3% + 1.5% (insig.) Deliberative session effect is 
6.6% among women, with no 
such effect among men. 
Enforce Laws Probability of “yes” 63%/ 71% Increase of 7.8% + 1.7% (insig.)
Provide Oversight of Gov’t Probability of “yes” 59%/ 66% Increase of 8.7% + 1.7% (insig.)
Note: This table summarizes the main results. More detailed results are available in the appendix. Results marked as “insig.” do not meet the standard measures of 
statistical significance of p <.1 and a two-tailed test. Statistically significant results are in bold. The CD baseline measures and analysis are based on 12 villages (six 
village pairs) and the deliberative session baseline measures and analysis are based on the 9 villages that received the CD treatment. The responses to “Decide Court 
Cases” and “Give Gifts” are inverted so that the anticipated direction of the effect is consistent across all components of the index. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-28
Table 4: Confidence in the Political Process, Summary of CD and Deliberative Session + CD Results 
Survey Question Outcome 
Variable 
Baseline CD / 
Deliberative 
Session
Estimated 
Effect of CD 
Estimated 
Effect of 
Deliberative 
Session + CD
Other Relevant Findings
CP1 –CP2: For each statement, tell me whether you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?
CP1_1: If I tell my MNAs about issues in 
my community, they will take action to 
help solve the problem.
Strongly 
Disagree\
Somewhat 
Disagree\
Don’t Know\
Somewhat 
Agree\
Strongly Agree
(-2 to 2)
0.92 / 0.81 Increase of 0.10
(borderline sig. 
p=.102)
+0.06 (insig)
CP1_2: I have a role to play in solving 
problems in my community.
0.53/ 0.27 -0.08 (insig) -0.14 (insig)
CP1_3: I have a role to play in solving 
problems in my country.
0.30/ 0.11 +0.02 (insig) -0.13 (insig)
CP1_4: None of the MNAs from my 
province are aware of issues facing the 
local people. 
0.13/ 0.26 +0.014 (insig) -0.06 (insig) Among men, the deliberative 
session has a statistically 
significant negative effect (-0.27). 
CP2_1: It is good for democracy in 
Cambodia to have opposition parties.
0.62/ 0.66 Increase of 0.16 +0.06 (insig) CD effect is primarily among 
women (+0.23)
CP2_2: All the opposition parties do is 
just ‘criticize.’
0.26/ 0.29 -0.04 (insig) +0.00 (insig)
CP2_3: I can discuss political issues 
openly in my community.
0.57/ 0.37 -0.05 (insig) -0.08 (insig)
CP2_4: If I disagree with the government, 
I can vote for another party.
0.80/ 0.85 +0.11 (insig) +0.07 (insig)
CP2_5: It does not matter how I vote, 
nothing will change.
0.31/ 0.39 -0.04 (insig) -0.07 (insig) Among women, the deliberative 
session has a significant negative 
effect (-0.26).
CP2_6: I can choose who represents me in 
government.
1.03/ 0.87 Increase of 0.09 
(borderline sig. 
p=.12)
-0.05(insig) CD effect is significant among 
women (+0.11). 
CP1 and CP2 Index 0.64/ 0.58 + 0.011 (insig) +0.01(insig)
Note: For the CP1 and CP2 Index, all variables are converted to such binary measures such that somewhat and strongly agree are coded as one, and all other responses 
are zero. CP2_2 and CP2_5 were inverted for consistency. This table summarizes the main results. More detailed results are available in the appendix. Results marked as 
“insig.” do not meet the standard measure of statistical significance of p <.1 in a two-tailed test. Statistically significant results are in bold. The CD baseline measures and 
analysis are based on 12 villages (six village pairs) and the deliberative session baseline measures and analysis are based on the 9 villages that received the CD 
treatment. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-29
Table 5: Familiarity with Politics, Summary of CD and Deliberative Session + CD Results 
Survey Question Outcome Variable Baseline 
CD / 
Deliberative
Session
Estimated 
Effect of CD 
Estimated 
Effect of 
Deliberative 
Session + CD
Other Relevant Findings
FP1: How interested are you in 
politics? 
I reject politics/Not 
Interested/Don’t know/ 
Somewhat 
Interested/Very 
Interested (-2 to 2)
0.65 /0.69 Increase of 0.23 -0.006 (insig) Exposure to the CD causes a 5.6% 
increase in the probability that 
individuals say they are “very 
interested” in politics and a 8.7% 
decrease in the probability that 
individuals say they are “not 
interested” in politics. 
FP2: How often do you talk 
about politics with other 
people, such as friends or 
neighbors?
Don’t Know/Never/ 
Rarely/ A few times a 
month/ A few times a 
week/ Everyday
1.8/ 1.8 Increase of 0.17 -0.018 (insig) Exposure to the CD causes an 
8.1% decrease in the probability 
that individuals say they “never” 
talk about politics and a 5% 
increase in the probability that 
individuals say they talk about 
politics “a few times a month.”
FP3: Would you have a hard 
time discussing controversial 
issues with your friends or 
neighbors if you had different
political opinions?
Don’t Know/ Very hard/ 
A bit hard/ Not too 
hard/ Not hard at all
2.4/ 2.4 + 0.05 (insig.) -0.090 (insig)
Note: This table summarizes the main results. More detailed results are available in the appendix. Results marked as “insig.” do not meet the standard measure of 
statistical significance of p <.1 in a two-tailed test. Statistically significant results are in bold. The CD baseline measures and analysis are based on 12 villages (six village 
pairs) and the deliberative session baseline measures and analysis are based on the 9 villages that received the CD treatment. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-30
Table 6: Engagement with the Political Process I, Summary of CD and Deliberative Session + CD Results 
Survey Question Outcome Variable Baseline CD 
/ Deliberative 
Session
Estimated 
Effect of CD 
Estimated 
Effect of 
Deliberative 
Session +CD
Other Relevant Findings
EPP1: Did you vote in the last 
national election in 2008?
Probability of “yes” 84%/ 86% Increase of 5% +0.4% (insig.) CD effect surprising because
respondents could not have 
voted between pre- and post￾test. 
EPP2: Do you intend to vote in the 
Commune Council election in 2012?
Probability of “yes” 97%/ 97% Increase of 
1.3%
+0.6% (insig.) CD effect primarily among 
women, 1.8%.
EPP3: Do you belong to any groups 
or associations in your 
neighborhood? (INDEX)
Percent “yes” out of all 
groups listed
8%/ 7% 1.0% (insig.) +0.3% (insig.) 
Religious Association Probability of “yes” 6%/7% 1.8% (insig.) +0.9% (insig.)
Cultural Association Probability of “yes” 2.0%/ 1% -0.3% (insig.) +0.5% (insig.)
NGO Probability of “yes” 5%/ 5% 0. 7% (insig.) Increase of 
2.3%
Deliberative session effect 
only among men, 4.8%. 
Developmental association Probability of “yes” 6.4%/ 5% 2.1% (insig.) -0.5% (insig.)
Farmer’s Association Probability of “yes” 10%/7% -1% (insig.) +0.9% (insig.)
Fisher’s Association Probability of “yes” 3%/ 5% -1.7% (insig.) Increase of 
1.1%
Deliberative session effect 
significant only among men.
Forest Community Association Probability of “yes” 12%/ 5% -3.8% (insig.) 0.1% (insig.) CD effect significant only for 
women, 5.2%
Political Party Probability of “yes” 18%/ 21% Increase of 
6.3%
-2.1% (insig.)
Self-help group Probability of “yes” 16.4%/ 19% Increase of 
9.5%
-1.1% (insig.) The effect of the CD on joining 
a self-help group was more 
pronounced for women, 8.3% 
for men, 10.9% for women
Other Probability of “yes” 1%/ 1% 0.1% (insig.) 0.2% (insig.)
Note: This table summarizes the main results. More detailed results are available in the appendix. Results marked as “insig.” do not meet the standard measure of 
statistical significance of p <.1 in a two-tailed test. Statistically significant results are in bold. The CD baseline measures and analysis are based on 12 villages (six village 
pairs) and the deliberative session baseline measures and analysis are based on the 9 villages that received the CD treatment. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-31
Table 7: Engagement in the Political Process II, Summary of CD and Deliberative Session + CD Results 
Survey Question Outcome 
Variable 
Baseline CD 
/ Deliberative 
Session
Estimated 
Effect of CD 
Estimated Effect 
of Deliberative 
Session + CD
Other Relevant Findings
EPP4: I’m going to list some other actions people sometimes take as citizens. I will mention some of these actions and ask you to please tell me if you have 
already done it (in the last ten years), would do it, it is not allowed, or would never do it?
File a case in court
Probability of “I 
would do it” or “I 
have already 
done it.” All 
other responses 
0.
37%/ 49% Increase of 9% -2% (insig)
Deliberative session effect negative 
among men (-6.4%), borderline 
significant, p=.103)
Attend a political party event 50%/ 58% Increase of 15% -1.4% (insig)
Participate in a peaceful demonstration
22%/ 28% Increase of 4.4% -2.8% (insig)
Participate in a strike in the workplace 17%/ 20% Increase of 2.5% +3% (insig) CD effect significant only for 
women, 6.04%
Sign/thumb-print a petition 50%/ 55% Increase of 12% -1.1% (insig)
Write a letter/complaint to a 
government authority 34%/ 40% Increase of 9% 2.6% (insig) Deliberative session effect among 
women (6.5%)
Attend a Commune Council meeting 83%/ 86% Increase of 4% 0.4% (insig)
Speak at a Commune Council Meeting 50%/ 48% -2.1% (insig.) +2% (insig)
File a Complaint with the Commune 
Council
35%/ 36% +4% (insig). -0.2% (insig)
Deliberative session has a negative 
effect among men and positive 
effect among women (difference is 
statistically significant. Among 
men, -6.6%, p score =.101, women, 
0.0579, p score (.129)
Write a letter or contact a MNA 28%/ 35% Increase of 11% -1.5% (insig)
Visit the National Assembly or the 
provincial office 34%/43% Increase of 11% 0.5% (insig)
Distribute information on political 
issues. 30%/ 29% Increase of 8% 1.8% (insig)
EPP4 INDEX
Percent “I would 
do it” or “I have 
already done it.”
34%/ 39% Increase of 6.3% -0.4% (insig)
The CD makes women more likely 
to file a case in court, participate in 
a demonstration, participate in a 
strike, sign a petition, and write a 
letter or contact an MNA.
Note: This table summarizes the main results. More detailed results are available in the appendix. Results marked as “insig.” do not meet the standard measure of 
statistical significance of p <.1 in a two-tailed test. Statistically significant results are in bold. The CD baseline measures and analysis are based on 12 villages (six village 
pairs) and the deliberative session baseline measures and analysis are based on the 9 villages that received the CD treatment. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-32
Table 8: Engagement in the Political Process III, Summary of CD and Deliberative Session + CD Results 
Survey Question Outcome 
Variable 
Baseline CD 
/ 
Deliberative 
Session
Estimated 
Effect of CD 
Estimated Effect of 
Deliberative 
Session + CD
Other Relevant Findings
EPP5: People sometimes need help to resolve personal, family, or village problems, or problems with government officials and government policies. For the 
following people and offices please tell me if you have already, would consider, are not allowed, would never, or do not know if you would contact 
them about these types of problems.
Village Chief
Probability of “I 
would do it” or “I 
have already 
done it.” All 
other responses 
0.
89%/ 94% Increase of 6.8% +0.2% (insig)
Commune Councilor/ Commune Chief 86%/ 93% Increase of 9.9% +0.2% (insig)
District official/ District Council 68%/ 81% Increase of 15% -0.3% (insig)
Provincial official/ Provincial Council 62%/ 76% Increase of 13% +1.0% (insig)
NGO office 65%/ 80% Increase of 18% -0.1% (insig)
Political Party Office 48%/ 65% Increase of 19% +2.2% (insig)
Commune or District Administrative 
Police
74%/ 86% Increase of 9.4% +1.6% (insig)
Member of the National Assembly 61%/ 81% Increase of 22% -0.9% (insig)
EPP5 INDEX 69%/ 82% Increase of 
11.4%
-0.2% (insig)
Note: This table summarizes the main results. More detailed results are available in the appendix. Results marked as “insig.” do not meet the standard measure of 
statistical significance of p <.1 in a two-tailed test. Statistically significant results are in bold. The CD baseline measures and analysis are based on 12 villages (six village 
pairs) and the deliberative session baseline measures and analysis are based on the 9 villages that received the CD treatment. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-33
Table 9: Responsiveness of MNAs, Summary of CD and Deliberative Session + CD Results 
Survey Question Outcome Variable Baseline 
CD / 
Deliberative 
session
Estimated 
Effect of CD 
Estimated 
Effect of 
Deliberative 
Session +CD
Other Relevant Findings
Do your MNAs communicate 
with your community about 
what they are doing in 
parliament?
Probability of “yes” 16% Increase of 
27%
+0.2% (insig) Respondents were more likely to 
say “through public meetings and 
forums” and less likely to say 
“through local government 
offices” or “by visiting by home or 
village.” Other options were 
unchanged by the CD.
Can you think of any MNAs 
activities on behalf of your 
community in the last 6 
months?
Probability of “yes” 8.5% Increase of 
15%
1.3% (insig) When asked an open ended 
follow-up, the responses that 
became more likely because of the 
CD were “building a pagoda, road, 
health center, or school;” 
“meeting with commune 
authorities;” “meeting with village 
chief;” a “political party meeting,” 
or “visit my community.” 
Note: This table summarizes the main results. More detailed results are available in the appendix. Results marked as “insig.” do not meet the standard measure of 
statistical significance of p <.1 in a two-tailed test. Statistically significant results are in bold. The CD baseline measures and analysis are based on 12 villages (six village 
pairs) and the deliberative session baseline measures and analysis are based on the 9 villages that received the CD treatment.
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-34
Table 10: Priority Issues, Summary of CD and Deliberative Session + CD Results 
Survey Question Outcome 
Variable 
Baseline 
CD / 
Deliberative 
Session
Estimated 
Effect of CD 
Estimated 
Effect of 
Deliberative 
Session + CD
Other Relevant Findings
Of the following issues, which are the top three you would like your Members of the National Assembly to 
take action on?
Land Seizure
Probability 
of being in 
top three
53%/ 49% +6.4 (insig.) +1.9 (insig.)
Lack of transparency (extortion and 
corruption)
26%/ 25% Decrease of 
8.3%
-2.5 (insig.)
Agriculture-related problems 19%/ 17% +0.3 (insig) -2.8 (insig.) Deliberative session negative 
effect significant for men, -6.4%, 
p=0.053)
Lack of infrastructure (roads) 40%/ 52% Increase of 
22.3%
+1.7 (insig.) CD effect stronger for women, 
26% for women, 17% for men)
Inability of farmers to pay off debts (high 
interest rates of local banks)
9%/ 11% Decrease of 
4.2%
-2.5 (insig.) Deliberative session negative 
effect significant for women (-
3.7%, p=0.095)
High price of commodities (electricity and 
oil)
42%/ 50% +0.6 (insig) -2.1 (insig.)
Police and local officials discrimination / 
unfair practices based on party affiliation
11%/ 8% Decrease of 
3.9%
+1 (insig.) CD effect 2.5% for men, 5% for 
women
Vietnamese-Cambodia border (influx of 
Vietnamese workers)
12%/ 13% -0.3 (insig) -2.2 (insig.)
Leaders don’t represent citizens 5.4%/ 4% -1.4 (insig) 0.0 (insig.)
Illegal logging and fishing 21%/ 20% +2.4 (insig) +0.8 (insig.)
Poor quality of education 6.8%/ 5% Decrease of 2% +1.6 (insig.) CD effect led by men, decrease of 
3.9% significant, women 0% 
change not significant
Political party platforms don’t fight poverty 10%/ 10% Decrease of 
1.7%
+1 (insig.)
Lack of security (youth gangs) 37%/ 32% Decrease of 
10%
+4 (insig.) CD effect only significant for 
women -18%. Deliberative 
session effect significant for men, 
+8.9%,( p=0.02)
Note: This table summarizes the main results. More detailed results are available in the appendix. Results marked as “insig.” do not meet the standard measure of 
statistical significance of p <.1 in a two-tailed test. Statistically significant results are in bold. The CD baseline measures and analysis are based on 12 villages (six village 
pairs) and the deliberative session baseline measures and analysis are based on the 9 villages that received the CD treatment.
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-35
Table 11: Group Type and Facilitator Gender Survey Analysis 
Survey Question Outcome Variable Effect of Female 
Group 
Effect Male 
Group
Effect of Mixed 
Group
Effect of Male 
Facilitator 
Knowledge Index I Percent “yes” of KPP1-KPP5 (0-1) -4.7% (insig.) +5.2% (insig.) -0.6% (insig.) +5.4% 
(borderline 
significant)
Knowledge Index 2 Percent correct of KPP6_1 to KPP6_6 (0-1) +0.0 (insig.) +2.7% (insig) -2.4% (insig.) +0.6% (insig)
Confidence in the Political Process 
Index
Average score on 5 point scale ranging 
from Strongly Disagree to Strongly Agree
+0.47 
(borderline 
significant)
-0.03 (insig.) -1.0% (insig.) Decrease of 0.05
FP1: How interested are you in 
politics? 
I reject politics/Not Interested/Don’t 
know/ Somewhat Interested/Very 
Interested (-2 to 2)
+0.08 (insig.) +0.09 (insig.) -0.08 (insig.) -1.9% (insig.)
FP2: How often do you talk about 
politics with other people, such as 
friends or neighbors?
Don’t Know/Never/ Rarely/ A few times a 
month/ A few times a week/ Everyday
-0.17 (insig.) +0.09 (insig.) 0.06 (insig.) +0.12 (insig.)
FP3: Would you have a hard time 
discussing controversial issues with 
your friends or neighbors if you had 
different political opinions?
Don’t Know/ Very hard/ A bit hard/ Not 
too hard/ Not hard at all
-0.18 (insig.) -0.03 (insig.) +0.12 (insig.) 0.04 (insig.)
EPP1: Did you vote in the last national 
election in 2008?
Probability of “yes” -0.2% (insig.) -2.5% (insig.) +2.3% (insig.) -2.9% (insig.)
EPP2: Do you intend to vote in the 
Commune Council election in 2012?
Probability of “yes” Increase of 2% +1.3% (insig.) Decrease of 
2.8% 
0.4% (insig.)
Engagement with the Political Process 
Index I (EPP3)
Average probability of “yes” for group 
membership
-1.3% (insig.) 1.5% (insig) -0.3% (insig.) 0.5% (insig.)
Engagement with the Political Process 
Index II (EPP4)
Probability of “I would do it” or “I have 
already done it.” All other responses 0.
-0.00 % (insig.) -0.7% (insig.) +0.6% (insig.) -0.4% (insig.)
Engagement with the Political Process 
Index III (EPP5)
Probability of “I would do it” or “I have 
already done it.” All other responses 0.
Increase of 4.7% -1.0% (insig.) -2.8% (insig.) -0.8% (insig.)
IMP1: Do your MNAs communicate 
with your community about what they 
are doing in parliament?
Probability of “yes” -1.0% (insig.) +4.9% (insig.) -3.4% (insig.) +2.0 % (insig.)
IMP3: Can you think of any MNAs 
activities on behalf of your community 
in the last 6 months?
Probability of “yes” -2.8% (insig.) Increase of 6.0% -2.9% (insig.) +3.3% (insig.)
Note: There is no true control group in the group type or facilitator gender analyses. Rather, the comparisons are to the other types. The “Female Group” comparisons 
are the estimated effect of being female in a female group relative to mixed and all male groups. The “Mixed Group” comparisons are the estimated effect of being in a 
mixed group relative to an all-female or all-male group. The effect of having a male facilitator is compared to having a female facilitator. Results marked as “insig.” do not 
meet the standard measure of statistical significance of p <.1 in a two-tailed test. Statistically significant results are in bold. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-36
Table 12: Group Type and Facilitator Gender Effects on Matrix Voting 
Characteristic Male Group 
(relative to all other 
group types)
Female Group 
(relative to all other 
group types)
Mixed Group 
(relative to all other 
group types)
Effect of Male Facilitator 
(relative to female 
facilitator)
Effect on External Solution Vote Share
(Mean = 64.4%) Increase of 9.6% Decrease of 8% - 1.8% (insig.) Increase of 16.1%
Effect on On-Diagonal Voting
(Mean = 48.6%) + 6.7% (insig.) - 4% (insig.) - 2.5% (insig.) + 5.6% (insig.)
Effect on Vote Clustering
(Mean = 6.63 Votes) +1.46 votes (insig.) - 1.72 votes (insig) +.17 votes (insig) Increase of 2.29 votes
Effect on Max Solution Vote Share
(Mean = 30.3%) + 2.6% (insig) - 4.1% (insig) + 1.1% (insig.) Increase of 5.5%
Note: Results marked as “insig.” do not meet the standard measures of statistical significance of p <.1 and a two-tailed test. Statistically significant results are in bold. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-37
Table 13: Facilitator Bias in Deliberative Session Matrix Voting, Summary of Results 
Characteristic
Facilitator 
1
(Male)
Facilitator 
2 
(Female)
Facilitator 
3
(Male)
Facilitator 
4
(Male)
Facilitator 
5 
(Female)
Facilitator 
6 
(Female)
Facilitator 
7 
(Female)
Facilitator 
8 
(Female)
Facilitator 
9
(Male)
Facilitator 
10
(Male)
Percent of Sample 10.3 9 10.3 10.3 11.5 9 10.3 9 9 11.5
Effect on External 
Solution Vote Share
Mean= 64.4%
Increase 
of 20%
-2.6% 
(insig.)
3.2% 
(insig.)
+ 5.9% 
(insig.)
- 5.9% 
(insig.)
Decrease 
of 12.3%
Decrease 
of 12.8%
Decrease 
of 12.6%
Increase 
of 9.7% 
+ 5. 4% 
(insig.)
Effect on On-Diagonal 
Voting
Mean = 48.6%
- 8.4% 
(insig.)
- 4.6% 
(insig.)
Increase 
of 18.6% 
Increase 
of 15.4%
Decrease 
of 10.1%
+ 8.6% 
(insig.)
+ 4.6% 
(insig.)
Decrease 
of 13.5%
Decrease 
of 18.7%
+ 5.5% 
(insig.)
Effect on Vote 
Clustering Mean = 
6.63 Votes
+0.13
votes 
(insig.)
- .95 votes
(insig)
Increase 
of 3.76
votes
+ 1.08
votes 
(insig.)
Decrease 
of 2.5
votes
+ 1.74
votes 
(insig.)
- .94 votes 
(insig.)
Decrease 
of 3.61
votes
-.24 votes
(insig.)
+ 1.31
votes 
(insig.)
Effect on Maximum 
Solution Proportion
Mean = 30.3%
+ 3% 
(insig.)
- .02% 
(insig.)
+ 7.8% 
(insig.)
Decrease 
of 3.3%
- 2.6% 
(insig.)
0.01% 
(insig.)
Decrease 
of 5.5 %
Decrease 
of 4.4%
Increase 
of 6.9%
+ 1.1 % 
(insig.)
Note: Results marked as “insig.” do not meet the standard measures of statistical significance of p <.1 and a two-tailed test. Statistically significant results are in bold. 
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-38
Works Cited 
Beath, Andrew, Fotini Christia, and Ruben Enikolopov. 2012. Direct Democracy and Resource Allocation: 
Experimental Evidence from Afghanistan. SSRN eLibrary. Available from 
<http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1935055>. . Accessed17 August 2012. 
Brown, Stephen. 2005. Foreign Aid and Democracy Promotion: Lessons from Africa. The European Journal of 
Development Research 17 (2): 179–198.
Bush, Sarah. 2011. International Politics and the Spread of Quotas for Women in Legislatures. International 
Organization. Available from <http://ssrn.com/abstract=1608004>. 
Carothers, Thomas. 2004. Critical Mission: Essays on Democracy Promotion. Carnegie Endowment for 
International Peace. 
Carothers, Thomas. 1997a. Democracy assistance: The question of strategy. Democratization 4 (3): 109–132. 
Carothers, Thomas. 2006. The Backlash Against Democracy Promotion. Foreign Affairs 85 (2): 55. 
Carothers, Thomas. 1997b. The Observers Observed. Journal of Democracy 8 (3): 17–31.
Crawford, Gordon. 2001. Foreign Aid and Political Reform: A Comparative Analysis of Democracy Assistance 
and Political Conditionality. New York: Palgrave Macmillan. 
Fearon, James D., Macartan Humphreys, and Jeremy M. Weinstein. 2009. Can Development Aid Contribute to 
Social Cohesion after Civil War? Evidence from a Field Experiment in Post-conflict Liberia. American 
Economic Review 99 (2): 287–291.
Finkel, Steven E., Aníbal S. Pérez Liñan, and Mitchell A. Seligson. 2007. The Effects of U.S. Foreign Assistance 
on Democracy Building, 1990–2003. World Politics 59 (3): 404–440.
Finkel, Steven E., Aníbel Pérez-Liñán, Mitchell A. Seligson, and Dinorah Azpuru. 2006. Effects of US Foreign 
Assistance on Democracy Building: Results of a Cross-national Quantitative Study. 
Gillespie, Richard, and Richard Youngs. 2002. Themes in European Democracy Promotion. Democratization 9
(1): 1. 
Goldsmith, Arthur A. 2008. Making the World Safe for Partial Democracy? Questioning the Premises of 
Democracy Promotion. International Security 33 (2): 120–147.
Humphreys, Macartan, and Jeremy M. Weinstein. 2009. Field Experiments and the Political Economy of 
Development. Annual Review of Political Science 12. 
Humphreys, Macartan., William A. Masters, and Martin E. Sandbu. 2006. The Role of Leaders in Democratic 
Deliberations: Results from a Field Experiment in São Tomé and Príncipe. World Politics 58 (4): 583–
622.
Hyde, Susan D. 2010. Experimenting in Democracy Promotion: International Observers and the 2004 
Presidential Elections in Indonesia. Perspectives on Politics 8 (02): 511–527.
Knack, Stephen. 2004. Does Foreign Aid Promote Democracy? International Studies Quarterly 48 (1): 251–
266.
McCargo, Duncan. 2005. Cambodia: Getting Away with Authoritarianism? Journal of Democracy 16 (4): 98–
112.
McFaul, Michael. 2009. Advancing Democracy Abroad: Why We Should and How We Can. First Edition. 
Rowman & Littlefield Publishers. 
McFaul, Michael. 2004. Democracy Promotion as a World Value. The Washington Quarterly 28 (1): 147–163.
Moehler, Devra C. 2010. Democracy, Governance, and Randomized Development Assistance. The ANNALS of 
the American Academy of Political and Social Science 628 (1): 30–46.
National Research Council, Committee on Evaluation of USAID Democracy Assistance Programs. 2008. 
Improving Democracy Assistance: Building Knowledge Through Evaluations and Research. Washington, 
DC: The National Academies Press. 
Olken, Benjamin A. 2007. Monitoring Corruption: Evidence from a Field Experiment in Indonesia. Journal of 
Political Economy 115 (2): 200–249.
Ottaway, Marina, and Thomas Carothers. 2000. Funding Virtue: Civil Society Aid and Democracy Promotion. 
Carnegie Endowment for International Peace. 
Paluck, Elizabeth Levy, and Donald P. Green. 2009. Deference, Dissent, and Dispute Resolution: An 
Experimental Intervention Using Mass Media to Change Norms and Behavior in Rwanda. American 
Political Science Review 103 (04): 622–644.
Pevehouse, Jon C. 2002. With a Little Help from My Friends? Regional Organizations and the Consolidation of 
Democracy. American Journal of Political Science 46 (3): 611–626.
ANNEX I to CEPPS NDI Cambodia Impact Evaluation Report Page I-39
Schraeder, P. J. 2003. The State of the Art in International Democracy Promotion: Results of a Joint European￾North American Research Network. Democratization 10 (2): 21–44.
Wantchekon, Leonard. 2003. Clientelism and Voting Behavior: Evidence from a Field Experiment in Benin. 
World Politics 55 (3): 399–422. 
Wantchekon, Leonard. 2011. Deliberative Electoral Strategies and Transition from Clientelism: Experimental 
Evidence from Benin. Manuscript. Princeton. Available from 
<http://www.princeton.edu/rppe/speaker-series/speaker-series-2011-12/WantchekonDraft.pdf>. . 
Accessed26 November 2012. 
Youngs, Richard. 2001. The European Union and the Promotion of Democracy. Oxford: Oxford University Press. 
ANNEX II to CEPPS NDI 
Cambodia Impact 
Evaluation Report
NDI Cambodia: Qualitative Findings from a 
Random Control Trial
Linda Stern, Director of Monitoring, Evaluation & Learning
Feburary 2013

ANNEX II[Type text] 
NDI Cambodia: Qualitative 
Findings from a Random Control 
Trial
National Democratic Institute 
 
Linda Stern, Director of Monitoring, Evaluation & Learning
This is a companion report to the quantitative findings from a field experiment 
conducted on NDI’s Multiparty Constituency Dialogue program in Cambodia from 
2011-2012. The report presents the qualitative findings from citizen deliberative 
sessions held the day before the Constituency Dialogue event.
Companion 
Report
February 2013
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-1 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-2 
The findings for the qualitative report were developed by the Monitoring, Evaluation & Learning team 
(MEL) at the National Democratic Institute. Linda Stern designed the proposed random control trial, 
program variation and qualitative data collection and analysis protocols in consultation with the Asia team 
and the Principle Investigator, Susan Hyde. Molly Watts served as research assistance during the data 
collection process in Cambodia, and later as the quantitative data analyst on the MEL team in DC. 
Ornanong Maneerattana served as the qualitative data analyst on the MEL team, supporting the 
management and coding of all qualitative datasets presented in this report. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-3 
TABLE OF CONTENTS 
I. EXECUTIVE SUMMARY .................................................................................................................. 4
II. PROGRAM VARIATION – CITIZEN DELIBERATION .............................................................. 9
DEMOCRATIC DELIBERATION – THEORY AND PRACTICE ........................................................................ 10
III. METHODOLOGY .......................................................................................................................... 17
QUALITATIVE METHODS: ......................................................................................................................... 17
RELEVANT HYPOTHESIS QUESTIONS: ..................................................................................................... 18
IV. QUALITATIVE FINDINGS: ......................................................................................................... 20
A. DELIBERATIVE SESSIONS .................................................................................................................. 20
B. CONSTITUENCY DIALOGUES ............................................................................................................. 32
V. DISCUSSION OF FINDINGS ......................................................................................................... 46
A. TESTING A VARIATION IN THE CD PROGRAM –DELIBERATIVE SESSIONS ..................................... 46
B. PROGRAM HYPOTHESIS – CONSTITUENCY DIALOGUE PROGRAM .................................................... 48
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-4 
I. Executive Summary 
In 2009 NDI had the opportunity to design and implement a random control trial of its long-standing 
Multiparty Constituency Dialogue (CD) program in Cambodia. The CD program was based on a theory 
of change which posits that the creation of new and competitive political space, where citizens can voice 
their concerns, will inform policy and increase representatives’ accountability to their constituents. Along 
with understanding the effects of CD attendance on citizens attitudes and knowledge, NDI wanted to 
know what would happen if rural citizens had more time and space before the constituency dialogue to 
formulate their concerns, deliberate on solutions and build consensus and confidence with their peers on 
the issues that should be brought to the attention of the MNAs. To that end NDI varied the program so 
that some citizens would participate only in the CD and others would participate in an additional 
deliberative session before the CD. NDI used a mixed-methods approach to collect both quantitative and 
qualitative data within the experimental framework. 
The field experiment used a survey to test four of the eight hypothesis questions posed in the study, the 
results of which are covered in the companion report authored by Dr. Susan Hyde. The qualitative 
findings in this companion report draw on data from the deliberative sessions to address three of the eight 
hypothesis questions (#3, 5 and 7) posed in the random control trial. It should be noted that the 
qualitative data under hypothesis question three serve as descriptive data on citizens’ priorities and cannot 
be considered experimental. However, because qualitative data were gathered on all deliberative groups 
that participated in the program variation, these data do speak to the experimental findings on the effects 
of gender, group composition and facilitator under hypothesis questions 5 and 7. In this way the 
qualitative report provides a context for the quantitative findings – particularly for citizen priorities--
lends insight into NDI’s program innovation in citizen deliberation; and speaks to the broader program 
hypothesis and theory of change for constituency dialogue programs. 
Based on results from 80 deliberative sessions; observational data from nine CD events; and facilitator 
documentation of both the CDs and deliberative groups, the following offers a summary of findings and 
their implications: 
Hypothesis Question #3: How did participation in a deliberative session before the CD impact 
citizens’ priorities? 
1. Priority issues for citizens who participated in the deliberative session before the CD were: lack of 
infrastructure, land seizures, corruption, high cost of living, water & sanitation and health clinics. 
Many of the same issues were ranked among citizens’ top priorities on the baseline survey (e.g., lack 
of infrastructure, land seizure, corruption, high cost of living). However, when asked to deliberate 
with their peers citizens placed lesser importance on some priorities (e.g., Vietnamese migration, 
illegal logging and fishing); placed more emphasis on other priorities (e.g., education); identified new 
priorities (e.g., water & sanitation, health clinics inter alia); and omitted other priorities altogether 
(e.g., leaders don’t represent citizens; political party platforms don’t fight poverty). 
2. The issues identified through the collaborative, matrix-ranking process used during the deliberative 
sessions revealed the interrelated nature of citizens’ problems. The qualitative data illuminated the 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-5 
way in which these issues impact rural Cambodian’s lives, livelihoods, families, communities and the 
natural environment. 
3. Matrix-identified priorities from the deliberative sessions varied slightly by province and commune, 
but serious corruption cut across communes and took many forms – corruption in service delivery, in 
distribution of humanitarian aid and in land concessions. Of particular note were those communes 
where land seizure was an issue. In these deliberative sessions citizens recounted dispossession of 
rural families by private companies, clear cutting of forests, killing of livestock, chemical pollution of 
water, and violent retribution and imprisonment for citizens who mobilized in protest – all with the 
support of local authorities. 
Hypothesis Question #5: How did the gender composition of the deliberative groups 
differentially impact the priority issues of the participants? 
4. The gender composition of the deliberative sessions influenced how groups prioritized three of the 19 
problems identified using a matrix. During deliberation all-male groups tended to place a higher 
priority on the lack of infrastructure and were less likely than all other groups to rank corruption as a 
top priority. Mixed groups were also more likely than any other group types to identify lack of health 
clinics among their top priorities. 
Hypothesis Question #7: Did the gender of the facilitator differentially influence the priority 
issues of participants? 
5. The gender of the facilitator also caused a difference in the way in which a deliberative group ranked 
three of the 19 problems identified using a matrix, with male-facilitated groups more likely than 
female-facilitated groups to vote for lack of infrastructure; and less likely to vote for high cost of 
living and access to markets as a top priority. The gender of the facilitator also influenced how 
groups voted on solutions to problems. While all groups tended to identify external solutions (those 
that required MNA action) over internal solutions (those that required citizen action) groups 
facilitated by a male tended to identify external priorities more often than other groups. Furthermore, 
when compared to female-facilitated groups, male-facilitated groups tended to vote in consensus, 
“clustering” their votes for single solutions to individual problems, as well as for their top three 
priority solutions. 
How did the deliberative group participants behave at the CD? 
6. Most deliberative groups agreed that they would raise their priorities with their MNAs at the CD 
event the next day. Although only 65% of those citizens were able to attend their CDs, the evaluation 
team found that on the day of the Constituency Dialogue those citizens who had participated in the 
deliberative sessions were more active than the citizens who had not participated in a deliberative 
session. Deliberative session participants raised their hands and spoke at their CD events more than 
twice as often as citizens who had not participated in a deliberative session. Women who had 
participated in deliberative sessions spoke at their CDs one and a half times more often their female 
counterparts; and male participants in deliberative sessions raised their hands and spoke at their CD 
events two and half times more often than men who had not attended a deliberative session before the 
CD. These findings cannot be considered experimental since deliberative session participants self-
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-6 
selected their attendance at the deliberative session and the CD. However, the observational data 
suggests that participation in the deliberative sessions contributed to differences in citizen 
engagement at the constituency dialogue the next day. 
What was the environment at the CD event? 
7. In general there was a healthy balance between opposition and ruling party at all nine CDs and for the 
most part MNAs respected the a code of conduct to interact civilly at the CD. However, the CD 
environment varied by commune and mapped with many of the issues identified during the 
deliberative sessions. Of particular note was the strong police presence at three of the nine CDs, and 
in many cases their active attempts to intimidate attendees who raised concerns. In all three of these 
communes corruption was ranked among the deliberative groups’ highest matrix priorities and were 
related to land seize and/or service delivery. 
Implications of Findings:
The USAID-sponsored random control trial provided NDI with the opportunity to test a programmatic 
innovation in its long-standing constituency dialogue program. Together the analysis of the descriptive 
and causal data provide insights into micro-level democratic processes and the ways in which the context 
as well as gender inform and influence citizens’ deliberations, as well as their subsequent participation in 
the constituency dialogues. In this way the findings contribute to NDI’s understanding of: a) democratic 
deliberation; and b) the broader program hypothesis and theory of change for constituency dialogue 
programs. 
a) Program Innovation – Citizen Deliberation 
Proponents of deliberative democracy posit that public reasoning and debate not only enhance social 
cohesion, but create outcomes of better citizens, better decisions and better or more legitimate systems. 
In contrast to aggregative democracy in which preferences are formed in private and then expressed and 
added together in public through polling, focus groups and elections, deliberative democracy provides a 
fair and transparent forum for citizens to identify and solve concrete problems despite their differences. 
Moreover, experiments in deliberative polling in 15 countries have demonstrated that citizen deliberation 
enhances citizen knowledge of issues and politics; strengthens their ability to formulate opinions; and 
stabilizes opinions on issues once they are formulated.1
 At the same time, other experiments2
in citizen 
deliberation have demonstrated that deliberative processes can be influenced by gender norms and 
inequities, while deliberative outcomes can be influenced by moderators of deliberative forums. 
Unlike experiments in deliberative polling, this study’s survey results did not find that deliberative 
session participants improved their knowledge of issues, politics or confidence beyond the effect created 
by participation in the constituency dialogue alone (see companion report). However, the qualitative 
details from the deliberative sessions did align with the proposition that purposeful peer discussion and 
 
1
 Fishkin, James S. and Luskin, Robert C. (2005). “Experimenting with a Democratic Ideal: Deliberative Polling 
and Public Opinion.” Acta Politica, 2005, 40, (284-298). 
2
 Macartan Humphreys, William A. Masters and Martin E. Sandbu (2006). The Role of Leaders in Democratic 
Deliberations: Results from a Field Experiment in São Tomé and Príncipe. World Politics, 58, pp 583-622.
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-7 
analysis is substantively different than polls and focus groups that measure citizens’ opinions of the issues. 
Rather, the deliberative sessions yielded outcomes that reflected citizen judgment of their priorities after 
careful reflection and discussions of the issues and their solutions. Moreover, the experimental findings 
demonstrated the importance of creating deliberative space for marginalized groups within the population 
so that social norms and inequities such as gender do not unduly influence the deliberative process or its 
outcomes. Lastly, the difference in activity level at the CDs between deliberative session participants and 
non-participants, although not experimental, underscored the value-added of citizen deliberation before 
the CD. 
These findings have implications for future programming designed to strengthen the demand side of the 
democracy equation. Future investment in citizen engagement should move beyond simply extracting 
information from citizens via focus groups and/or opinion polls, to more purposefully investing in citizens’ 
capacity to formulate and press their concerns in the public sphere. While deliberative sessions would 
enhance the quality and reliability of information on citizens’ priorities, they should not be used as a 
stand-alone intervention or as an appendage to the current program. Given the risks that citizens run in 
discussing issues like corruption and intimidation, it would not be prudent to simply extract information 
from citizens through deliberative session of focus groups. Rather the deliberative process should be seen 
as a foundation for bolstering and supporting citizen voice, agency and empowerment to re-negotiate 
relationships of power within and outside their communities. Indeed, the deliberative sessions, when 
structured along gender lines, have an intrinsic value of strengthening the public voice of women and men 
within their communities. As such, support for citizen deliberation should be considered a starting point 
for more comprehensive programming for strengthening the demand side of the democracy equation in 
Cambodia, and not simply to serve the informational needs of MNAs involved in NDI’s CD program. 
b) Program Hypothesis – Constituency Dialogues 
The broader experimental study was meant to better understand the demand side of the constituency 
dialogue program’s results chain – if constituents better understand their roles and options within the 
democratic process; and if constituents create pressure on MNAs to address their concerns; then 
constituents will contribute to more responsive and accountable MNAs. The random control trial 
demonstrated that participation in the CD did indeed improve constituents’ understanding of their roles 
and option in the democratic process – the first link in the demand side of the program’s results chain. 
However, the experiment did not demonstrate that this new knowledge was linked to citizens putting 
additional demands on their MNAs to address their concerns – the next link in the results chain. The 
deliberative sessions were a relatively modest program activity, and the descriptive findings suggest that 
providing time and space for citizens to discuss and come to consensus on their concerns may contribute 
to increasing their activity during the CD event. However, these findings were not experimental as 
participation in the CD event was self-selected and not randomized, and therefore not conclusive. Further 
data are needed to explore a) whether changes in knowledge and attitudes were sustainable beyond the 
study’s timeline; and b) whether differences in citizen activities at the CD are due to citizen deliberation 
or some other variable. 
In addition, the broader context of the intervention and its variation is important to take into account. 
While the nine CD events presented a picture of healthy political competition in which ruling and 
opposition parties adhered to a code of civility, the details from the deliberative sessions documented a 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-8 
context of corruption, intimidation and sometimes violent retribution for citizens who raised their voices 
at the local level. The deliberative sessions and the CDs create temporary space for citizens to voice their 
concerns, but individuals do so in the context of great personal risk. Even when forums are hosted by an 
international entity such as NDI or USAID, a third of the constituency dialogues in this study had a heavy 
police presence where observers documented attempts to intimidate the attendees. During the deliberative 
sessions a number of facilitators also documented veiled intimidation by local officials during the small 
group sessions. The program’s critical assumption that citizens’ improved understanding of democracy 
will lead to increased pressure on MNAs to address their concerns did not hold true in this study. 
The CD program is creating increased knowledge, along with temporary space and opportunity for 
citizens’ voices to emerge. However, the broader intervention strategy should be reviewed to identify 
ways to more systematically strengthen accountability between local or national representatives and their 
constituents where feasible in the Cambodian context. Although the CD program records and tracks the 
promises MNAs make to their constituents during the CDs – and some positive outcomes have emerged 
from this process – the lines of accountability are between the MNAs and NDI, and not between MNAs, 
local officials and their constituents. While the ‘thin edge of the wedge’ strategy to creating new political 
space and increasing accountability is an effective entry point for a new program, the strategy is not 
sustainable and any gains in accountability and voice may be at risk of disappearing at the end of the life 
of the program. In the long run the public performance of political competition and debate may 
contribute to a change in public discourse, citizen expectations and socio-political norms. However, 
against a backdrop of state capture, systemic corruption and intimidation, and no real mechanisms for 
accountability between citizens and their representatives, or public officials and the judiciary, the 
improved public dialogue of MNAs and their constituencies may not be contributing to real or lasting 
change. . 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-9 
II. Program Variation – Citizen Deliberation
NDI’s long-standing Constituency Dialogue (CD) program is based on a Theory of Change which posits 
that the creation of new and competitive political space, where citizens can voice their concerns, will 
inform policy and increase accountability between representatives and their constituents. 
Figure 1: Results Chain for the Multiparty Constituency Dialogue Program 
In 2004 NDI designed its multiparty CDs to engage Members of the National Assembly (MNAs) and 
their constituents in dialogue on issues that concern citizens at the local level. By including MNAs from 
multiple parties, citizens not only have the opportunity to publically voice their concerns to 
representatives, but to hear the perspectives of different political actors. NDI believed this forum would 
not only increase citizens’ understanding of the political process, but their understanding of their political 
options within Cambodia’s emerging democracy. For the MNAs, the multiparty constituency dialogues 
not only provide a forum for hearing the concerns of their constituents, but also serve to raise MNAs’ 
awareness of political competition, thus incentivizing greater responsiveness to their constituents’ 
concerns. Briefly stated, NDI’s program hypothesis in Cambodia is: 
 IF MNAs’ perceptions of political competition are increased 
 AND IF MNAs understanding of constituents’ needs is increased 
 AND IF constituents create pressure on MNAs to address their concerns 
 THEN MNAs will be more responsive and accountable to their constituents. 
Because the program’s overarching objective was to increase the responsiveness and accountability of 
MNAs, NDI targeted the majority of its program’s inputs and activities at the supply side of the 
democracy equation – MNAs. However, a critical assumption in NDI’s program hypothesis was that 
citizens’ attendance at the CD was necessary and sufficient to not only change citizens’ attitudes and 
increase their knowledge and understanding, but change their behavior – i.e., to pressure MNAs to 
address their concerns. Although NDI’s post-CD focus groups had shown a shift in attitudes immediately 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-10
after the CDs, it was unclear if these attitudes were representative of the broader community; if they were 
sustained; or if they lead to behavioral change. In addition, NDI’s experience in other countries suggested 
that citizens need smaller discussion forums for deliberating on issues – especially in politically uncertain 
environments – before change in behavior occurs. 
Through this study, NDI wanted to explore the effects of citizens having more time and space -- before 
the constituency dialogues -- to formulate their concerns, deliberate on solutions, and build consensus and 
confidence with their peers on the issues that should be brought to the attention of their MNAs. NDI
therefore proposed using a field experiment to test an innovation in programming aimed at the demand 
side of the program’s results chain – the citizens. NDI proposed randomizing the introduction of a new, 
citizen deliberation component to the CD intervention in three multiparty provinces – Kratie, Kampong 
Cham and Kampong Chhnang. 
Democratic Deliberation – Theory and Practice
Democratic deliberation takes place in many forums, is used by diverse democratic actors, with differing 
levels of accountability and formality. In exploring the characteristic of deliberative democracy David 
Crocker juxtaposes it with aggregative democracy in which bargaining, political maneuvering, clientalism 
and agitation are essential processes in political competition, and voting is the primary mechanism for 
resolving difference. In aggregative democracy “preferences or interests are formed in private and then 
expressed and added together in public.” 3
 
In contrast, deliberative democracy uses public discussion and consensus as a means to resolve difference 
in the political sphere. Voting is not the primary means for resolving conflict, but serves as the 
culmination of the deliberative process; a closure device that summarizes the group’s discussion and 
decision making process.4
 According to Crocker, the aim of citizen deliberation is two-fold: to “identify 
and solve concrete problems or to devise general policies for solving specific problems”; and to “provide 
a fair way in which free and equal members of a group can overcome their differences and reach 
agreement about action and policy.” 5
 
Proponents of deliberative democracy argue that the resolution of competition through fair and 
transparent decision-making processes -- despite conflicting values –promotes and enhances social 
cohesion6
. Further, that the process of deliberation has three categories of benefits or effects– better 
citizens; better decisions and better or more legitimate systems.7
 
Better Citizens: Participation in public reasoning and debate enhances citizen knowledge of issues and 
politics; strengthens their ability to formulate opinions; and stabilizes opinions on issues once they are 
 
3
 Croker, David A. (2008). Ethics of Global Development: Agency, Capability, and Deliberative Democracy. 
Cambridge University Press 
4
 Richardson, Henry (2002). Democratic Autonomy: Public Reasoning About the Ends of Policy. Oxford University 
Press 
5
 ibid 
6 Ross, Carne (2011). The Leaderless Revolution: How Ordinary People Can Take Power and Change Politics in the 
21st Century. Simon & Schuster
7
 Hibbing, John and Elizabeth Theiss-Morese (2002). The Perils of Voice: Political Involvement’s Potential to De￾legitimate. Presented at the annual meeting of the American Political Science Associations, Boston, MA 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-11
formulated.8
 Moreover, inclusive participation in public deliberation promotes the emergence of a public 
“voice” for more marginalized citizens. In this way the aim of deliberative democracy is not to merely 
aggregate interest, but to transform socio-political relationships of power by establishing the legitimacy of 
citizen participation in transparent policy formation and public decision-making.9
 
Better Decisions: As a consequence of inclusive deliberation, collective decision-making is achieved 
with a broader and more diverse array of inputs. Decisions therefore reflect a fuller understanding of the 
complexities of issues, the contexts in which they occur and their multiple impacts on different members 
of society. In this way, theorists argue, deliberative democracy helps to mitigate the inequities of race, 
class and gender and lead to more equitable policy formation.10
Better Systems: Lastly, deliberative decision-making builds confidence and support for the systems and 
institutions from which they flow. Transparent and public deliberation and decision-making builds trust, 
especially where conflict is longstanding. 
While many observational studies have been done on citizen deliberation, little experimental or quasi￾experimental research has tested the proposed processes and/or outcomes of citizen deliberation outlined 
above11. Notable experiments include Fishkin’s work on deliberative polling -- a rejoinder to public￾opinion polling that takes the collective pulse of citizens through surveys and focus groups, but “measure 
little more than the public’s impressions of sound bites and headlines” that are then filtered through the 
“persuasion industry that is more Madison Avenue than Madisonian.”12
 
Fishkin and Luskin’s research in over fifteen developed countries has demonstrated the impact of 
deliberation on citizen opinions.13 Their experiments begin with a random representative sample of 
citizens who are polled to establish a baseline of opinions. A smaller number of these citizens are then 
randomly invited to take part in a weekend intervention in which they are given balanced briefing 
materials. They are further randomized into smaller deliberative groups. These deliberative groups are 
exposed to sessions with experts and political leaders, and they then take part in dialogues on the issues 
with their smaller deliberative groups. After the deliberative sessions the researchers administer an 
endline survey with the original sample of citizens to compare the change in opinions between those who 
participated in the deliberative sessions and those that did not. Fishkin’s deliberative experiments 
attempted to demonstrate the counterfactual -- how citizens’ opinion would have changed had they had 
the opportunity to become more fully informed and participative in their democracies. 
 
8
 Fishkin, James S. (1995). The Voice of the People. New Haven, CT: Yale University Press 
9 Young, Iris Marion. (2000)Inclusion and Democracy, Oxford: Oxford University Press, 
10 (e.g., Chambers 1996; Fishkin 1995; Gutmann and Thompson 1996; Habermas 1989; 1996; Cohen 1989, 74). 
11 Karpowitz, Christopher F. and Mendelberg, T. (2011) An Experimental Approach to Citizen Deliberation, 
Cambridge Handbook of Experimental Political Science: Cambridge University Press 
12 Fishkin, James S. (2006) The Nation in a Room: Turning Public Opinion into Policy Boston Review March/April 
2006
13 Fishkin, James S. and Luskin, Robert C. (2005). “Experimenting with a Democratic Ideal: Deliberative Polling 
and Public Opinion.” Acta Politica, 2005, 40, (284-298). 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-12
It is important to point out that the process of deliberation itself can influence the outcomes of citizen 
priorities. In their field experiment in Sao Tome y Principe Humphreys, Masters and Sandbu 14
demonstrate how participatory decision-making processes are vulnerable to manipulation by political 
elites. By randomizing discussion leaders in a nation-wide experiment on democratic deliberation, 
Humphreys et al found a significant leader effect on the oil revenue priorities identified in over a third of 
the deliberative sessions. Their research points to the importance of ensuring that the design and 
procedures of deliberative sessions provide an authentic opportunity for democratic dialogue. Indeed, 
Freire’s seminal work on participation15 long ago identified the pitfalls of “false dialogue” in which elites 
attempt to replace authentic deliberation with a consultative process that mimics public discourse, but 
allows for no real decision making with the poor. 
Gender can also influence the outcomes of deliberative processes. In their review of gender and 
deliberation Karpowitz et al note three important effects of gender composition on the outcomes of 
deliberative groups. First, they proffer that being in a numerical minority lowers the status of women in a 
deliberative group. Second, social norms tend to create the perception that women are less competent 
than men in discussing issues of public concern. Third, women tend to speak less when there are fewer 
women in deliberative groups.16 At the same time Paluck’s research in Southern Sudan found that trained 
moderators had an equalizing effect on deliberative participation, especially as it pertained to women. 
The presence of a trained moderator helped to ensure that women and men participated equally in small 
deliberative groups that discussed issues of constitutional change, democracy, corruption and civic 
engagement. Moreover, all moderator-led deliberative groups had higher outcomes in knowledge and 
behavior than un-moderated deliberative groups.17
Fishkin’s experiments point to the potential positive outcomes of citizen deliberation, while Paluck’s 
research indicates that effectively trained facilitators may be able to mitigate the potential for a leader 
effect seen in Sao Tome y Principe. Lastly, Karpowitz et al suggest that attention to the gender of group 
composition – and the rules for deliberative decision-making -- may mitigate inequalities inherent in a 
population. 
Deliberative Sessions 
Given the research, NDI needed to design a deliberative process that: promoted practical problem-solving; 
was free and transparent; used voting as the culmination of deliberation; mitigated the potential for 
facilitator bias; and balanced power inequities at the local level, such as gender. In addition, it had to be 
a process that was practical for a rural, largely illiterate, working population with little time to engage in 
deliberation outside of their own households and villages. 
 
14 Macartan Humphreys, William A. Masters and Martin E. Sandbu (2006). The Role of Leaders in Democratic 
Deliberations: Results from a Field Experiment in São Tomé and Príncipe. World Politics, 58, pp 583-622.
15 Freire, Paulo (1970). Pedagogy of the Oppressed Herder and Herder, NY New York 
16 Karpowitz, Christopher F., Mandelberg, T, and Shaker, L. (2012) “Gender Inequality in Deliberative 
Participation” American Political Science Review Vol. 106, No3, August 2012 
17 Paluck, E. (2009). The Impact Of The ‘Let’s Talk” Civic Education Program: Examination Of Listener Discussion, 
Attitudes, and Behavior. Evaluation Report for the National Democratic Institute, Southern Sudan. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-13
To achieve this, NDI designed a deliberative intervention inspired by participatory action research (PAR) 
approaches such as Participatory Rural Appraisal (PRA) and Participatory Learning & Action (PLA). 
Because of its emphasis on collaborative inquiry, analysis and collective action, participatory action 
research promotes deliberation, generates data, and identifies potential action for changing the conditions 
under inquiry. For this reason PAR has become a central feature in international aid aimed at sustainable 
community development, literacy, youth engagement, public health, civic engagement and participatory 
governance. 
Matrix Ranking Tool: 
To create deliberative forums for citizens to discuss issues in a fair and transparent manner, NDI trained 
facilitators to guide small groups of rural citizens through a deliberative process that used a matrix￾ranking tool. Part of a suite of participatory action research tools, the matrix helped citizens to: identify 
and discussions their communities’ problems; come to consensus on their top three priority problems; 
generate internal and external solutions for each of the top three problems; and then vote on the solutions 
that were most strategic for addressing the problems. The following describes the deliberative sessions 
held with Cambodian citizens in the rural communes of 
Kampong Cham, Kampong Chhnang and Kratie. 
STEP ONE: BRAINSTORM COMMUNITY PROBLEMS
Participants in the deliberative sessions begin by 
brainstorming the challenges they face in their 
communities. The facilitator uses photos and drawings 
as visual symbols for each problem, recording the 
groups’ discussion and descriptions on the back of the 
picture. Once the group finishes discussing the 
problem the facilitator holds up the picture and reads 
back to the group their descriptions of the problem. 
She then asks the group if there are any other aspects of 
the problem that should be recorded before the group 
moves on to discuss the next problem. 
STEP TWO: RANKING THE TOP THREE PROBLEMS
Once the deliberative group exhausts their list of 
problems, the facilitator lays out all the problem 
pictures for the group to review. S/he gives each group 
member their own unique set of colored voting buttons 
and explains that each individual will vote on their top 
three problems, distributing their buttons over three 
picture problems, according to their importance 
(weighted voting). To ensure that everyone 
understands what each individual picture represents, 
before voting the facilitator reviews each of the 
problems by picking up each picture and once again reading the description of the problem to the group. 
The facilitator then asks the group to vote, distributing their buttons in silence. Once the voting is 
STEP 2: Citizens use colored buttons to rank the top three 
problems they face in their commune
STEP 1: The group brainstorms their problems using 
photos and drawings; the facilitator notes their discussion 
and descriptions on the back
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-14
complete the group watches as the facilitator counts the votes for each problem and records the number of 
votes on the picture. She then confirms the top three problems with the group. 
STEP THREE: GENERATE INTERNAL & EXTERNAL 
SOLUTIONS
Once the group agrees on the top three problems the 
facilitator places the three problem pictures on the left 
side of a 5 x 7 matrix. Before attaching a problem to 
the matrix the facilitator shows the picture to the group, 
again reading the description of the problem recorded 
on the back of the picture. The facilitator asks the 
group to consider each problem one at a time, 
discussing the causes and consequences of the problem 
and its possible solutions. The group must come to 
consensus on two feasible solutions for each problem. 
One of the solutions is an internal solution – an action 
the community can take to address the problem; and the other is an external solution – an action the group 
would like their MNA to take to address the problem. Once the participants came to consensus on the 
two solutions, they dictate their ideas to the facilitator who records their solutions on a post-it note, 
drawing a picture or symbol to represent the group’s solution. The two solutions are placed at the top of 
the matrix. This process is then repeated with each of the three problems until the top of the matrix has 
three pairs of internal/external solutions generated in response to the group’s three priority problems. 
STEP FOUR: RANK THE SOLUTIONS FOR ACTION
Once the participants generate all six solutions to their 
priority problems the facilitator tells the group that 
they are going to vote on the solutions that are most 
important for addressing the their priority problems. 
The facilitator begins by covering up the bottom two 
problems so that only the first problem is showing. 
She then asks the participants to consider that single 
problem in relationship to all six possible solutions. 
She instructs the group to distribute their buttons 
across the solutions, weighting their votes accordingly. 
Before participants vote in silence, the facilitator once 
again reviews each of the six solutions, summarizing the group’s discussions. Participants then vote in 
silence using their unique buttons. After all participants are finished voting they watch as the facilitator 
tallies the buttons for each cell. Once each problem is reviewed against all solutions, the facilitator sums 
the votes in the solution columns to identify the top three to four solutions for action. 
STEP 3: Problem/solution matrix is used to generate a 
menu of solutions to the priority problems
STEP 4: Participants rank each problem against all six 
solutions
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-15
STEP FIVE: GROUP REFLECTION ON THE MATRIX 
Once the group completes the matrix the facilitator 
reviews the priority problems and solutions and asks 
the group: Does the matrix accurately represent the 
problems and desired solutions for citizens in your 
village? Does the matrix represent issues that group 
members would raise in the constituency dialogue the 
next day? Do the solutions reflect actions that citizens 
or MNAs could take, or both? And Why did the group 
members think that MNA solutions or citizen solutions 
received the most votes? 
STEP SIX: MATRIX DATA ENUMERATION
Once the deliberative session is complete the 
facilitator uses a set of forms to record: the 
descriptions of all problems and solutions discussed; 
the voting scores for each problem discussed; the vote 
tallies for each cell of the matrix, along with the 
column totals. Lastly, the facilitator records the 
group’s answers to the reflection questions, along with 
any additional comments or reflections from the group. 
Randomization of Deliberative Sessions 
Creating a deliberative component to the CD program allowed NDI to vary the intervention so that some 
individuals would receive: 
a) only the CD as an intervention; 
b) the CD plus a Deliberative Session before the CD; or 
c) no intervention at all. 
NDI’s assumption was that citizen participation in the small deliberative sessions would increase the 
likelihood that citizens would shift their attitudes, enhance their knowledge of democratic processes, 
reorder their priorities and increase their engagement with their MNAs during the CD. In addition, NDI 
hypothesized that the CD intervention and its deliberative variation would have differential impacts on 
men and women. In this way NDI used the USAID-sponsored field experiment to answer the following 
key evaluation questions: 
STEP 6: Facilitator records matrix data
STEP 5: The facilitator scores the voting transparently and 
then the group discusses the results 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-16
Evaluation Questions 
1. Does exposure to the CD change citizen attitudes, knowledge, and reported or observed behavior? 
Are men and women affected differently by experience with the CD? 
2. Does participation in a deliberative session before the CD change citizen attitudes, knowledge, 
and reported or observed behavior? 
3. How does participation in the deliberative sessions before the CD differentially impact the 
participants’ priority issues? 
4. Does the gender composition of the deliberative sessions differentially impact attitudes, 
knowledge, and reported or observed behavior? 
5. Does the gender composition of the deliberative group differentially impact the priority issues of 
participants? 
6. Does the gender of the facilitator of the deliberative sessions differentially impact attitudes, 
knowledge, and reported or observed behavior? 
7. Does the gender of the facilitator differentially influence the priority issues of participants? 
8. Do individual facilitators differentially impact the priorities of participants? 
The following section briefly outlines how the deliberative sessions were randomized and the qualitative 
data collected within the experiment to answer evaluation questions 3, 5 and 7. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-17
III. Methodology 
In order to understand the effects of citizen deliberation and the variable of gender, NDI randomly 
assigned treatment participants to deliberative sessions, varying the program intervention and the gender 
composition of participant groups as follows: 
Figure 2: Randomization of Participants to Delibeative Group Types 
NDI used a mixed-methods approach in which the structure of random control trial (RCT) was used to 
systematically collect and analyze qualitative data that would complement and inform the quantitative 
data collection and analysis from the survey experiment. In this way the matrix ranking process served as 
both an intervention as well as a means to generate qualitative data. 
Qualitative Methods: 
Data Collection: 
The evaluation team collected three sets of qualitative data outlined below. These datasets provided 
details on citizens’ priorities, the CD environment, the behavior of deliberative session participants 
observed at the CD, and the facilitators’ reflections on their experiences as moderators (who resided in the 
capital city Phnom Penh) of the deliberative sessions (conducted in rural communities). 
1. Deliberative Sessions: During and after the deliberative sessions, facilitators recorded the matrix 
ranking scores of each problem as well as the qualitative data detailing citizens’ discussions. These 
data included explanations of all the problems citizens discussed in small groups. Facilitators also 
recorded participants’ internal and external solutions, detailing solutions that required citizen actions 
(internal) and the solutions that required MNA action (external). In addition, the qualitative data 
captured citizens’ reflections on their matrix results, such as whether or not the group felt that the 
results reflected their communities’ priorities, and if the members of the group would raise their 
concerns at the CD to be held on the following day. 
2. CD Observations: Facilitators also served as observers at the CD on the following day. There, they 
recorded information on the CD environment using a Likert scale to rate their perceptions about: 
political party presence; police presence and citizen intimidation; MNA behavior; and citizen 
satisfaction with the responses of their local and national representatives (see appendix 2). Observers 
also recorded their qualitative observations under each of the categories. Further, the observers 
CD plus 
Deliberative Session
(80)
All-male Deliberative 
Sessions
(24)
All-female Deliberative 
Sessions (24)
Mixed gender Deliberative 
Session facilitated by a 
female
(16)
Mixed gender Deliberative 
Session facilitated by a 
male
(16)
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-18
monitored the behavior of the 3-6 deliberative session participants from their group, recording: 
attendance; gender; hand raising; speaking – and if they spoke – the issues they raised at the CD.
3. Facilitator Reflections: After each deliberative group session and CD, facilitators completed a 
reflection form that asked them to self-assess the most significant change in their: ability to facilitate 
the deliberative sessions; interactions with rural citizens; collaboration with the team of facilitators; 
understanding of the practice of democracy in Cambodia. The evaluation team also asked the 
facilitators to provide any other reflections that they felt were important. After the first round of 
deliberative group sessions, facilitators were also asked to record a compelling story they heard 
during the deliberative sessions and/or CD. These qualitative data provided additional details on 
priorities that participants raised in the deliberative sessions and the issues discussed at the CD. 
Data Analysis 
The evaluation team managed and analyzed the qualitative data using Nvivo.18 The team coded all of the 
qualitative data for recurring themes on citizen priorities, and organized and analyzed the qualitative data 
at different levels: province, commune, individual matrix results, and individual facilitators from specific 
deliberative sessions. The team utilized the matrix coding to run reports on qualitative priorities by 
attribute: group composition (all-female, all-male, and mixed gender), commune and gender of facilitator. 
This process allowed the evaluation team to triangulate between the three qualitative datasets, to 
corroborate qualitative findings, and to compare qualitative findings with the quantitative survey findings. 
Relevant Hypothesis Questions: 
The qualitative data supplemented the quantitative findings for three of the eight hypothesis questions 
under study – hypothesis questions 3, 5, and 7. It should be noted that the qualitative data under 
hypothesis question three serve as descriptive data on priorities and cannot be considered experimental. 
However, the findings under hypothesis questions 5 and 7 are experimental because qualitative data were 
gathered on all groups that were randomly assigned as part of the program variation. 
Hypothesis Question #3: How did participation in a deliberative session before the CD impact 
citizens’ priorities? 
To contribute to Hypothesis Questions #3 the evaluation team coded the top priorities of each deliberative 
group. The matrix-identified priorities were then disaggregated by province and commune to better detail 
the different issues facing citizens at each level. Again, because similar data were not collected for the 
control group, this qualitative dataset cannot speak to experimental findings under hypothesis question 
three. However, as descriptive data they are invaluable for understanding citizens’ priorities, as well as 
the context in which NDI’s intervention takes place. The evaluation team also compared the matrix￾identified priorities with survey-identified priorities to detail the qualitative and substantive differences 
between the way citizens identified and ranked priorities as individuals responding to a survey, and the 
way they identified and ranked priorities as a deliberative group using the matrix tool. 
 
18 NVivo is a qualitative data analysis (QDA) computer software package designed for social science research that 
facilitates the management, coding and analysis of qualitative data. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-19
Hypothesis Question #5: How did the gender composition of the deliberative groups 
differentially impact the priority issues of the participants? 
The evaluation team disaggregated matrix-identified priorities by group type (all-female, all-male and 
mixed groups) to identify differences in priority issues by group composition. A regression analysis was 
done to determine statistically significant differences in priorities across group types. In addition, the 
gender-sensitive matrix priorities were also reviewed against matrix measures that gauged voting patterns 
for solutions. These measures were used to analyze the effect of group type on voting pattern. Because 
the qualitative data were collected across randomly assigned groups – randomizing the gender 
composition of the deliberative groups -- they are experimental findings (see appendix 1 for details on the 
matrix measurements). 
Hypothesis Question #7: Did the gender of the facilitator differentially influence the priority 
issues of participants? 
The matrix-identified priorities were also disaggregated by facilitator gender, again using the four 
measures on matrix voting patterns to determine the effect of facilitator gender on participants’ priorities. 
As with hypothesis question 5, because the qualitative data were collected across randomly assigned 
groups – randomizing facilitator gender – they are experimental findings. 
How did the deliberative group participants behave at the CD? 
The attendance and behavior of deliberative session participants were analyzed and compared with the 
larger population of citizens in attendance at the CD. The results were disaggregated by gender, behavior 
and participation in a deliberative session. 
What was the environment at the CD event? 
Likert scales from facilitator observations were graphed and analyzed in the aggregate as well as by each 
commune in order to document the environment of each CD. These were triangulated with the facilitators’ 
qualitative observations. CD observation data were inventoried along with the matrix-identified priorities 
at the commune level. The two datasets – CD observation and matrix-identified priorities – were 
compared in order to identify the patterns of issues across provinces, party presence, police presence, 
MNA behavior and citizen response to each CD. 
Analysis of Program Hypothesis 
The evaluation team also reviewed the qualitative data in the context of: a) NDI’s larger program 
hypothesis; and b) the current Cambodian political situation. The following sections present a summary of 
qualitative findings from the deliberative sessions and the constituency dialogue events, along with the 
implications of the findings for the study, citizen deliberation and for the CD program design and 
evaluation. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-20
IV. Qualitative Findings: 
The day before the CD event, NDI held deliberative sessions with a randomly invited group of rural 
citizens from the same commune. When the citizens arrived NDI randomly assigned some of the citizens 
to participate in all-male, all-female or mixed-gender groups. The following presents the qualitative 
findings for: (A) the deliberative sessions; and (B) the constituency dialogue (CD) observations. These 
findings inform and complement answers to three of the eight hypothesis questions that make up the 
study design, offering some insight into the behavior of deliberative session participants at the 
Constituency Dialogue. 
A. Deliberative Sessions 
Hypothesis Question #3: How did participation in a deliberative session before the CD impact 
citizens’ priorities? 
A week before the CD the research team administered a baseline survey to individuals from the 
surrounding CD village. Survey respondents were given a closed list of priority problems and asked to 
rank their top three in order of importance. At baseline the research team invited all survey respondents to 
the CD, but randomly invited some respondents to participate in a deliberative session the day before the 
CD. During the deliberative sessions participants were asked to discuss their problems with a group of 3-
6 of their fellow citizens. 
Comparison of Survey & Deliberative Session Priorities
Rank Order
Baseline Survey Identified Priorities: Deliberative Session Matrix-Identified Priorities:
Survey Question: Of the following issues, which are the top 
three you would like your Members of the National Assembly 
to take action on?
Deliberative Session Instructions: After brainstorming, 
rank your top three priorities by distributing buttons on the 
three most important problems
% of individual respondents who listed priority as one of 
their top 3 priorities (997 respondents in CD communes)
% of groups who voted priority as one of their top 3 
priorities (79 groups)
1 Lack of infrastructure (roads): 52% Lack of Infrastructure (roads): 57%
2 High price of commodities (electricity and oil): 50% Land Seizure: 43%
3 Land Seizure: 48% Corruption: 32%
4 Lack of security (youth gangs): 32% Money high cost of living: 27%
5 Lack of transparency (extortion and corruption): 25% Water & Sanitation: 24%
6 Illegal Logging and Fishing: 20% Health Clinics: 23%
7 Agriculture- related problems: 17% Lack of Security (youth gangs):20%
8 Vietnamese-Cambodia Border (influx of Vietnamese 
workers): 13%
Education:14%
9 Inability of Farmers to Pay Off Debts: 11% Agriculture Related Problems:13%
10 Political party platforms don’t fight poverty: 10% Illegal Logging and Fishing: 11%
11 Police and local officials discrimination/ unfair practices: 8% Electricity: 10%
12 Poor quality of education: 5% Market: 8%
13 Leaders don’t represent citizens: 4% Bridge: 8%
14 -- Consecrated Buildings: 3%
15 -- Poverty: 3%
16 -- Vietnamese Migration: 3%
17 -- Drought: 1%
18 -- Stealing: 1%
19 -- Canal: 1%
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-21
Table 1: Top priorities identified by individual survey respondents vs. deliberative groups 
The comparison of priorities between the baseline survey and the deliberative sessions illustrate the 
qualitative and substantive difference between how individual preferences are aggregated on a survey and 
how preferences are aggregated as part of a deliberative group process. While many of the same issues 
rank among citizens’ top three survey-identified priorities (e.g., lack of infrastructure, land seizure, 
corruption, high cost of living), when asked to deliberate with their peers citizens placed lesser 
importance on some priorities (Vietnamese migration, illegal logging and fishing); placed more emphasis 
on other priorities (e.g., education); identified new priorities (e.g., water & sanitation, health clinics inter 
alia); and omitted other priorities altogether (e.g., leaders don’t represent citizens, political party 
platforms don’t fight poverty). 
While it must be said that the survey offered a limited choice of priorities to respondents and no 
opportunity to add priorities, the deliberative session priorities illustrate how the creation of time and 
space for citizens to discuss problems yields a more granular perspective on citizens’ lives. At the same 
time a content analysis of the deliberative sessions also revealed the interrelated nature of many of the 
citizens’ priorities, and the ways in which corruption cuts across the issues. The following provides a 
qualitative review of the top six matrix-identified issues, highlighting where they were most prominent 
across the provinces and communes. 
1. Infrastructure 
Poor infrastructure is a common concern for communities across rural Cambodia. Not surprisingly, 
whether identified through a deliberative process or on a survey, infrastructure ranks among citizens’ top 
priorities. During deliberations citizens discussed the web of interrelated issues that affect and are 
affected by the lack of infrastructure. 
During the deliberative sessions citizens complained that local roads were not maintained and their 
conditions were made worse by large trucks carrying heavy cargo. Citizens reported their produce 
spoiling before they were able to get it to market. In the dry season, dust kicked-up from unregulated 
traffic creates health problems for residents living on the sides of these dirt roads. During the wet season 
the unpaved roads are flooded and bridges are often left unrepaired. The poor road conditions not only 
make it difficult for residence to travel to market, or students to schools, but also increases the risk that 
residents will be struck by vehicles. 
Deliberative groups also noted that the lack of electricity in many of the communes also serves to further 
isolate residents, not only cutting them off from information through radio and television, but affecting 
their children’s education. Residents reported their children being forced to study by candlelight or 
kerosene lamp. The lack of electricity not only makes residents feel unsafe traveling after dark, but when 
coupled with poor road conditions makes it nearly impossible to reach medical services if their children 
are struck ill during the night. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-22
2. Land Seizure 
The confiscation of lands and the forced eviction of residents is a phenomenon affecting both rural and 
urban Cambodian citizens and is an issue of increasing concern among the international human rights 
community19
. Not surprisingly, land seizure was ranked second on both the survey and in the deliberative 
groups. As part of a shift to a market economy, the Cambodian government established the 1992 Land 
Law, which reinstated legal private ownership of land along with the government’s right to grant land 
concessions to support the national economy. The government’s reform agenda was purportedly aimed at 
food security, sustainable agricultural development and economic growth. However, the lack of clear 
regulations on government land concessions, unclear procedures for private land registration and the 1994 
Law on Investments gave the government unrestrained authority to appropriate lands and forcibly evict 
residents. Indeed, a 2004 UN report referenced the Land Law of 1992 as “a ‘get rich quick’ manual for 
the upwardly mobile” in Cambodia.20 Since 2003 almost 400,000 citizens have been displaced or affected 
by land seizures. 
Cambodian Land Concessions (red) from 1993-201221
Cambodia Provincial Maps
Figure 3: Land Concessions Over Time 
The priority of land seizure was largely driven by deliberative group priorities in Kampong Chhnang and 
Kratie provinces where they were most often ranked among the top three priorities. When comparing 
maps of Cambodian land concessions with provincial maps we see that both Kampong Chhnang and 
Kratie provinces are at the center of some of the most dramatic land concession activities in Cambodia.22
 
 
19 UN Office of the High Commissioner for Human Rights, Report of the Special Rapporteur on The Situation of 
Human Rights in Cambodia (12 July 2012) US Department of State, 2011 Country Report on Human Rights 
Practices; Amnesty International Urgent Action Appeal 17 December 2012; International Crisis Group, Cambodia: 
the Elusive Peace Divided and Human Rights Watch World Report 2012: Cambodia. 
20 UNHCHR 2004: Special Representative of Secretary-General for Human Rights in Cambodia. Land Concessions 
for Economic Purposes in Cambodia: a human rights perspective. 
21 Visualizing Land Concessions Over Time: 1993-2012 Interactive Map. Cambodian League for the Promotion and 
Defense of Human Rights (LICADO) http://www.licadho-cambodia.org/concession_timelapse 
22
 A 2007 UN Report (Economic Land Concessions in Cambodia: a human rights perspective) noted Kampong 
Chhnang as one of the province where the greatest amount of land has been conceded; and the Sambo district in 
Kratie province where large tracks of land have been conceded to private companies 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-23
In Kampong Chhnang province land seizure was ranked the number one priority for deliberative groups 
in the Pech Chanvar and Ta Ches communes. In Kratie province, Kon Knher commune deliberative 
sessions ranked land seizure as their top priority issue, while the Krang Leav commune ranked land 
seizure as the third priority issue across all deliberative groups. These communes – along with other 
communes -- discussed the nature of the land seizures, the impact of land seizures on their communities, 
and the lack of redress for the destruction of their homes and livelihoods. 
Private Companies: In Kratie the government has made numerous land concessions to private companies 
who have set about seizing land without regard to the de facto or de jure ownership of local residents.23
 
During the deliberative sessions many citizens complained about various companies encroaching on their 
lands, leveling their fields, clear cutting the forests and destroying the wildlife. When residents’ animals 
wander onto company lands they are confiscated, and payment is demanded for their return. 
There are Chinese and Indian companies clearing the forests to plant sugar cane, 
potatoes, or for mining. They are clearing the land near the village. The people say 
that there are dividing marks on their lands. The companies’ compounds are 
surrounded by a canal and when the people’s cattle enter the compounds they fall in 
and drown. The mining has very serious affects when the water flows into the lake. It 
contains chemicals that cause the cattle to die and the people to have stomach 
problems, vomiting and diarrhea. When [the people] make appeals to these 
companies they do not take responsibility [for their actions], so there is not a solution. 
(Facilitator Reflection #3, Kon Knher, Kratie) 
Land Titles: Almost all the deliberative groups in Kampong Chhnang discussed the worsening land crisis. 
In the Krang Leav commune of Kampong Chhnang citizens reported that the local authorities delay the 
issuance of land titles to local citizens in order to facilitate the confiscation of occupied lands. 
The story I have heard in the [deliberative session] was that the participants said that 
they are worried about their rice fields -- located in the Patlang village -- because they 
are in the airport zone. They are worried about the possibility that their rice fields will 
be seized, especially since they have already been flatted once before. (Facilitator 
Reflection #1, Krang Leav, Kampong Chhnang) 
In the discussion group the citizens raised the problem that the authorities had not yet 
issued certificates of land title to them; and that their lands were reserved for building 
an airport. Worried, the citizens went twice to register a complaint with the provincial 
authorities. As a result a high-ranking official came from Phnom Penh to help. Their 
lands were given back to them on the condition that when [the government] needed the 
land to build the airport they would take back the land. The airport project is affecting 
the whole Krang Leav Commune. (Facilitator Reflection #4, Krang Leav, Kampong 
Chhnang). 
 
23 Ibid. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-24
The nature of land seizure in rural Cambodia is exacerbated by the isolation of farmers. Not only do their 
appeals go unheeded, but local authorities are often complicit in the dispossession of whole villages from 
their lands and livelihoods: 
In the group I facilitated there was a participant who is the victim of land confiscation 
by [a private company].24 The conflict has [been] going on for ten years but still there 
is no proper solution so far. [She] added that, regarding her protests of the company, 
her mother was sent to prison for one year and eight months. Her sister died in a 
traffic accident after visiting her mother at the prison. She suspects that her sister’s 
traffic accident was murder, since her mother described her whole story to her sister. 
Her brother, who refused to leave and built a cottage on the disputed land, has been 
charged with a crime. He escaped to Thailand, where he remains today. Her husband 
has also been charged with a crime and was ordered to pay compensation to the 
company in the amount [of] 10 million Riel. She went on to say that the confiscation by 
the company totally violates her family’s rights as well as those of other citizens [in Ta 
Ches commune]. On the day of the seizure the company took over the lands without 
any regard for the fact that she was still in the house recovering from delivering a 
baby 15 days earlier. Facilitator Reflection #3, Ta Ches, Kampong Chhnang)
Intimidation: The theme of protest and incarceration ran throughout the discussion on land seizures. In 
Ta Ches commune (Kampong Chhnang) many deliberative groups discussed their unsuccessful attempts 
at protesting the land seizures, as well as the international company’s use of local authorities to punish 
them for raising their voices. 
During the discussion on land disputes the group said that they have tried to protest 
against the confiscation of their lands. However, it is of no use since there isn’t 
anyone around who will be concerned about the dispute, especially not the local 
authorities. More importantly, the leaders of the protest usually face confinement and 
the same protests have been taking place for about 10 years. (Facilitator Reflection #9, 
Ta Ches, Kampong Chhnang)
The management team of Ta Ches commune, Kampong Tralach district, Kompong 
Chhang province, are from the CPP Party. The threat is that the management team 
will put a lot of pressure on the village people in the Ta Ches commune [because of] 
the conflict with the international company. The company did commit violence against 
the village people and sent six village people to prison in Kompong Chhang for a long 
time. For example, Mr. [X], has been sent to prison for 4 years and ordered to pay 6 
million Riels in order to be released. But he has yet to be released. Through my 
observation as the facilitator in Group 8, I have seen that land encroachment, violence 
and confinement in the Tra Ches commune are [all] related to the international 
 
24 The State Department’s 2011 Country Report on Human Rights Practices in Cambodia has documented the 
questionable relationship between private international companies and a key Ministers within the government..” 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-25
company. The village people are made to pay for their protests [against the company]. 
(Facilitator Reflections #8, Ta Ches, Kampong Chhnang)
The qualitative data revealed systematic intimidation of local residents. In some cases the deliberative 
sessions provided a forum for the victims of the land seizures to discuss their concerns with each other, 
while in other sessions victims expressed fear of discussing the issue at all: 
The process of discussions in my group went smoothly, because the citizen raised 
existing problems as well as provided encouragement to other members to share their 
opinions, especially regarding the land grabbing which has taken place [in] Lok Chum 
Tiev Chea Keng. The company’s name is [omitted]. Of the 512 hectares confiscated, 
208 hectares of the disputed land belongs to citizen. The citizens say they only sold 204 
hectares [to the company]. (Facilitator Reflection #2, Ta Ches, Kampong Chhnang) 
The threating to the people who protest about land dispute is a hot topic in the Lor 
Peang village. That is the reason why the people dare not talk about the land dispute 
in the group discussion because they afraid the authorities or the company may find 
out. (Facilitator Reflection #6, Ta Ches, Kampong Chhnang)
Development: Under the pretext of development, land seizures have increased unabated in the form of 
government land concessions to national and foreign companies, government agency and/or powerful 
land prospectors. The ever-looming threat of land concessions not only disrupt people’s livelihoods, but 
dispossess families for generations to come. As one facilitator notes: 
All village people want development in their village but this kind of development only 
worsens their living conditions. Moreover, people live in fear. [One participant] said 
that he was threatened many times because he fought back for his freedom and for his 
land. In Cambodia … legal owners of the land are being accused of illegally 
possessing the land ….. I always hear people saying “the land is our life. How can we 
live without our land and house? We would rather die.” (Facilitator Reflection #4, 
Sangkat Orussey, Kratie) 
Isolation and lack of legal recourse place an already vulnerable population at risk of predation by more 
powerful land speculators who are able to dispossess citizens with impunity across rural Cambodia. Data 
from the deliberative sessions echoed former Special Representative Michael Kirby’s 1996 report to the 
UN Commission on Human Rights. As noted in that report, “the complaints of villages concerning non￾consultation, the use of armed guards, the presentation for signature of an unexplained contract, the 
shooting of cows which wander onto concession areas and the feared endangerment of village survival 
and security of traditional sites of grave, pasture and farming land,” and these complaints have not only 
continued, but are met with increasingly violent measures that violate civil, political and human rights. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-26
3. Corruption 
Deliberative groups ranked corruption as third on their list of priority issues, reporting that corruption cuts 
across all aspects of rural citizens’ lives – infrastructure, land, humanitarian aid and service delivery. 
While land seizure provides the most dramatic examples of systemic corruption, the deliberative sessions 
detailed the myriad forms of corruption that plague the daily lives of rural Cambodian citizens.
Corruption & Land: In Sangkat Orussey, Kratie there has been a similar history of government land 
seizures, but with increasingly violent tactics to hold the dispossessed residents in check. 
Citizen’s lands were confiscated and granted to a foreign company, a Vietnamese 
company. Moreover, they used violence, such as gun fire, to stop people from 
protesting. According to the deliberative session participants law enforcement officials 
were involved in corruption. [The participants want] the media to widely broadcast 
all this information to the public. (Facilitator Reflection #7, SanKat Orussey, Kratie) 
The deliberative group’s concerns were not unfounded. Those who attempt to protest or investigate 
irregularities have been threatened, arrested and/or killed.25
 
Corruption & Infrastructure: Many of the deliberative groups also discussed the influence of party 
partisanship in delivering public goods such as road repairs. As one facilitator notes: 
The group members said that one part of the road in the Srer Sdoav village is smooth 
and good for travelling. This part is in where the village chief’s supporters are living. 
The other part of the road is damaged. The village chief does not restore this part of 
the road because his supporters’ houses are not located on this side of the road. 
(Facilitator Refleciton #5, Santgat Orussey, Kratie) 
Corruption & Humanitarian Aid: In Kampong Cham a number of deliberative groups reported 
partisanship in the distribution of humanitarian aid to flood victims: 
Some of the victims did not receive the aid while some victims received three times the 
aid. Moreover, some people who are not the victims of the flood also received the aid 
while the real victims didn’t receive anything because of bias and partisanship. 
(Kampong Reab Commune, Kampong Cham Province).
Citizens reported local officials using humanitarian aid to coerce flood victims -- recording their names 
and identification numbers and extracting a promise to vote for their party in exchange for emergency 
assistance. 
 
25 Human Rights Watch November 2012 News Release. Hun Sen Government Intensifying Abuses, Moving Toward 
One-Party State. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-27
Corruption & Service Delivery: Public services are not equally accessible to all citizens. In Kratie, 
deliberative groups described how petty corruption, as well as partisanship, exclude the poorest citizens 
from the most needed services.
During deliberative session group members were talking about the discrimination and 
inequality between rich and poor people. This is their concern. They said that there are 
many forms of discrimination [that take place]. Poverty cards: some people who have 
average living condition and can support their families still receive poverty cards. In 
contrast, very poor people do not receive the card. [The group said] it is because of 
partisanship. Education: teachers teach their students less during normal school hours. 
This is because they want to earn money from teaching students extra-time [outside of 
class]. For poor students who have no money, they can only stand outside the 
classroom watching the lecture. Health Care: Medics do not take care of poor 
patients because they think that poor patients have no money. But they are very 
friendly with rich patients. (Facilitator Reflection #6, Sangkat Orussey, Kratie)
4. High Cost of Living 
The high cost of living was ranked most prominently in the Pech Chanvar commune (Kampong Chhnang 
province) but, like corruption, cut across all provinces. In Pech Chanvar and Thmar Andeauk 
deliberative groups contrasted their falling incomes against the rising price of market good; creating a 
situation in which they are unable to purchase basic goods to meet their families’ needs. In Ta Ches 
deliberative groups contrasted their rising unemployment with the high price of gasoline, limiting their 
ability to run farm machinery, water pumps or to travel or work using a motor taxi. In Krang Leav, 
residents emphasized the lack of land for farming, even for subsistence farming. Many residents have 
been forced to migrate to the city or to other countries in search of employment. In Kon Knher 
deliberative groups cited the high cost of living and its impact on the quality of available goods. With the 
increasing difficulty to make a small profit, market vendors survive by incorrectly weighing rice and other 
staple; selling goods past their expiration data, diluting gasoline, or selling imitation goods. Across the 
nine communes deliberative groups noted increasingly harsh living conditions, especially for the elderly, 
the orphans and the poorest members of their communities. Despite the increasing hardships, public 
benefits are also increasingly scarce, forcing many families to relocate or perish. 
5. Health Clinics 
Although not prominent in the survey results, lack of health clinics emerged as a priority issue in eight out 
of nine communes where the deliberative sessions were held. It was ranked most highly in all the 
communes in Kratie (Kon Knher, Thmar Andaeuk, and Sangkat Orussey) and one commune in Kampong 
Cham (Ta Prok). In Ta Prok, Kg Cham citizens noted that there was no health clinic or pharmacy in the 
village and that the lack of infrastructure made it difficult for residents to travel when they needed 
medical attention. With no clinic or pharmacy, residents are forced to buy drugs from informal suppliers 
and often do not know what drugs they actually receive. In Thmar Andaeuk, Kratie session participants 
also complained that the Health Center in Kan Tout commune was too far away and difficult to reach, 
especially when it floods. One participant recounted that his pregnant wife had died because she was not 
able to reach the hospital in time. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-28
Even when citizens can reach the health centers, service is often poor. The participants noted that the 
staff ignore or blame the patients for their problems. Furthermore the health care center is inadequately 
equipped, and medics often diagnose patients without a proper examination of their problems. As one 
facilitator recorded: 
Lack of a health center makes our lives more difficult. When we are sick, we have to go 
to the provincial hospital, which is located far from the village. We have to spend more 
time. It is especially difficult for pregnant women. They have to deliver their babies in 
a private clinic, which is more expensive. [At the provincial hospital] they demand 
money for filling out documents and use rude words. We cannot send our children to 
the communal health center even though they may be seriously ill. We have to send 
them to private clinics [and] private medical services are much more expensive than 
the services from public hospitals (Facilitator notes #9, Thmar Andaeuk). 
6. Water & Sanitation 
Water & sanitation – also not found in the survey results -- was ranked as a high priority in the 
deliberative sessions. This was especially true in Kampong Cham, where recent flooding has exacerbated 
the unmaintained infrastructure. This has affected citizens’ access to clean drinking water and damaged 
the irrigation of their rice paddies, particularly in Kampong Reab: 
During the deliberative session my group members raised the issue of their living 
conditions in the village. During the rainy season their village is flooded. The road is 
flooded and cuts them off, making it difficult for them to travel. During the dry season, 
people don’t have enough water to drink. They have to buy water (2000 riels per jar). 
Farming activities are badly affected due to the lack of water. Their living conditions 
are becoming worse and worse. (Facilitator reflections, Kampong Reab, Kampong 
Cham Province) 
People reported a lack of clean water for both themselves and their animals, forcing them to use shallow 
wells containing unclean water. This has caused serious health problems for community members and 
their children. 
Hypothesis Question #5: How did the gender composition of the deliberative groups 
differentially impact the priority issues of the participants? 
To understand how gender might impact deliberation and priority ranking, the evaluation team randomly 
assigned citizens to deliberative groups that differed in gender composition. The evaluation team then 
analyzed how the different groups prioritized their problems and voted on solutions. 
Priority Problems: When the evaluation team disaggregated the top nine matrix-identified priorities by 
gender – comparing the top three priority problems of all-male groups, all-female groups and mixed 
groups -- they found some statistically significant differences in how the groups ranked their priorities. 
For example, all-male groups were 32% more likely to than any other group type to rank lack of 
infrastructure among their top three priority issues, with all-female groups being 37% less likely to rank 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-29
infrastructure among their top three priorities. All-male groups were also 22% less likely than all other 
group types to identify corruption as a top priority. Lastly, while lack of health clinics appeared with 
equal frequency on all-male and all-female matrices, mixed groups were 14% more likely to rank health 
clinics as a priority problem. 
Priority Solutions: During the deliberative sessions participants were asked to generate two kinds of 
solutions for each of their top three priority problems – an internal solution around which communities 
could take action; and an external solution that would require action from their MNA. The deliberative 
groups then voted on the solutions, identifying the top three solutions that received the most votes. 
All deliberative groups tended to vote more frequently for external solutions to their priority problems. 
However, all-male groups tended to vote slightly more often for external solutions than did all-female and 
mixed groups. All-female groups cast an average 60 percent of their votes for external solutions. All￾male groups cast an average 71 percent of their votes for external solutions. Mixed-groups with female 
facilitators cast an average of 53 percent of votes for external solutions, while mixed-groups with male 
facilitators cast 74 percent of their votes for external solutions. 
In answer to Hypothesis Questions #5, the gender composition of the deliberative groups did impact how 
a group ranked its priority problems and the kinds of solutions it identified to address those problems. 
Further content analysis of the qualitative data by group type may provide more insight into why all-male 
groups gave stronger preference to lack of infrastructure or de-emphasized corruption in comparison to 
other group types. On the other hand, the propensity of mixed-groups to prioritize the need for health 
clinics may be a function of both males and females being in general agreement on this need in their 
communes, as reflected in the priority of health clinics in both gender-homogenous groups. In the post￾Figure 4: Effect of group composition and facilitator gender on solution ranking
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-30
deliberation reflections all groups noted that most of their priority problems required some kind of 
external support. However, the data indicate that participation in all male-groups makes preference for 
external solutions a more a more likely outcome of deliberation. 
Hypothesis Question #7: Did the gender of the facilitator differentially influence the priority 
issues of participants? 
Not only did the gender composition of the group influence deliberative outcomes, but so too did the 
gender of the facilitator. Male-facilitated groups were 37 percent more likely than female-facilitated 
groups to identify lack of infrastructure among their top three priorities. Male-facilitated groups were 
also 24 percent less likely than female-facilitated groups to identify the high cost of living among their top 
three priority problems. Similarly, male-facilitated groups were also 10 percent less likely than female￾facilitated groups to identify access to markets among their top three priority problems. 
When voting for solutions, male-facilitators tended to increase the deliberative group’s vote for external
solutions more than any other group type. In addition, when voting for solutions to specific problems 
groups facilitated by a male tended to “cluster” their votes. That is, when compared to female-facilitated 
groups, male-facilitated groups on average allocated 2.29 more of their votes for any single solution. 
Similarly, when compared to female facilitated groups, male-facilitated groups on average allocated 5.5 
more of their votes for the group’s top three solutions. In short, in answer to Hypothesis Question # 7, the 
gender of the facilitator did differentially influence three of the 19 matrix-identified problems, along with 
the type of solutions for which a deliberative group voted (see Appendix 1). 
Summary of Deliberative Sessions Findings: 
 Priority Concerns: While the descriptive data on priorities could not answer causal questions on the 
relationship between participation in deliberative sessions and priorities, they do provide critical 
qualitative and substantive details on how small groups of citizens – if given the opportunity – 
deliberate, debate and come to consensus on priority problems and solutions. A comparison between 
baseline survey priorities and matrix-identified priorities highlighted many similarities in priority 
problems, but also revealed: a shift in rank order of priorities; additional priorities not identified on 
the survey; and the omission of other priorities listed on the survey. In short, the descriptive data 
from the deliberative process presented a more granular, close-to-the ground understanding of rural 
citizens’ problems that may have been partially captured with an open-ended survey question. That 
said, a content analysis of the deliberative discussions of priorities also highlighted the interconnected 
nature of participants’ priorities and their impacts on the daily lives of citizens, families, communities 
and the natural environment. Themes of corruption were heavily featured in the qualitative data, 
along with intimidation and the systematic dispossession of the rural poor from their lands and 
livelihoods. Indeed, corruption cut across many of the priority issues. 
 Gender & Priority Problems: The data show that the gender composition of the deliberative groups 
did influence the way in which a group ranked some priority problems using a matrix. During 
deliberation all-male groups tended to place a higher priority on lack of infrastructure, but were less
likely to rank corruption as a top priority, as compared to all other group types. Mixed-groups were 
more likely than any other group type to identify lack of health clinics among their top priorities. 
This may reflect a general consensus of both genders on the need for health clinics, as indicated by 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-31
the equal frequency with which all-female and all-male groups also ranked health clinics a priority. 
The gender of the facilitator also influenced some of the priority issues. Groups facilitated by males 
were less likely than groups facilitated by females to identify the high cost of living or access to 
markets among their top three priority problems. Across all nineteen matrix-identified priorities, five 
priorities demonstrated a gender bias due to group composition or facilitator gender -- approximately 
26% of all priorities. 
 Gender & Priority Solutions: As with priorities, the data also demonstrated that the gender of the 
facilitator had an impact on how participants in deliberative sessions voted for solutions. While all 
groups tended to vote for external solutions to their priority problems, all-male groups and groups 
facilitated by a male tended to vote more often for external solutions than any other group type. 
Furthermore, when compared to female-facilitated groups, male-facilitated groups tended to “cluster” 
their votes for single solutions to individual problems as well as for their top three priority solutions. 
Together the descriptive and causal data analysis provides insight into micro-level democratic processes 
and the ways in which the context as well as gender inform and influence citizen deliberation. The 
content analysis of citizen deliberation demonstrated the interconnected nature of the problems facing 
rural Cambodians, detailing the ways in which corruption plagues many aspects of citizens’ lives. The 
statistical analysis of matrix priorities and matrix voting patterns speak to the causal relationship between 
deliberative group composition, facilitator gender and deliberative outcomes. The data clearly 
demonstrate that the gender composition of a deliberative group influences ranking preferences, while the 
gender of the facilitator influences how strongly certain solutions are preferred over other. 
Moreover, the qualitative data not only provide a context for citizens priorities, but insight into the factors 
that may enable or inhibit citizens in exercising their voices within newly emerging political space such as 
the Constituency Dialogues. The following section details the environment in which the CDs took place 
and the behavior of the deliberative group participants. Environment and behavior are then set against the 
backdrop of the priority concerns identified in each of the nine communes the day before the CD event. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-32
B. Constituency Dialogues 
The day after local citizens participated in their small deliberative sessions their commune hosted a CD 
event. NDI’s underlying assumption for the deliberative sessions was that if citizens were given some 
additional time and space to discuss their concerns and come to consensus on their priority issues they 
might gain additional clarity and confidence to more fully engage their representatives at the constituency 
dialogue the next day. This section presents the qualitative data in answer to three questions: 
 What was the behavior of deliberative session participants on the day of the CD? 
 What was the environment at the CD event? 
 What were the deliberative group priorities in each commune where the CD was held? 
What Was the Behavior of Deliberative Session Participants at the CD? 
After the completion of the matrix-ranking exercise the facilitator asked the group to reflect on: the 
accuracy of the matrix; which kinds of solutions received the most votes and why; and if they would raise 
their priority issues during the CD event the following day. Facilitators recorded that majority of the 
deliberative groups (78 of 80) agreed that the priorities accurately represented their problems and desired 
solutions for citizens in their villages. Seventy-nine of the eighty groups agreed that someone in their 
group would raise some of their matrix-identified concerns at the constituency dialogue the next day. 
During the constituency dialogues deliberative session facilitators acted as observers and monitored and 
recorded the behavior of their deliberative group members in attendance at the CD, recording their gender, 
hand-raising to speak; and speaking at the CD. 
Attendees That Raised Concerns at the CD
Gender
% of CD event attendees 
(excluding deliberative session 
participants)
% of Deliberative session 
participants in attendance at CD
Male 2.73% (65 of 2384) 6.90% (8 of 116)
Female 1.32 % (24 of 1820) 1.96% (2 of 102)
Both 2.12 % (89 of 4204) 4.56 % (10 of 218)
Table 2: CD vs. Deliberative Session Participant Behavior at the CD Event (disaggregated by gender) 
Although participants in almost all deliberative sessions agreed the day before that they would raise their 
concerns at the CD event the next day, only 65.7% of those citizens were able to attend their CD events 
the following day. Of those deliberative session participants who attended the CD, 46% were female and 
a little more than half were males (53%). However, when the evaluation team compared the observed 
behavior of deliberative session participants at the CD with those citizens who had not participated in the 
deliberative sessions, they found some interesting trends. Deliberative session participants raised their 
hands and spoke at their CD events more than twice as often as citizens at the CD who had not 
participated in a deliberative session. When the data were disaggregated by gender the evaluation team 
found that women who had participated in deliberative sessions spoke at their CDs one and a half times 
more often than female attendees who had not participated in a deliberative session. Most dramatically, 
male participants in deliberative sessions raised their hands and spoke at their CD events two and half 
times more often than their male counterparts who attended the CD but had not attended a deliberative 
session. It should be noted that these findings cannot be considered experimental since deliberative 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-33
session participants self-selected their attendance at the deliberative sessions and CD, and it is possible 
that some individuals who were invited to participate in the deliberative session but did not were also in 
attendance at the CD. It is also possible that some individuals who attended the deliberative session and 
the CD were not among the group randomly invited to attend the deliberative session. However, the 
differences across categories suggest that participation in the deliberative sessions contributed to 
differences in citizen engagement at the constituency dialogue. 
What Was the Environment at Constituency Dialogue Event? 
In addition to the behavior of their deliberative group participants, the deliberative session facilitators 
observed and recorded the environment of the CDs. The observers used a Likert scale and qualitative 
descriptions to record their perceptions and observations of the general environment of the CD, recording: 
political party presence; police presence; MNA behavior; and general audience satisfaction (see 
Appendix 2). The following summarizes the qualitative findings from these observations, first presenting 
the general environment across all nine CDs and then detailing the specific contexts for each CD. The 
latter presentation highlights the issues that citizens in each commune face, providing some insights into 
the challenges of citizens in raising their voices at some of these CD forums. 
Figure 5: Observer perceptions of party support across all nine CD events 
Party Presence at the CD Events: Facilitator observations indicated that the ruling party did not have an 
overwhelmingly strong presence of its members at the CD events. In fact, the observers tended to report a 
slightly higher presence of opposition party members at most of the CD events. In Kampong Cham 
province, the observers reported a roughly equal presence of the opposition and ruling party members in 
the Ta Prok commune CD and a slightly stronger presence of opposition party members in the Chaom Ta 
Mau and Kampong Reap commune CDs. In Kratie province, while it was unclear which party had a 
stronger presence in the Sangkat Orussey commune, the observers reported a roughly equal presence of 
opposition and ruling party members at CDs held in the Kon Knher and Thmar Andaeuk communes. In 
Kampong Chhnang province, observers reported a slightly higher presence of opposition party members 
at the Pech Chanvar and Ta Ches commune CDs, and again approximately equal presence of both 
opposition and ruling parties at Krang Leav commune CD. 
0%
20%
40%
60%
Agree Disagree No Answer (NA)
39%
44%
18%
57%
26%
17%
Perceived Presence of Ruling vs Opposition Party 
Supporters at CD Event
Q1A The CPP had a
significant number of
its members present at
the Constituency
Dialogue
Q2A. The Opposition
parties had a
significant number of
their members present
at the Constituency
Dialogue
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-34
Figure 6: Observer perceptions of police presence and intimidation across all nine CD events 
Police Presence at the CD Events: In most communes, observers agreed that police presence was not 
particularly strong at the CD events. In seven of the nine communes, the observers did not report that 
citizens attending the CDs were intimidated by the police. However, in two communes – both in Kratie 
province – observers did agree that there was a strong police presence at both the Kon Knher and Thmar 
Andaeuk commune CDs. Further, at the Kon Kmher CD, some observers reported that people felt 
intimidated by the strong police presence, while in Thmar Andaeuk, observers seemed to be split on
whether or not CD participants were intimidated by the strong police presence. 
Figure 7: Observer perceptions of interaction between MNAs at CD events
0%
20%
40%
60%
80%
Agree Disagree No Answer (NA)
42% 43%
14% 15%
73%
14%
Perceived Police Presence and Intimidation at CD Event
Q3A. There was a large
police presence at the
Constituency Dialogue
Q4A. The police caused
people to be nervous
at the Constituency
Dialogue
0%
20%
40%
60%
80%
Agree Disagree No Answer
(NA)
23%
73%
4%
Perceived Interaction between MNAs at CD Event
Q10A.Based on
your own opinion,
MNAs used the
time to debate
and attack each
other.
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-35
Figure 8: Observer perceptions of MNA interaction with citizens across all nine CD events 
MNA Behavior at the CD Events: Almost all of the observers agreed that the MNAs attempted to 
respond to the concerns raised by their constituencies at the CD. The observers agreed that during most 
of the CDs the MNAs did not use their time to attack and/or debate with each other. However, observers 
were slightly divided on MNAs’ interaction with each other in Ta Prok, Kampong Cham and Pech 
Chanvar, Kampong Chhnang. Observers at both CDs also reported that CPP MNAs spoke harshly to the 
CD attendees. 
Figure 9: Observer perceptions of citizen satisfaction of MNA’s responses across all nine CD events 
0%
20%
40%
60%
80%
Agree Disagree No Answer
(NA)
16%
70%
14%
Perceived MNA Interaction w/Citizens at CD Event
Q11A.Based on your
own opinion, MNAs used
the time to speak
harshly to the people.
0%
20%
40%
60%
80%
100%
Agree Disagree No Answer (NA)
26%
54%
20%
97%
1% 1%
77%
5%
18%
Perceived Citizen Satisfaction with MNA Responses
Q5A. People were
satisfied with the CPP
MNA's responses to their
concerns and questions
Q6A. People were
satisfied with the
opposition party MNA's
responses to their
concerns and questions Q7A. People were overall
satisfied with the MNA's
remarks
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-36
Figure 10: Observer perceptions citizen satisfaction with local officials’ responses across all nine CD events 
Citizen Satisfaction with Representatives’ Responses: The observers largely agreed that constituents 
were generally satisfied with the MNAs’ response at all nine CD events. When asked to decipher the 
attendees’ satisfaction with the responses of MNAs from ruling and opposition parties, observers 
consistently agreed that citizens were satisfied with MNA responses from opposition parties; however, 
observers disagreed that citizens were satisfied with MNA responses from the ruling party. In eight 
communes, observers were either split—as in Ta Prok, Thman Andaeuk and Krang Leav—or disagreed—
as in Chaom Ta Mau, Pech Chavar, Ta Ches, Kampong Reab, and Sangkat Oressy—that citizens were 
satisfied with MNA responses from the CPP party. In only one commune, Kon Knher, Kratie, did 
facilitators agree that citizens seemed satisfied with MNA responses from the ruling party, the CPP (see 
Figure 12). 
When asked to assess citizen satisfaction with the responses of their local officials, observers were less 
decisive. In three communes (Chaom Ta Mau, Thman Andaeuk and Krang Leav) observers disagreed 
that citizens were satisfied with their local officials responses; in three communes (Ta Prok, Ta Ches and 
Sangkat Oressy), observers agreed that citizens were satisfied with the responses of their local officials; 
and in three communes (Kon Knher, Pech Chavar, and Konpong Reab) observers were split or unclear if 
citizens were satisfied with the responses of their local officials (see Figure 13). 
What Were the Deliberative Group Priorities in Each Commune? 
Kampong Cham Province
Matrix-Identified Priorities
Chaom Ta Mau Ta Prok Kampong Reab
Infrastructure Infrastructure Infrastructure 
Markets Health clinics Water & Sanitation
Youth gangs Education Agriculture
Agriculture Electricity 
Table 3: Deliberative Group Priorities by Province and Commune 
Like other provinces in the study, the people in the Kampong Cham province complained of a poor 
infrastructure that impedes the production and marketing of their agricultural products. This threat to 
0%
10%
20%
30%
40%
Agree Disagree No Answer (NA)
27%
34%
39%
Perceived Citizen Satisfaction with 
local officials' responses
Q8A. People were
satisfied with the
local officials'
responses to their
concerns and
questions
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-37
their livelihoods has been exacerbated by severe flooding. Although humanitarian aid was greatly needed 
by citizens in the province, the deliberative group participants reported that aid delivery was intercepted 
by partisan local officials who meted out aid in exchange for political support. 
Chaom Ta Mau: This CD was characterized by a slightly higher presence of opposition party members, 
and a modest police presence that did not particularly intimidate the attendees. Observers agreed that 
MNAs generally responded to citizens’ concerns, did not waste time attacking or debating with their 
opponents, nor did they speak harshly to the citizens. Observers disagreed that citizens were satisfied 
with the responses from the ruling party MNAs or their local officials. However, observers did agree that 
citizens were satisfied with the responses from opposition parties MNAs. A facilitator observed that: 
There was a considerable number of opposition party members [in the audience] 
because whenever the opposition parties MNA made any remark on the concerns 
raised by the voters, the [remarks] were most welcomed and greeted with applause. 
(Chaom Ta Mau Commune, Kampong Cham Province on 30 July 2011) 
Chaom Ta Mau Matrix-identified Priorities: Infrastructure, markets, youth drug gangs 
Ta Prok: Observers noted an equal presence of ruling and opposition party supporters and some police 
presence, which did not lead to intimidation of attendees. They agreed that the MNAs generally 
responded to citizen concerns and questions, but some observers noted that some MNAs did spend time 
debating and/or attacking each other. The observers were also split on MNAs responses to the people, 
noting that some people were spoken to harshly by the MNAs. Observers were split on whether or not 
citizens were satisfied with the responses of MNAs from the ruling party, but agreed that citizens were 
satisfied with the responses from the opposition party, as well as their local officials. 
Facilitators reported the above characteristics as follows: 
During the CD, participants were very active in asking questions as well as raising 
their concerns to the MNAs. Female participants raised more questions than male 
participants. Participants also wondered why the CPP MNAs made responses which 
are not in accordance to their questions regarding irrigation systems. Some other 
participants said that the MNAs did not directly respond to the questions. (Ta Prok 
Commune, Kampong Cham Province on 26 November 2011)
In the first stage, CPP MNAs used some harsh words toward the people who 
questioned them about the government’s policy regarding forest land concessions, 
mining and taxes, etc. (Ta Prok Commune, Kampong Cham Province on 26 
November 2011) 
Citizens were angry with Her Excellency Chem Savai because she wasted time and 
didn’t respond directly to the problems raised. (Ta Prok Commune, Kampong Cham 
Province on 26 November 2011)
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-38
I think that MNAs used some of the time to debate and attack each other. However, 
they mostly attacked each other on issues such as the national budgets, liability and 
foreign loan. (Ta Prok Commune, Kampong Cham Province on 26 November 2011)
Ta Prok Matrix-Identified Priorities: Infrastructure, Health Clinics, Education, Agriculture 
Kompong Reab: Observers reported a stronger presence of opposition supporters than CPP supporters at 
the CD. Most observers disagreed that there was a strong police presence and did not think the citizens 
were intimidated. They observed that the MNAs attempted to respond to citizens’ questions and concerns; 
did not spend time attacking or debating the other MNAs; nor did they harangue or speak harshly to the 
attendees. While they felt citizens were generally satisfied with MNA responses, they disagreed that they 
were satisfied with the ruling party’s responses. They did agree that the citizens were satisfied with the 
opposition parties’ responses, but were unable to decipher if citizens were satisfied with the responses of 
their local officials. 
The MNAs responded directly to the concerns raised [by the citizens] but their 
responses didn’t really address what the people needed. (Kampong Reab Commune, 
Kampong Cham Province on 14 January 2012)
During the deliberative group sessions in Kompong Reab citizens consistently discussed the partisanship 
and corruption of their local official in distributing humanitarian aid after a recent flood. One of the 
facilitator noted the fear that one citizen expressed in raising his concerns at the CD: 
Even in the case of humanitarian aid for flood victims in the Kampong Reab commune, 
some rich people received 4-5 times the amount of aid, while the poor never got 
anything. Although people had asked the local official to stop discriminating against 
people, the local official didn’t listen. [They complained that] humanitarian aid for 
flood victims never reached the poor people. The [local officials] only distributed the 
aid to their party members or relatives. My group member did not dare to raise his 
concerns to the MNAs [at the CD]. He asked the facilitator to write a letter to inform 
the MNAs about his concerns. Local officials listed the names and ID card numbers of 
people and they only offered people aid in exchange for the promise that the people 
would vote for the CPP. This is a secret activity, and it also violates the rights of voters. 
None of MNAs had visited the people [of Kampong Reap] for a very long time. People 
never knew their names or faces. MNAs come only when it is time for the election 
campaign. If there were no NDI event, those MNAs would not come to see people. 
(Kampong Reab Commune, Kampong Cham Province on 14 January 2012)
Kampong Reab Matrix-Identified Priorities: Infrastructure, water & sanitation, agriculture, electricity 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-39
Kratie Province
Matrix-Identified Priorities
Kon Knher Thmar Andaeuk Sangkat Orussey
Land seizure Infrastructure Corruption 
Infrastructure Health clinics Infrastructure 
Heath clinics Corruption Health clinics 
Corruption Poverty Water & Sanitation 
Table 4: Deliberative Group Priorities by Province and Commune 
Citizens in the province of Kratie not only suffer the challenges of a weak infrastructure, but they also 
face serious problems of land seizure and corruption. In the deliberative sessions citizens spoke of 
government confiscation of their lands for both public and private use, with little to no compensation. 
Facilitators recorded a particularly strong police presence in Kon Knher and Thmas Andaeuk communes. 
In Sangkat Orussey, CD facilitators recorded a modest turnout of either opposition or ruling party 
supporters. However, observers did note that citizens were courageous in voicing their concerns.
Kon Knher: Observers approximated an equal number of CPP and opposition party members at the CD 
event. Almost all of the facilitators recorded a strong police presence at the CD. Some of the observers 
agreed that the police presence intimidated attendees. Nevertheless, they also agreed that the MNAs 
responded to citizens’ questions and concerns; refrained from attacking and debating the other MNAs; 
and did not speak harshly to the attendees. The observers agreed that citizens were generally satisfied 
with MNA responses – both from the CPP and opposition parties; but, they were unclear if citizens were 
satisfied with the responses of their local officials. 
The following facilitators’ observations described the strong police presence and the effect on participants 
at the Kon Knher CD event: 
The presence of police officials in the constituency dialogue was remarkable. [There 
were] about 10 of them. The police officials spread out everywhere -- at the entrance 
to the forum, in the middle, in front, and behind the forum. They were also mobile 
around the forum. (Kon Knher Commune, Kratie Province on 6 August 2011)
The police officers walked around the forum when the MNAs were making their 
opening remarks. They also had a voice recording device at the forum. The police 
officers took turns observing the situation. And when the people raised any concern for 
discussion, the police stood up and looked at their faces. (Kon Knher Commune, 
Kratie Province on 6 August 2011)
There were about ten police officers at the program. They spread around the forum, 
some at the entrance, some at the sides, and three or four of them sat in the front row 
of the forum. I saw the police officers had papers and pens, and it looked like they 
were noting things down. (Kon Knher Commune, Kratie Province on 6 August 2011)
There were many police officials wearing stripes which affected some people in 
expressing their opinions and made them afraid. The police officials were from the 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-40
province, district, and station in different levels. (Kon Knher Commune, Kratie 
Province on 6 August 2011) 
Kon Knher Matrix-identified Issues: Land seizure, Infrastructure, health clinics, corruption 
Thmar Andaeuk: Observers noted that there was approximately equal presence of CPP and opposition 
party supporters at the CD. However, they agreed to strongly-agreed that there was a heavy police 
presence at the event. The observers were split on whether or not the police presence intimidated the 
attendees. They agreed that the MNAs generally tried to respond to citizen questions and concerns, but a 
number of observers noted that the MNAs spent time attacking and debating each other. Nevertheless, 
they did not agree that the MNAs spoke harshly to the people in attendance. While observers agreed that 
the citizens were generally satisfied with the MNAs, they were split on citizen satisfaction with MNA 
responses from the ruling party. They generally agreed, however, that attendees were satisfied with the 
responses of the MNAs from the opposition parties, but disagreed that they were satisfied with the 
responses of their local officials. Observers provided details on the heavy police presence at the Thmar 
Andaeuk CD event: 
There were too many police officers such as security police, criminal police, military 
police, and a soldier. In total, there are 17. (Thmar Andaeuk Commune, Kratie 
Province on 19 November 2011)
There were many police officers presented at the forum of NDI. People felt nervous 
about speaking and raising their concerns because of the presence of local officials. 
(Thmar Andaeuk Commune, Kratie Province on 19 November 2011)
I noticed that all police officers and local officials sat together with the people. 
Sometimes, the local officials stared and laughed at people who raised concerns and 
the opposition party MNAs who made comments [in response] to the people. (Thmar 
Andaeuk Commune, Kratie Province on 19 November 2011)
The most interesting thing about the CD Program was too [heavy] a presence of local 
authorities. There were about……..people. If they came as protector, it would be good. 
However, it was otherwise. They accompanied the members of National Assembly from 
the ruling party and were political tools for the ruling party… They sat with 
participants at the rear, and when the people talked about their problems and concerns, 
those authorities stared at them unhappily. Through this action, village people worried 
about their safety. In addition, those authorities laughed at villagers and whispered 
among themselves while the Members of the National Assembly from opposition 
parties made a welcome speech to villagers. (Thmar Andaeuk Commune, Kratie 
Province on 19 November 2011)
Thmar Andaeuk Matrix-Identified Issues: Infrastructure, health clinics, corruption, poverty 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-41
Sangkat Orussey: Observers did not report a significant presence of either opposition or ruling party 
supporters at the CD. They also did not report a heavy police presence, nor did they indicate that the 
attendees were intimidated by the police. They noted the behavior of the MNAs as responsive to 
questions and civil in refraining from attacking or debating their opponents. Neither did the MNAs 
harangue the attendees. Similar to the observations at other CDs, observers felt citizens were generally 
satisfied with MNA responses, but disagreed that they were satisfied with the ruling party’s responses. 
They did agree that the citizens were satisfied with the opposition parties’ responses, as well as the 
responses of their local officials. Observers noted the modest presence of the CPP party at the Sangkat 
Orussey CD event as well as the courage of participants who raised their voices: 
It is hard to observe [the presence of ruling or opposition party supporters] because 
there is a small amount of participants at the CD. (Sangkat Orussey Commune, 
Kratie Province on 19 November 2011)
The CPP did not have a significant number of its members present at the CD. 
(Sangkat Orussey Commune, Kratie Province on 19 November 2011)
On the day of the CD, one participant from Srer Kreng village, Sandan commune, 
Sombo district, Kratie province said that his land was confiscated to use for university 
construction. He was sent to live in forest land. This forest land belongs to other 
person. He said there is no compensation for his previous land, which is a productive 
land. (Facilitator reflection #5, Sangkat Orussey Kratie Province on 19 November 
2011)
During the CD event, people are courageous in raising their ideas. One participant 
mentioned that some criminals dressed in police or military uniform. However, the 
police officers do not dare arrest those criminals because they are afraid that those 
offenders might have some powerful person behind them. At the end of the event, local 
officials raised some solutions and asked people to have faith in them while they are 
trying to solve the problems. They asked people to be patient; that they needs time to 
develop our country. (Facilitator Reflection. Sangkat Orussey Commune, Kratie 
Province on 19 November 2011)
CPP MNA did not get support from people because he avoided responding directly to 
the concerns raised. He spoke unclearly because he is old. He always repeated his 
words. (Sangkat Orussey Commune, Kratie Province on 19 November 2011)
People were satisfied with opposition parties MNA's responses because MNA made 
direct responses. [The opposition] MNA said that the government must pay proper 
compensation to the land owners. Land owners -- 25 families -- did not receive proper 
compensation after their lands were confiscated for a university construction project 
(200 hectares). (Sangkat Orussey Commune, Kratie Province on 19 November 2011)
Sangkat Orussey Matrix-Identified Issues: Corruption, Infrastructure, health clinics, water and sanitation 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-42
Kampong Chhnang Province
Matrix-Identified Priorities
Pech Chanvar Ta Ches Krang Leav
Land seizure Land seizure Land seizure
Infrastructure Corruption Infrastructure 
Money/cost of living Infrastructure Land 
Youth gangs & drugs 
Table 5: Deliberative Group Priorities by Province and Commune 
In Kampong Chhnang, citizens’ farms are located in the midst of intensive land seizure activities. 
Deliberative groups across all three communes ranked land seizure and corruption among their top 
priorities. In Kran Leav commune, where citizens ranked corruption as their number one priority, 
facilitators recorded a strong police presence. Citizens in Krang Leav discussed farmers and families 
being pushed off their lands by land concessions made to the Pheapimex Company26 and for the building 
of an airport. 
Pech Chanvar: The observers noted a weaker presence of CPP supporters and a stronger presence of 
opposition party supporters at the CD. Observers did not detect any significant police presence and did 
not feel that the attendees were intimidated. Observers agreed or were uncertain if MNAs responded to 
citizens’ questions during the CD. However, some observers noted that some of the MNAs spent time 
attacking and debating the others. The observers were also split on whether or not the MNAs spoke 
harshly to the people. While the observers agreed the people were generally satisfied with the MNAs 
responsiveness to questions, they disagreed that the people were satisfied with the ruling party’s 
responses. Conversely, observers agreed that the people were satisfied the opposition parties’ responses, 
but were split on citizens’ satisfaction with the responses of their local officials.
The following facilitators’ observations describe the environment of the Pech Chanvar commune CD and 
the way in which MNAs spoke harshly to CD participants: 
I found that there were few CPP members present because when the CPP MNA made 
an opening remark there was little sound of applause. (Pech Chanvar Commune, 
Kampong Chhnang Province on 13 August 2011) 
The opposition party had a significant number of its members present at the 
constituency dialogue. In the beginning, there were only three or four members but 
after the program [had been going for] about an hour, some ten other member of the 
opposition party participated. (Pech Chanvar Commune, Kampong Chhnang 
Province on 13 August 2011) 
 
26 A 2007 UN Report notes that the Kampong Chhnang is one of the two provinces where the greatest amount of 
land has been conceded. Land concessions to the Pheapimex Company spans both provinces and amounts to 
315,025 hectares (Economic Land Concessions in Cambodia: A Human Rights Perspective. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-43
CPP MNAs just said that what a participant said was not true --the participant had 
accused the CPP [of something]. The other 3 parties’ MNAs did not speak harshly to 
people. (Pech Chanvar Commune, Kampong Chhnang Province on 13 August 2011)
The CPP MNA used the time for responding to the people's concerns by speaking 
threateningly and blaming the people for not knowing the laws or the policies, while 
the Sam Rangsy party MNA used the time to respond to all the people's questions and 
requests. (Pech Chanvar Commune, Kampong Chhnang Province on 13 August 
2011)
When H.E. Kim Sophearin made his responses, the majority of people paid the most 
attention and listened. The participants showed their satisfaction by nodding their 
heads to his responses and when he finished his responses, almost all the people gave 
strong applause. (Pech Chanvar Commune, Kampong Chhnang Province on 13 
August 2011)
I think that the MNAs used the time to discuss and seek the root causes of the problems 
in order to find the solutions but there was also some verbal attack on the party's 
policies. (Pech Chanvar Commune, Kampong Chhnang Province on 13 August 2011) 
When the opposition party MNA answered to the problems as well as giving solutions, 
he addressed the real problems as well as the problems in Pech Chanvar commune; 
then those problems were gradually reduced to some extent. Also he stated clearly the 
methods to solve all the people's problems in the community with strong determination 
and confidence. (Pech Chanvar Commune, Kampong Chhnang Province on 13 
August 2011) 
Pech Chanvar Matrix-Identified Issues: Land seizure, infrastructure, cost of living, youth drug gangs 
Ta Ches: Similar to Pech Chanvar, observers perceived a slightly higher presence of opposition party
members at the CD. Most of the observers disagreed that there was a significant police presences at the 
CD and did not observe intimidation of the citizenry. Observers reported that the MNAs were generally 
responsive to questions; that they did not attack or debate each other; and they did not harangue the 
audience. The observers noted a general satisfaction with the MNA’s, but disagreed that citizens were 
satisfied with CPP responses. They did agree that citizens were satisfied with the responses of the 
opposition MNAs as well as those of their local officials. 
Facilitators shared the following observations about the Ta Ches Commune CD event: 
There are not many CPP members presented at the CD. They didn't show off their 
power. Their participation is normal. (Ta Ches Commune, Kampong Chhnang 
Province on 17 November 2011)
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-44
We can notice that there are many opposition parties members present at the CD as we 
can see through the satisfaction made to the solutions raised [offered]. (Ta Ches 
Commune, Kampong Chhnang Province on 17 November 2011)
Ta Ches Matrix-Identified Issues: Land seizure, corruption, Infrastructure 
Kran Leav: Observers estimated a roughly equal presence of opposition and ruling party supporters at the 
CD. However, observers were divided on the strength of the police presence, but they did not perceive 
that citizens felt intimidated by the presence of the police. They agreed that the MNAs generally 
attempted to respond to citizen questions, did not waste time debating and attacking each other or the 
members of the audience. Observers agreed that attendees were generally satisfied with the MNAs 
responses, especially the opposition party MNAs. However, observers were split on citizen satisfaction 
with the MNAs from the ruling party: roughly half agreed that citizens were satisfied and the other half 
disagreed that citizens were satisfied with CPP responses. Observers largely disagreed that citizens were 
satisfied with the responses of their local officials. Facilitators observed the following CPP MNA 
responses to the people’s questions at the Krang Leav commune CD event: 
There were [a] few CPP members attending the constituency dialogue and people 
showed their support by giving applause and listening attentively to the issues Her 
Excellency raised and [how she] addressed the people’s concerns. (Krang Leav, 
Commune, Kampong Chhnang Province on 10 March 2012)
MNAs responded directly to the concerns raised, but the CPP MNA's mostly responded 
based on the law and the local official's solutions. (Krang Leav, Commune, Kampong 
Chhnang Province on 10 March 2012)
People were not satisfied with the CPP MNA's responses because the responses were 
based on the local officials. The MNA didn't focus on the questions raised by voters. 
(Krang Leav, Commune, Kampong Chhnang Province on 10 March 2012)
When responding to the questions raised, CPP MNA did not show how to solve the 
problems. MNA only [repeated] what he/she heard from other village people. MNAs 
did not have a proper solution regarding the concession of lands granted to the 
Peapimex company. People were unhappy and said that ‘if so, we should not demand 
our land back. Let's give all of the lands to the company’. (Krang Leav, Commune, 
Kampong Chhnang Province on 10 March 2012)
Kran Leav Matrix-Identified Issues: Corruption, Infrastructure, Land Seizures 
Summary of Findings for Deliberative Session Participant Engagement at CDs 
 Behavior of Deliberative Session Participants at CD: Deliberative session participants were more 
active at the CD than other attendees, raising their hands and speaking at their CD events more than 
twice as often as citizens at the CD who had not participated in a deliberative session. Women who 
had participated in deliberative sessions spoke at their CDs one and a half times more often than 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-45
female attendees who had not participated in a deliberative session. Most dramatically, male 
participants in deliberative sessions raised their hands and spoke at their CD events two and half 
times more often than their male counterparts who attended the CD but had not attended a 
deliberative session. 
 The Constituency Dialogue Environment: Opposition support seemed to be healthy at almost all 
nine CD events. At most of the events MNAs behaved civilly with each other and their constituents. 
However, there was also a significant police presence at a third of the CDs and in some cases police 
seemed to be actively attempting to intimidate citizens. In two out of nine CD events, MNAs 
attacked each other and harangued their constituents. In only one commune Observers reported
citizen satisfaction with the ruling party’s responses to citizens’ concerns. In a third of the communes 
citizens seemed satisfied with the responses of their local officials; in a third observers were split on 
their satisfaction with local officials’ responses; and in a third observers were unable to decipher 
citizen satisfaction with the responses of local officials. 
 Constituency Dialogue & Priority Concerns: Although the CD environment was often civil, the CDs 
took place against a backdrop of corruption and intimidation that was not always apparent during the 
public CD event. In two CD events– Pech Chanvar, Kratie and Ta Ches, Kampong Chhnang – where 
observers reported a healthy opposition party presence, deliberative session participants ranked land 
seizure at the top of the communes’ concerns, and described intimidation by local authorities. Indeed, 
the deliberative sessions discussed land seizure activities most intensively in the Ta Ches commune. 
In the three communes where there was a heavy police presence, deliberative groups ranked 
corruption highly on their matrix priorities. Although observers perceived general citizen satisfaction 
with the public performance of the MNAs, and some satisfaction with local officials, in the majority 
of the communes deliberative groups listed corruption among their top priorities: corruption in aid 
and infrastructure (Kampong Reab); corruption and land seizures (Pech Chanvar, Ta Ches, and Krang 
Leav); and corruption in service delivery (Thmar Andaek and Sangkat Orussey). 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-46
V. Discussion of Findings 
NDI sought to examine the effect of rural citizens having more time and space before the constituency 
dialogue to formulate their concerns, deliberate on solutions and build consensus and confidence with 
their peers on the issues that should be brought to the attention of the MNAs. The qualitative findings 
contributed to answering three of the hypothesis questions relevant to deliberation in the random control 
trial. The qualitative data on citizen priorities (hypothesis question #3), while not experimental, provided 
important contextual details on the problems Cambodian citizens face. The qualitative data contributed 
experimental evidence relevant to hypothesis questions #5 and 7, providing insight into the influence of 
gender on deliberative processes and outcomes. Further, the observational data from the CDs helped to 
identify differences in behavior between those citizens who had participated in deliberative sessions and 
those who did not. These qualitative findings: a) lend insight into NDI’s programmatic innovation in 
citizen deliberation; and b) speak to the broader program hypothesis and theory of change for 
constituency dialogue programs. 
a. Testing a Variation in the CD Program –Deliberative Sessions 
Proponents of deliberative democracy posit that public reasoning and debate not only enhance social 
cohesion, but create outcomes of better citizens, better decisions and better or more legitimate systems. 
In contrast to aggregative democracy in which preferences are formed in private and then expressed and 
added together in public through polling, focus groups and elections, deliberative democracy provides a 
fair and transparent forum for citizens to identify and solve concrete problems despite their differences. 
Moreover, experiments in deliberative polling in 15 countries have demonstrated that citizen deliberation 
enhances citizen knowledge of issues and politics; strengthens their ability to formulate opinions; and 
stabilizes opinions on issues once they are formulated. At the same time, other experiments in citizen 
deliberation have demonstrated that deliberative processes can be influenced by gender norms and 
inequities, while deliberative outcomes can be influenced by moderators of deliberative forums. 
Deliberation and Citizen Priorities: Unlike Fishkin’s experiments in deliberative polling, the study’s 
survey results did not find that deliberative session participants improved their knowledge of issues, 
politics or confidence beyond the effects created by their participation in the constituency dialogue 
program alone (see quantitative findings in Evaluation Report: The Constituency Dialogue Program in 
Cambodia). However, the qualitative differences between baseline survey- and matrix-identified 
priorities; the shift in rank order of priorities; and the emergence of new priorities underscore the 
substantive differences between citizens’ collective deliberation on an issue and individual survey 
responses. Thus the qualitative findings align with Fishkin’s proposition that purposeful peer discussion 
and analysis reflects citizen judgment and not mere opinion, and therefore yield outcomes that are 
substantively different from opinion polls or focus groups. Furthermore, the details on sensitive issues 
such as land seizure and corruption indicate citizens’ increased confidence and willingness to discuss such 
matters with a small group of peers. Moreover, the deliberative process provides better and richer 
information on citizens’ analysis of their problems, the multiple impacts and their prospective solutions, 
making the outcomes of deliberative sessions more relevant to specific programmatic needs than 
aggregated opinion polls or one-off focus groups before the CD. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-47
These findings on the deliberative sessions have implications for future programming designed to 
strengthen the demand side of the democracy equation. Future investment in citizen engagement should 
move beyond simply extracting information from citizens via focus groups and/or opinion polls, to more 
purposefully investing in citizens’ capacity to formulate and press their concerns in the public sphere. 
While deliberative sessions would enhance the quality and reliability of information on citizens’ priorities, 
they should not be used as a stand-alone intervention or as an appendage to the current program. Given 
the risks that citizens run in discussing issues like corruption and intimidation, it would not be prudent to 
simply extract information from citizens through deliberative session of focus groups. Rather the 
deliberative process should be seen as a foundation for bolstering and supporting citizen voice, agency 
and empowerment to re-negotiate relationships of power within and outside their communities. Indeed, 
the deliberative sessions, when structured along gender lines, have an intrinsic value of strengthening the 
public voice of women and men within their communities. As such, support for citizen deliberation 
should be considered a starting point for more comprehensive programming for strengthening the demand 
side of the democracy equation in Cambodia, and not simply to serve the informational needs of MNAs 
involved in NDI’s CD program. 
Gender and Deliberation: When the study’s survey results were disaggregated by gender the random 
control trial did detect some differential shifts in priority problems caused by the deliberative sessions. 
Similarly, the deliberative data in this report demonstrate that the gender composition of the deliberative 
groups themselves influenced the way in which groups ranked their priority problems and voted on 
priority solutions. This affirms Karpowski’s view that women’s minority status in a group, along with 
gender norms tend to thwart the emergence of women’s public voices. In rural Cambodia where women 
are not free to participate in public forums without their small children, engaging with a small group of 
women and a woman facilitator creates an enabling environment where childcare responsibilities may be 
momentarily shared or at least understood by other women in the deliberative group. When given the 
opportunity to participate in an enabling environment women do engage in the deliberative process and 
their outcomes are their own – free from the potential for stronger male voices or social norms to obscure 
them. 
The influence of gender on deliberative processes and outcomes speaks to the importance of providing a 
fair process through which more marginalized citizens may collaboratively analyze their situation with 
peers on equal footing. This is true for other variables such as age, ethnicity, and/or economic status of 
deliberative participants. In this case, providing an all-female forum and/or a female facilitator 
temporarily removed some of the inherent gender inequities that rural women face in Cambodia. 
Similarly, all-male groups helped to distinguish the priorities of men within the communities. In this way 
the gendered priorities identified in the deliberative sessions have the potential to inform more gender￾sensitive policies and budgets at the local and national levels, creating the potential for representatives to 
make better policy decisions. 
Deliberative Session Participants’ Engagement at CD: Despite their enthusiasm the day before, not all 
deliberative session members were able to participate in their CDs the following day. However the 
behavioral observations of deliberative session participants suggest that those that did attend were 
motivated to bring their matrix-identified priorities to the attention of their MNAs. Within the mixed 
gender context of the CDs women generally participated less than men. However, both male and female 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-48
deliberative session participants were more active in raising their concerns at the CD than were their 
counterparts at the CD. While not experimental, this speaks to the notion that the opportunity for 
deliberation makes for better citizens who not only understand their options within a democracy, but 
actively exercise their rights and responsibilities. That said, the survey measurements taken after the CD 
did not indicate that deliberative session participants increased their confidence in the political process. 
Rather, like their counterparts who only participated in the CD, deliberative session participants had 
mixed outcomes in confidence and comfort in discussing politics publically. Clues to what might be 
perceived as skepticism of the political process in Cambodia were found in the qualitative data from the 
deliberative sessions and the observational data from the CDs, both of which illuminated the mediator 
factors of corruption and intimidation.
The Constituency Dialogue Environment: The balance between opposition and ruling parties, along 
with the general adherence of MNAs to the code of conduct, presented a picture of healthy political 
competition in which MNAs publically and civilly addressed citizens’ concerns. However, the qualitative 
data revealed that many of the CDs took place against a backdrop of corruption and intimidation that was 
not always apparent during the public event. When viewed against the qualitative descriptions of land 
seizure, corruption and complicity of local officials, the heavy police presence at a third of the CDs 
becomes more ominous. Indeed, a number of observers documented the thinly veiled attempts of police 
to intimidate attendees who publically raised their concerns at the CD. The day before their CDs, 
deliberative session participants had recounted stories of land grabbing, killing of animals, leveling of rice 
paddies, poisoning of water sources, dispossession of farmers and rural flight to escape retaliatory 
prosecution and imprisonment. 
Data from the deliberative sessions and the CD observations underscore the importance of understanding 
the local context in which a program takes place, especially for democracy and governance interventions. 
Unlike Fishkin’s experiments which were conducted in largely stable democracies, Cambodia is a fragile 
democracy afflicted with systemic corruption and repression of free speech. While the deliberative 
sessions – and indeed the CDs themselves – demonstrate that a fair and transparent process is needed to 
bring citizens’ concerns to the fore, within a context where the state has allied itself with private 
enterprise to captured public institutions and resources for private gain, citizens who raise their concerns 
do so at great personal risk. Given this environment at the local level, it becomes clear that public 
reasoning and debate, while necessary, is not sufficient for pressuring MNAs to address citizens’ concerns. 
b. Program Hypothesis – Constituency Dialogue Program
Returning to the program hypothesis of the CD program itself, the broader experimental study was meant 
to better understand the demand side of the program’s results chain – constituents create pressure on MPs 
to address concerns, thereby contributing to more responsive and accountable MNAs. 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-49
Figure 11: Program Hypothesis for NDI’s Multiparty Constituency Dialogue program in Cambodia 
The survey data from the random control trial demonstrated that participation in the CD did indeed 
improve constituents’ understanding of their roles and options in the democratic process – the first link in 
the results chain. However, the experiment did not demonstrate that this new knowledge and shift in 
attitudes was linked to citizens putting additional demands on their MNAs to address their concerns – the 
next link in the results chain. 
The deliberative sessions were a relatively modest program activity and the descriptive findings suggest 
that providing time and space for citizens to discuss and come to consensus on their concerns may 
contribute to increasing their activity during the CD event. However, these findings were not 
experimental -- as participation in the CD event was self-selected and not randomized – and therefore not 
conclusive. Further data are needed to explore: a) whether changes in knowledge and attitudes are 
sustainable beyond the study’s timeline; and b) whether differences in citizen activities at the CD are due 
to citizen deliberation or some other variable. 
In addition, the broader context of the intervention and its variation is important to take into account. 
While the nine CD events presented a picture of healthy political competition in which ruling and 
opposition parties adhered to a code of civility, the details from the deliberative sessions documented a 
context of corruption, intimidation and sometimes violent retribution for citizens who raised their voices 
at the local level. The deliberative sessions and the CDs may create temporary space for citizens to voice 
their concerns, but individuals do so at great personal risk. Even when forums are hosted by an 
international entity such as NDI or USAID, a third of the constituency dialogues in this study had a heavy 
police presence where observers documented the attempts of the police to intimidate the attendees. 
During the deliberative sessions a number of facilitators also documented veiled intimidation by local 
officials during the small group sessions. The program’s critical assumption that citizens improved 
understanding of democracy will lead to increased pressure on MNAs did not seem to hold true in the 
study. 
The CD program is creating increased knowledge, along with temporary space and opportunity for 
citizens’ voices to emerge. However the broader intervention strategy should be reviewed to identify 
more systematic ways to strengthen accountability between citizens and their local and/or national 
representatives where feasible in the Cambodian context. Although the CD program records and tracks 
ANNEX II to CEPPS NDI Cambodia Impact Evaluation Report Page II-50
the promises of MNAs to their constituents during the CDs – and some positive outcomes have emerged 
from this process – the lines of accountability are between the MNAs and NDI and not between MNAs, 
local officials and their constituents. While the ‘thin edge of the wedge’ strategy to creating new political 
space and increased accountability is an effective entry point for a new program, the strategy is not 
sustainable and temporary gains in accountability and voice may be at risk of disappearing at the end of 
the life of the program. In the long run the public performance of political competition and debate may 
contribute to a change in public discourse, citizen expectations and socio-political norms. However, 
against a backdrop of state capture, systemic corruption and intimidation, and no real mechanisms for 
accountability between citizens and their representatives, or public officials and the judiciary, the 
improved public dialogue of the MNAs and their constituencies may not be contributing to real or lasting 
change. 
National Democratic Institute 
455 Massachusetts Ave, NW, 8th Floor 
Washington, DC 20001