FINAL REPORT: IMPACT EVALUATION OF USAID/NEPAL’S POLICY DIALOGUE ACTIVITY (NITI SAMBAD) ELECTORAL DEBATES AND DISCUSSIONS DRG Learning, Evaluation, and Research (DRG-LER) Activity TASKING N033 Contract No. GS-10F-0033M/AID-0AA-M-13-00013 AUGUST 2018 This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago, with Nahomi Ichino from the University of Michigan as Principle Investigator. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | i IMPACT EVALUATION OF USAID/NEPAL’S POLICY DIALOGUE ACTIVITY (NITI SAMBAD) ELECTORAL DEBATES AND DISCUSSIONS FINAL REPORT DRG Learning, Evaluation, and Research (DRG-LER) Activity TASKING N033 AUGUST 2018 Prepared under Contract No.: GS-10F-0033M / AID-OAA-M-13-00013 PREPARED BY: Nahomi Ichino, University of Michigan SUBMITTED TO: Brandy Witthoft CONTRACTOR: NORC at the University of Chicago Attention: Renée Hendley, Program Manager Bethesda, MD 20814 Tel: 301- 634-9489; E-mail: Hendley-Renee@norc.org DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | ii TABLE OF CONTENTS EXECUTIVE SUMMARY....................................................................................................................................... 1 BACKGROUND .................................................................................................................................................... 1 DEBATES AND DISCUSSION............................................................................................................................ 2 Research Questions and Hypotheses......................................................................................................... 4 Development of the Debate Questions.................................................................................................... 5 ORIGINAL METHODOLOGY........................................................................................................................... 6 Research Site Selection.................................................................................................................................. 6 Respondent Sampling, Invitation, and Baseline Survey ........................................................................... 7 Random Assignment....................................................................................................................................... 7 Discussion Groups .......................................................................................................................................... 8 Survey Data Collection and Permissions................................................................................................... 8 IMPLEMENTATION .............................................................................................................................................. 8 Training and Deployment.............................................................................................................................. 8 Debates .............................................................................................................................................................. 9 November 27 EC Press Conference.......................................................................................................... 9 REVISED METHODOLOGY .............................................................................................................................10 Key assumptions for the updated methodology....................................................................................11 IMPACT OF SCREENING/DISCUSSION SESSIONS..................................................................................14 Positive impacts on efficacy and engagement in discussion................................................................. 15 Changes to issues and policy orientation, with differences by gender.............................................15 Gendered impact on views of the political parties, but little effect on election behavior ..........16 IMPACT OF DEBATE SCREENING AND TYPES OF DISCUSSION GROUPS..................................22 SUMMARY AND FUTURE PROGRAMMING..............................................................................................27 REFERENCES.........................................................................................................................................................28 APPENDIX A. SUMMARY ANALYSIS ............................................................................................................29 APPENDIX B. PLANNED DEBATE QUESTIONS.......................................................................................41 APPENDIX C. PARTICIPATING PARTIES (CANDIDATES) ...................................................................43 APPENDIX D. RANDOMIZATION AT SIGN-IN.......................................................................................44 APPENDIX E. RESPONDENT CHARACTERISTICS..................................................................................45 DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | iii LIST OF FIGURES Figure 1: Session Availability .............................................................................................................................10 Figure 2: Schematic of Updated Design .........................................................................................................11 Figure 3: Age Distribution of Participants, by district.................................................................................13 Figure 4: `Jhapa: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; all outcomes, all respondents (men and women together)...........................................................17 Figure 5: Jhapa: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. ...................................................................................................................18 Figure 6: Jhapa: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. ...................................................................................................................18 Figure 7: Sunsari: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; all outcomes, all respondents (men and women together)...........................................................19 Figure 8: Sunsari: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. ...................................................................................................................19 Figure 9: Sunsari: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only ....................................................................................................................20 Figure 10: Kanchanpur: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; all outcomes, all respondents (men and women together). ....................................................20 Figure 11: Kanchanpur: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. ...................................................................................................................21 Figure 12: Kanchanpur: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. ...................................................................................................................21 Figure 13: Summary of Debate and Discussion Design; attendees (red) were randomized into four groups ...................................................................................................................................................22 Figure 14: Vote for Nepali Congress, for men and women, pooled across all districts; comparison of single-gender discussion only with debate plus single-gender discussion (Data: baseline survey, post-discussion survey, endline survey)..........................................................25 Figure 15: Vote for Communist Party (MC or UML), for men and women, pooled across all districts; comparison of single-gender discussion only with debate plus single-gender discussion (Data: baseline survey, post-discussion survey, endline survey).........................25 Figure 16: Vote for Nepali Congress, for men and women, pooled across all districts; comparison of single-gender discussion, mixed-gender discussion, and no-discussion groups; all groups watched debate (Data: baseline survey, post-discussion survey, endline survey) 26 Figure 17: Vote for Communist Party (MC or UML), for men and women, pooled across all districts; comparison of single-gender discussion, mixed-gender discussion, and no￾discussion groups; all groups watched debate (Data: baseline survey, post-discussion survey, endline survey)......................................................................................................................26 Figure A 1: Baseline: Participation in political discussion and access to information ..............................30 Figure A 2: Endline: Participation in political discussion and access to information................................31 Figure A 3: Jhapa: proportion of discussion participants (by respondent gender and treatment group) willing to consider voting for a minor party, after the discussion and at endline32 Figure A 4: Sunsari: proportion of discussion participants (by respondent gender and treatment group) willing to consider voting for a minor party, after the discussion and at endline33 Figure A 5: Kanchanpur: proportion of discussion participants (by respondent gender and treatment group) willing to consider voting for a minor party, after the discussion and at endline..............................................................................................................................................34 Figure A 6: Jhapa: rating of whether candidate is qualified after watching the debate, before discussion (by respondent gender); each panel is a different party .......................................35 DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | iv Figure A 7: Sunsari: rating of whether candidate is qualified after watching the debate, before discussion (by respondent gender); each panel is a different party .......................................36 Figure A 8: Kanchanpur: rating of whether candidate is qualified after watching the debate, before discussion (by respondent gender); each panel is a different party .......................................36 Figure A 9: Jhapa: Trustworthiness of candidates after discussion (by respondent gender and treatment group)................................................................................................................................37 Figure A 10:Sunsari: Trustworthiness of candidates after discussion (by respondent gender and treatment group)................................................................................................................................38 Figure A 11:Kanchanpur (female respondents only): Trustworthiness of candidates after discussion (by treatment group).........................................................................................................................39 Figure A 12:Kanchanpur (male respondents only): Trustworthiness of candidates after discussion (by treatment group).........................................................................................................................40 LIST OF TABLES Table 1: Participation in screening/discussion sessions.............................................................................12 Table 2: Randomized treatment assignment among attendees...............................................................23 DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | v ACRONYMS CPN-UML Congress and the Communist Party Nepal-Unified Marxist Leninist CPN-MC Community Party Nepal-Maoist Centre EC Electoral Commission MP Members of Parliament NC Nepali Congress NDI National Democratic Institute NORC National Opinion Research Center (NORC at the University of Chicago) PI Principal Investigator RCT Randomized Control Trial SLC School Leaving Certificate USAID United States Agency for International Development DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 1 EXECUTIVE SUMMARY While candidate debates are a regular part of political life in the United States and many established democracies, they are relatively new in many parts of the developing world. Proponents of debates argue that they could nudge political discourse towards policy debates, away from personalities and identity politics, and could increase voter knowledge and help voters make more informed decisions, thereby helping voters hold elected officials accountable and improving governance. This impact evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) – Electoral Debates and Discussions finds generally positive but modest and mostly short-term impacts of screenings of candidate debates and small-group discussions hosted by community radio stations ahead of federal parliamentary elections in Kanchanpur, Jhapa, and Sunsari Districts in Nepal in 2018. They improved participants’ sense of political efficacy and swayed participants’ views about the role of government and their issue priorities towards the positions held by and issues discussed by the candidates taking part in the debates. Participants’ evaluations of candidates who were judged to be the best performers in the debate became more positive. But the debate screenings and discussions did not consistently raise participants’ expectations of their Members of Parliament (MPs). In addition, explicit discussions of gender and social inclusion in the debates did not sway participants’ attitudes on these issues. Also, they did not affect participants’ stated preferred candidate on the endline survey. Note also that there are significant differences across constituencies. Women-only discussion groups focus more on social and employment difficulties faced by women than do men-only or mixed-gender discussion groups. Participating in mixed-gender discussions as compared with single-gender discussions pull men more in favor of the Nepali Congress and more willing to consider voting for a minor party candidate, while we find the reverse effect among women. Watching the debate and participating in discussions appear to change participants’ evaluations of the trustworthiness and qualifications of the candidates, but these short-term changes do not translate into changes in which candidates they supported at endline. The lessons that can be learned from this impact evaluation are unfortunately more limited and less certain than originally hoped for due to two unexpected difficulties during the implementation. The first of these is that program and research activities had to be cancelled following a press conference by the Chief Election Commissioner on November 27, 2017, only halfway through the planned activities. The design and questions to be addressed were adapted after this event. The second is that the participating candidates were not asked questions in a consistent manner at the debates, making it difficult to assess participants’ knowledge and learning about the candidates. These issues are discussed in detail below. BACKGROUND Nepal, a country of nearly 29 million people as of 2016, is divided into three distinct geographical regions – the Terai, the hills, and the mountains. The low-lying Terai region adjoining India contains approximately half (50.3%) of Nepal’s population (Population, Education, & Health Research Center 2016). There is great social diversity in Nepal, with the 2011 census recording 125 castes, 7 religions, and 123 languages spoken in Nepal. Per capita income in Nepal has risen from 210 USD in 1990 to 730 USD in 2016, but 25% of the population was still under the poverty line in 2011 (World Bank 2016). Nepal has undergone several major changes to its political system over the last few decades. The Communist Party of Nepal-Maoist mounted a civil war from 1996 to 2006, which killed at least 13,000 people and displaced upwards of 200,000 people. The monarchy was abolished after the civil war, and Nepal held its first constituent assembly elections in 2008. Although the Maoist party won a majority of DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 2 the seats, it failed to draft a new constitution with disputes over the fairness of the elections, the meaning of federalism, and protests by several groups who feared political marginalization. Negotiations in the next constituent assembly, elected in 2013, were also stalled as two devastating earthquakes struck Nepal in the spring of 2015. The need for massive reconstruction spurred the Constituent Assembly to fast-track the constitution-writing process. On September 20, 2015, 84% of the Constituent Assembly endorsed Nepal’s new constitution. It declares the Nepali people to be a multi-ethnic, multi-lingual, multi-religious, and multi-cultural nation. It also declares Nepal an inclusive and socialism-oriented republican state with a parliamentary form of government. The new constitution changed Nepal from a unitary to federal state, creating local, provincial, and federal levels of government, with elections for each level. Any Nepali citizen above 18 years of age may stand as a candidate and may vote in federal, provincial, and local elections. The Federal Parliament consists of two bodies. The more important of the two is the House of Representatives with 165 members elected through a first-past-the-post system from 165 constituencies and 110 members elected through a closed-list proportional-representation electoral system, with the entire country as one constituency. The Prime Minister and Council of Ministers are formed from this chamber. Organizations representing Madhesi and Tharu ethnic groups led a number of violent protests against the new constitution over concerns about structural discrimination (International Crisis Group 2016) and demands to have their own province in which they would not be minorities. In response to these protests, the Constituent Assembly passed two amendments to the constitution in January 2016, prioritizing population over geography in the demarcation of constituencies (Freedom House 2017), but these changes did not result in provincial boundaries aligned with these minority ethnic groups. The boundaries of 165 federal electoral constituencies were finalized by a commission on August 30, 2017, and under constitutional law cannot be altered for 20 years. The first-ever local elections in Nepal were held in early 2017, with a turnout rate of around 75%, even in areas where one of the major political parties representing Madhesi interests called for a boycott. The impacts of the debates and the discussions that are evaluated in this report took place ahead of the concurrent elections to the provincial and federal legislatures scheduled in two phases on November 26 and December 7, 2017. DEBATES AND DISCUSSION In this context, USAID engaged NORC at the University of Chicago to evaluate one component of its programming to support accountability and good governance in Nepal. USAID and National Democratic Institute (NDI), together with the Principal Investigator (PI), Nahomi Ichino, chose to evaluate the impact of exposure to candidate debates, sometimes with accompanying small-group discussion, on citizens. The outcomes evaluated include citizens’ political participation and efficacy; citizens’ issue￾priorities and orientations towards the role of government; citizens’ evaluations of and support for political parties, including willingness to consider voting for small parties; and citizens’ expectations for MP performance and beliefs about patronage in the political system. Debates are opportunities for candidates to engage in policy discussions, which encourage them to state their policy positions for the public and to put gender and social inclusion issues in the forefront. Consequently, debates instigate policy-based contests as opposed to personality-based contests. They may also help set standards against which elected officials can be held accountable by voters. NDI organized candidate debates for two municipal races in previous elections in Nepal and has supported candidate debates in new democracies around the world. For 2017, NDI selected the DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 3 Samriddhi Foundation to organize three candidate debates and recruited the community radio stations Kanchanjunga FM (Jhapa), Saptakoshi FM (Sunsari), and Suklaphata FM (Kanchanpur) to host screenings of the recorded debates and discussions in their local areas. NDI also engaged Madhu Acharya as a consultant to support the local FM stations’ activities. Three recent evaluations, based on randomized controlled trials (RCTs), suggest some promise for debates in low-information environments. Bidwell, Casey, and Glennerster (2016) randomized exposure to screenings of recorded debates between parliamentary candidates from different parties in large gatherings at the polling centers during the 2012 parliamentary elections in Sierra Leone. They found increased knowledge of candidate qualifications and policy positions, greater openness to vote for candidates from all parties, and larger correlation between voters’ policy preferences and their selected candidates. They also found that voters, irrespective of their ethnic backgrounds, responded to candidates’ performance even where political parties are strongly identified with particular ethnic groups. With a similar design to the Sierra Leone study, Platas and Raffler (2017) found that exposure to video￾recordings of candidates responding to policy questions increased citizens’ knowledge about parliamentary candidates in both the 2015 ruling-party primary elections and the 2016 general elections in Uganda. This large-scale study found no overall effects on voter turnout, although voters who were disappointed by their preferred candidates’ performance or learned that those candidates held more distant policy positions in the inter-party (general election) debate were more likely to stay home during the general election. Exposure to recorded primary election debates made primary election voters more likely to switch to the debate winner, but no such effects were found for the general election. Finally, in Ghana, Brierley, Kramon, and Ofosu (2018) found that exposure to debates led partisan voters to view candidates from the opposing party more favorably, but this effect disappeared within 2 days in constituencies that are strongholds of one party or another. We also investigate what additional impacts group discussions have following debate screenings, and in particular, how the gender composition of the discussion groups changes these impacts. Although these discussions are not aimed at reaching a consensus or decision, research on gender in deliberative settings in the United States suggests that “women’s issues’’ get more attention in groups with a larger proportion of women (Mendelberg, Karpowitz, and Goedert 2014). Listening to or participating in a discussion oriented around issues that are relatively more important to women than to men may emphasize and reinforce different information from the debate than discussions that revolve around other issues, and further shape the criteria by which participants may evaluate the candidates. Furthermore, more women may be inclined to participate in a women-only discussion than in a mixed￾gender discussion because they are less likely to be interrupted and face competition for speaking time (Mendelberg, Karpowitz, and Oliphant 2014). This positive experience with participation may increase women’s internal efficacy and engagement in politics, as Preece (2016) finds with positive feedback on political knowledge in the United States. Understanding the gender dynamics of these discussions can help inform future programming that includes small-group discussions with a gender dimension. That the post-screening discussions were mixed-gender (and hence likely organized around issues of relatively greater importance to men) may explain why some of the impacts of the debates were stronger for men than for women in the Sierra Leone study. It would be unsurprising to find heterogeneous effects by gender in Nepal, as well. Despite significant social changes wrought by the Maoist insurgency (K.C., Haar, and Hilhorst 2017), there remain vast gender inequalities in Nepal, including a strong preference for sons (Hatlebakk 2017) and the exclusion of women from consideration for local leadership positions (Panta and Resurrección 2014). It was only in 2017 that Parliament passed DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 4 a law outlawing the custom of chhaupadi, the banishment of women from the home during menstruation or after giving birth. Women and men in Nepal have very different roles, obligations, and opportunities in ways that could create a divergence of interests. RESEARCH QUESTIONS AND HYPOTHESES This impact evaluation focuses on the following questions • What is the impact of exposure to small-group discussions and screenings of candidate debates on citizens’: − Orientations towards the role of government and issue priorities, including gender and social inclusion issues? − Evaluations of and support for the political parties and candidates? − Willingness to vote for a different or minor party? − Expectations for the performance of the MP and beliefs about patronage in the political system? − Political efficacy and participation in politics? • How do these impacts differ for men and women and for supporters of different political parties? The hypotheses are that the discussions and debates, which featured discussions of corruption, development, and gender and social inclusion, will: • Increase citizens’ efficacy and participation in politics (discussion and voting); • Increase expectations of MP performance in office; • Reduce the expected role of patronage in politics; • Increase support for policies addressing gender and social inclusion issues; • Increase citizens’ willingness to vote for a minor-party candidate; • Increase citizens’ willingness to vote for the candidate judged to be the strongest performer in the debate; and • Increase alignment between a citizen’s preferences and that of the candidate he or she supports. There are two key assumptions underlying these hypotheses. The first is that Nepali citizens have fairly limited information about their candidates, given the new political institutions set up by the Constitution and the short time frame for the election campaigns, and that this problem is worse for minor-party candidates than for the major-party candidates. The second is that voters can understand and absorb new information when presented in an engaging visual format and respond to that new information in ways that are consistent with their interests. However, strong partisans of a party may process new information through a partisan lens – rejecting information that does not concord with his or her pre￾existing views – and be less responsive. In addition, men and women may respond differently for several reasons – they may have different preferences, they may have different levels of information prior to the debate screening/discussions, they may be more or less willing to listen, and they may process DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 5 information differently in discussions that focus on different subjects. Exploring these heterogeneous effects can help interpret why the hypotheses do or do not bear out. Despite some of the positive findings from previous studies, exposure to debates may have a different impact in Nepal for several reasons. First, unlike the other cases, Nepal has a parliamentary system and voters in the parliamentary election could be focused on the qualities of party leaders, who could become the future prime minister, instead of individual candidates. Therefore, information about individual parliamentary candidates may be less important to voters, although information about their parties’ policy platforms may remain very important. Second, Nepal has a much more fragmented and ideologically diverse political party system than the three African cases. Political parties in Nepal range from communists and Maoists on the left to conservatives on the right, and several parties represent lower-caste groups or the Madhesi community. It could be difficult for voters to shift support to parties associated with different ideologies or different ethnic groups even when they receive more information on candidates from these different parties through debate screenings. If voters support parties for their particular world views, rather than for their different approaches to achieving a shared vision for society, then it may be difficult for information conveyed in a debate screening to change vote choice. Nevertheless, one of the persistent concerns about politics in Nepal is the lack of substantive debates about public policy. Although the parties can be distinguished ideologically, all the major parties are led by elites from privileged caste/ethnic groups, and voters see the major parties as clientelistic and patronage-ridden. Voters’ concerns gravitate towards infrastructure and development, much like the Liberian and Ghanaian cases. Moreover, because of heavy state regulation and a variety of other reasons, there is limited television penetration in rural areas of Nepal. Given the difficult geography and very short campaign periods, Nepali citizens do not usually have the opportunity to see their candidates or party leaders; nor have they had the opportunity to hear their candidates discuss policy issues in an organized format. The debates are thus likely to provide significant new information to Nepali voters. The original design, described in greater detail below, randomized attendees into various combinations of debate screenings and small-group discussions. The design sought to assess the effects on the outcomes related to the research questions of participating in a small-group discussion over only watching the debate screening, and whether the gender composition of these small-group discussions modified these effects. This design now makes up the secondary analysis. The original design used small￾group single-gender discussion only (no debate screening) as the control condition. In addition to the hypotheses above comparing a debate screening-only arm to this type of control, the hypotheses are: 1) For women, the effects described in the hypotheses above will be greater for mixed-gender discussion groups than women-only groups. 2) For men, the effects described in the hypotheses above will the same for mixed-gender discussion groups and men-only discussion groups. 3) In addition, for women, assignment to single-gender discussion groups will generate greater participation in the discussion than assignment to mixed gender-discussion groups. 4) For women, single-gender discussion groups will generate more discussion of issues that are relatively more important to women than mixed-gender discussion groups. We expect that the reverse will hold for men. DEVELOPMENT OF THE DEBATE QUESTIONS The format of the debate and the questions that the moderator would ask the candidates were developed together by the PI and Coen Pustjens, Resident Senior Manager at NDI. The debates were planned to begin with the moderator introducing the role of MPs, the new federal form of government, DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 6 and the format of the debate. Then the moderator would invite each candidate to introduce himself or herself to the audience. To accommodate the impact evaluation, we agreed that the same questions would be asked of all the candidates and that the candidates would all have equal time to respond. The candidates were not given the questions in advance, but were informed of the format and the general topics on a “tip” sheet for successful debate performances, consistent with past NDI procedures. The debates were conducted in Nepali and broadcast live at the time. The questions for the first portion of the debate covered the economy and unemployment, migration, local development and infrastructure, and gender and social inclusion, and are listed in Appendix B, Planned Debate Questions. They were phrased prospectively so that the same question could be asked of all candidates, but no candidates would be prevented from discussing their past work. In the second portion of the debate, the moderator would take questions from a panel of local experts and members of civil society groups tailored to local concerns. For the purposes of the impact evaluation, only the first portion of the debate with the common questions were screened at the sessions. This was to ensure that consistent questions were presented to the participants and that the risk of losing the viewers’ attention because of a long debate were reduced. The goal was to keep the first portion of the debate to approximately 45 minutes while maintaining as much as possible of NDI’s usual debate format which usually takes 90-120 minutes. ORIGINAL METHODOLOGY As noted earlier, the November 27, 2017 Press Conference by the Chief Election Commissioner of Nepal required a revision of the methodology halfway through the field work. This section describes the original methodology for the impact evaluation. RESEARCH SITE SELECTION USAID Nepal originally selected Kanchanpur, Jhapa, Kailali, and Sunsari districts as research study areas for several reasons. These districts fall within their ongoing geographic areas of interest, have terrain favorable for radio signals from local FM stations, are located in the Terai or Hill regions where approximately half of Nepal’s population lives, and are socially and ethnically diverse. Kailali was later dropped because it would be impossible to conduct the fieldwork in 4 constituencies in the short period of time between the official nomination of candidates and the “quiet period” before the election in which no election-related activities are permitted. After the demarcation of constituency boundaries in August 2017 and the political parties announced which candidates would run in these first-past-the-post constituencies, NDI selected specific constituencies by considering three factors: whether it was likely that candidates could be convinced to participate in a debate; proximity to community radio-station partners; and the likely competitiveness of the elections. Neither USAID nor the implementing partner wanted the debate screenings to affect who the ultimate winner of these single-member district (plurality rule) seats would be and so avoided constituencies anticipated to have very close races where the intervention might be the difference. By these criteria, Constituency 1 in Jhapa District, Constituency 1 in Sunsari District, and Constituency 3 in Kanchanpur District were selected. NDI worked with the community radio stations to select venues for the sessions that could accommodate four simultaneous discussion groups and had a large space that could be darkened for the debate screening. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 7 RESPONDENT SAMPLING, INVITATION, AND BASELINE SURVEY The eligible population for the study was adult men and women who were eligible voters in these three constituencies. However, the voters registry was not legally available to us to use as a sampling frame. Solutions Consultant and NORC’s Survey Director Brian Kirchhoff selected 5 wards from each constituency, with consideration for proximity to the session venues and higher population density, and each ward was divided into two clusters using satellite imagery, and each cluster was overlaid with a grid. Four grid squares from each cluster were randomly selected for start points for sampling through a random walk method skipping 3 or 5 households. Within each household, an eligible voter of the designated gender was selected with a Kish grid after listing only those household members who were eligible to vote in that constituency. The randomly selected respondent was invited to a debate screening session, and if interested, to sign up for a particular date. Each constituency had 10 consecutive dates available. Each session date was available to a man (woman) until it had 50 men (women) signed up. Each night, Solutions Consultant would check the total number of signups separately for men and women; a session date was closed to new signups for potential male (female) participants if there were already 50 men (women) signed up. This procedure allowed us to limit the total number of signups for any particular day. Only respondents who signed up for a particular date were then asked to participate in the baseline survey. This allowed us to recruit 3,000 respondents in the limited time frame before the election. This sampling procedure contacts a random sample of eligible adults in the selected wards within the three constituencies, which is effectively a random sample of eligible adults in the selected constituencies with a bias towards wards with higher population density. Time constraints precluded conducting a full listing of the eligible population prior to the baseline or surveying those people who were contacted but declined to sign up for a debate screening session. The evaluation team is not aware of any social surveys in the study areas that could be compared with the study sample, which we expect to have higher levels of engagement and interest in politics than the general public in these areas. RANDOM ASSIGNMENT Only those respondents who came to a session hosted by the radio stations (attendees) were considered for the impact evaluation. The attendees were blocked by gender and session date, and within each block, randomly assigned with equal probability to one of four conditions: • Single-gender discussion only (no debate screening); • Debate screening only; • Debate screening followed by single-gender discussion; and • Debate screening followed by mixed-gender discussion. To have discussion groups of similar size, we had two mixed-gender discussion groups for every single￾gender discussion group following a debate screening. These simultaneous mixed-gender discussion groups were designated A and B. In practice, women and men checked in at different stations on arrival at the session venue and were assigned to a treatment arm according to their order of arrival at the session. To ensure that the number of subjects in each arm would be similar and to reduce complications at the sessions, we created 14 blocks of 4 male (female) potential attendees each and randomized within each of these DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 8 blocks to create a sequence of random assignments for men (women). Every 2 blocks of 4 attendees were paired (arrivals 1–4 with arrivals 5–8, arrivals 9–12 with arrivals 13–16, etc.), so that one block could be randomly assigned to mixed-gender discussion group A and the other to mixed-gender discussion group B. The sign-in sheet was color-coded with this sequence so that attendees were assigned to a treatment arm in order of arrival. An example of this sign-in sheet is included in Appendix D, Randomization at Sign-In. The screening/discussion sessions began as scheduled in Kanchanpur on November 21, in Sunsari on November 23, and in Jhapa on November 25. DISCUSSION GROUPS Each discussion group had a moderator. The men-only and women-only discussion groups had a discussion moderator of the same gender as the group. The discussions were approximately 30 minutes long, and moderators asked the participants to discuss with each other their views on the upcoming elections. All discussion moderators were instructed to allow the discussion to flow naturally, as if everyone were chatting at a tea shop, and to only intervene in the second half of the discussion if someone had monopolized the discussion or if the discussion had died off. SURVEY DATA COLLECTION AND PERMISSIONS Baseline and endline surveys were administered to all respondents at their homes. Those who were assigned to a debate screening also filled out a very short survey after the screening; those who were assigned to a discussion also filled out a very short survey following the discussion. Data collection was implemented by Solutions Consultant. NDI’s implementing partners contacted the relevant district-level Electoral Commission (EC) offices with whom the radio stations have good working relationships to inform them of the program and accompanying research activities and invited them to observe all the activities at the radio stations. The district-level EC officers agreed that the program and research activities were permissible under the Electoral Code. IMPLEMENTATION TRAINING AND DEPLOYMENT Enumerator training was conducted by Solutions Consultant and the NORC Survey Director in Kathmandu from November 2 to November 8, 2017. Enumerator teams were deployed to the research sites on November 10 and began baseline field work on November 14 after the session venues and locations were confirmed. All discussion moderators were recruited by the local FM stations and signed a Code of Conduct pledging professionalism, neutrality, and confidentiality of the discussions. The PI and Deewa Ghimire of Solutions Consultant trained the discussion moderators in Kanchanpur on the day before the first session. The PI and Madhu Acharya trained the discussion moderators in Sunsari on the morning of the first session and in Jhapa on the day before the first session. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 9 DEBATES The Samriddhi Foundation, with support from NDI in recruiting and encouraging the candidates to participate, held the candidate debates as scheduled in Kanchanpur on November 17, in Sunsari on November 21, and in Jhapa on November 23. They were televised live at the time, but otherwise not broadcasted. The Kanchanpur debate had 8 of 9 candidates participate, including candidates from the Nepali Congress (NC) and the Communist Party Nepal-Unified Marxist Leninist (CPN-UML). The Sunsari debate had 3 of the 9 candidates participate, and this did not include the candidate from the NC. Finally, the Jhapa debate had 3 of 7 candidates participate, and this did not include the candidate from the Community Party Nepal-Maoist Centre (CPN-MC), due to the unexpected death of the son of the Maoist leader. The participating parties/candidates are listed in Appendix C (Participating Parties (Candidates). At each debate, the moderator welcomed viewers and candidates and gave an introduction to the format of the debate. The candidates were either introduced by the moderator or offered an opportunity to introduce themselves. One difficulty for the impact evaluation was that the moderator did not ask the questions in the order they were drafted to all the candidates for equal amounts of time. Rather, the moderator asked his own questions, and usually different questions to different candidates, who were generally allowed to speak as long as they wished. NDI staff who attended the debates passed notes to the moderator during the debates and spoke with him after each debate to press the importance of asking the questions exactly as drafted, to little avail. This problem may have been avoided by having a research assistant take on the role of debate moderator. However, our priorities were to have a debate that would represent the real-life scenario with a confident and well-known media personality as a moderator, who can enliven the audience and interact professionally and with authority with the candidates. The moderator’s questions covered policy priorities on issues related to local development, corruption, gender inclusion, and emigration of youth seeking work abroad. These broadly cover the same topics as the drafted questions, but the inconsistency in how and to whom the questions were asked and their incongruence with questions on the baseline survey make it difficult to assess how the debate may have affected citizens’ opinions or knowledge of the candidates’ views on these subjects. In order to even out the time given to the candidates, to make more consistent the topics discussed, and to shorten the debate, the PI and Dev Mahato selected which segments of the debates should be kept in an edited version of the debate for screening. Madhu Acharya and the local television stations were able to quickly make these edits for each constituency. We asked the candidates participating in the debates to fill out a short survey after the debate; all except one did so. All except one of the responding candidates said that the debate treated him/her fairly as compared with the other candidates, and all reported that they would participate if more debates were organized in the future. Similarly, all except one said that the debate gave him/her the opportunity to talk about the issues he/she wanted to discuss and a strong majority of the participating candidates reported that they were able to convince voters to vote for them through the debate. When asked what should be changed to make the debates better, several participating candidates suggested better time management, attendance by all the candidates, and increased accessibility of the general public to the debates. NOVEMBER 27 EC PRESS CONFERENCE Following the successful conduct of Phase 1 of the federal elections in the more mountainous regions of the country, the Chief Election Commissioner of Nepal held a press conference on November 27. The DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 10 Chief Election Commissioner stated in this press conference that no polls asking about support for specific parties and candidates should be conducted or reported in the electoral period. The EC’s main concerns appeared to be that any reports of polls on support for specific parties or candidates could affect voting in Phase 2 and that any reports that do not match up with then-unannounced official results could lead to problems later on. The Chief Election Commissioner’s clarification of the Electoral Code of Conduct was unrelated to the programming or research activities with the partner community FM radio stations. After discussion with USAID, NDI, Solutions Consultant, and NORC, we initially decided to cancel the events at least for November 28. After further discussions on the 28th, we concluded that it would be very unlikely that we could gain permission for our research activities from the central EC office or to do so in enough time to restart the debate screening and discussion sessions. Therefore, the remaining scheduled sessions were cancelled and Solutions Consultant informed all participants by phone. Figure I presents the timeline of the scheduled sessions in each district, along with the date after which sessions were cancelled (November 27). Figure 1: Session Availability REVISED METHODOLOGY Unfortunately, the cancellation of debate and discussion sessions scheduled after November 28 left the impact evaluation with only half the number of sessions and discussion groups as originally planned. Because we had higher than expected turnout of participants, our sample size of individual participants at these sessions is 957, or a little over 60% of the anticipated sample size. The smaller sample size meant that we did not have enough statistical power for evaluating impacts according to the originally planned design. However, we can use the unexpected cancellation of the sessions for reasons unrelated to our activities in an instrumental variables analysis to investigate the average effect of the debate screenings and discussions among “compliers,” with the control condition being defined as not participating in any session (no debate screening or discussions). “Compliers” are the people who would participate if the session they had signed up for were available, but not participate if no session were available. Sessions scheduled for on or before November 27 were available, but those scheduled for November 28 or later were not. In Figure 2, among those who signed up for sessions scheduled on or before November 27, the attendees (yellow) are compliers and the non-attendees (purple) are non-compliers. However, we don’t know exactly which individuals among those signed up for sessions on or after November 28 are compliers or non-compliers, since they did not have the opportunity to come to a session. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 11 In the updated design, we use whether the respondent signed up for a session scheduled for on or before November 27 as an instrument for whether the respondent attended and participated in a screening and/or discussion. This is very similar to an RCT using an encouragement design wherein what is randomized is the opportunity (encouragement) to participate in some program. The crucial difference is that we did not control the availability of the opportunity, but the Press Conference generated the “randomization” of which days’ sessions were available or unavailable. For the revised methodology, we expanded the endline survey field work to re-interview all baseline survey respondents, not just those who attended the sessions, and to use these respondents who did not have the opportunity to be a participant at a session as the control group. We also pushed back the start of the endline survey field work from immediately after the December 7 election to January 2018 to be conservative in our compliance with the Electoral Code of Conduct. Figure 2: Schematic of Updated Design KEY ASSUMPTIONS FOR THE UPDATED METHODOLOGY There are two key assumptions for the instrumental variables design. First, the availability of the opportunity to participate in a screening and/or discussion is as if random. This availability depends on the date of the session selected by the respondent. Second, the only way that signing up for November 27 or earlier instead of November 28 or later had an effect on our outcomes is through participation in a screening/discussion session. Neither of these assumptions is obviously violated. Whether a respondent could have signed up for a particular session date depends on the date on which he was initially interviewed, since session dates would be closed to sign-ups once they reached 50 men and 50 women. Respondents who were interviewed later in the baseline survey would have had fewer dates to select from and those dates would be more likely to be those cancelled. Because Solutions Consultant selected random start points for their sampling of respondents, there is no particular reason to expect that those who signed up for earlier dates would be systematically different from those who signed up for later dates. This mostly bears out in the comparison of those who signed up for November 27 or earlier with those who signed up for November 28 or later (See Appendix E, Respondent Characteristics (by district and session date)). However, education was a significant predictor of whether respondents in Sunsari District signed up for November 27 or earlier or instead signed up for November 28 or later when included in a regression with these other baseline variables. The overwhelming majority of respondents could read and write, but literacy was lower in Sunsari than in the other two constituencies. Therefore, we include whether the respondent finished only SLC (school leaving certificate, at the end of secondary school) or lower DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 12 levels of education as a control variable in the analyses. Analyses without this control variable produce largely similar results. Some of those who signed up for November 28 or later were upset that the sessions they had signed up for were cancelled. This has three potential negative consequences. The first is that these respondents would refuse the endline survey interview leading to differential attrition. However, this was not the case in practice due to Solutions Consultant’s extensive efforts. The second is that being unhappy with the cancellation would lead respondents to answer the endline survey items with less care or purposeful incorrectness. Although we cannot verify that this did not occur, the responses from respondents who had signed up for November 28 or later do not appear to be any less coherent than those from respondents who had signed up for November 27 or earlier. Moreover, this would be quite a bit of trouble and a waste of the respondent’s time when he or she could simply refuse the interview. Finally, the respondent may have sought out the debate screenings or discussed politics with strangers on their own time because he or she had signed up for the cancelled sessions scheduled on November 28 or later and this would attenuate the estimated treatment effect. However, fewer than 4% of respondents who had signed up for November 28 or later report watching or listening to the debates on their own, so this problem is likely to be minimal. Of the 1,499 people who had signed up for one of the days in which the screening/discussion sessions were available, 949 attended, for an attendance rate of 63.3% (Table 1). In addition, 8 out of the 1,501 people who had signed up for later dates turned up on an earlier date and participated in the screenings/discussion. Almost all respondents (both those who attended and did not attend the sessions) spoke Nepali, were predominantly Hindu, and had a mean age between 38 and 43 in each of the districts. Table 1: Participation in screening/discussion sessions JHAPA SUNSARI KANCHANPUR TREATMENT AVAILABLE Attended session 163 348 446 Did not attend session 136 154 260 TREATMENT NOT AVAILABLE Did not attend session 701 498 294 Figure 3 presents the age distribution of participants for each district. The pink indicates the number of women, the blue indicates the number of men, and the purple reflects the overlap of the two. Nearly one-third of participants were aged 30 or younger, and the younger participants skewed female. Appendix E presents summary statistics on other characteristics of respondents. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 13 Figure 3: Age Distribution of Participants, by district DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 14 IMPACT OF SCREENING/DISCUSSION SESSIONS The overall impact of the screenings and discussions in different combinations is quite modest by the endline survey. In general, the screening/discussions had larger effects on more outcomes in Jhapa than Sunsari or Kanchanpur, with very little effect in Kanchanpur. In Jhapa, the debate screening/discussions had larger effects on men than on women for many outcomes. In Kanchanpur and Sunsari, they sometimes had effects in different directions for men and women, leading to null effects when pooling men and women in the analysis. The results are summarized first by outcome category and then constituency by constituency. Instrumental variables analyses are also conducted for subgroups that were pre-specified for the original randomized design. While those subgroup analyses should be considered more exploratory than definitive, they help uncover substantial heterogeneity that help explain the overall effects. Despite extensive discussion of gender and social inclusion in the debates, we find no effects of screening/discussions on attitudes towards these issues with one exception. The screening/discussions increased support for identity-based federalism in Sunsari. Approximately 2/3 of respondents do not support identity￾based federalism for Nepal. At the same time, nearly all respondents (97%) support stronger enforcement of laws against marriage by women younger than 20 years. Participants are generally quite fixed in their views on these issues. It is important to note, however, that the moderator deviated from the planned questions and raised the gender and social inclusion question with respect to what the political parties are doing within the parties themselves, rather than a question of policy for Nepal. Most candidates’ responses were descriptions of the diversity of the parties’ slates of candidates, so it would be difficult to expect changes in participants’ views on these policy issues. The screening/discussions had no average effect on self-reports of whether one voted in the December 2017 election. This is likely due to most people reporting that they intended to vote and did so, without our ability to verify the claim. The screening/discussions also had no average effect on participants’ expectations of the MP’s performance in office going forward. However, in all three constituencies, the screening/discussion raised expectations for the MP’s performance among those who had voted for the Nepali Congress in the local election. In Jhapa and Kanchanpur, the screening/discussion lowered expectations for the MP’s performance among those who had not voted for the Nepali Congress in the local election. Approximately 1/5 of respondents expected that the MP will do many good things for the constituency, while approximately 3/5 of respondents expected that the MP will do a few good things for the constituency. Overall, the debate screening/discussions had some positive effects on efficacy and participation in political discussion. These effects appear to differ for men and women and appear to be concentrated among those who voted for the NC in the local elections in some constituencies. This is consistent with the result on expectations of the MP’s performance for those participants who had voted for the NC in the local election. The screenings/debates also shifted participants’ views about the appropriate role of government and what issues rise to their top two local or national concerns. All effects are fairly modest in size and many differ for men and women, but do not differ very much by what party the respondent supported in the local elections. The debate screening/discussions shifted voters in different directions in the three constituencies, consistent with participants paying attention to and responding to the substantive content of the candidate debates. Only in Jhapa, where almost all respondents agreed that the candidate from the NC was the best performer in the debate, we find that the debate screening/discussions moved opinion in favor of the best debater. In Sunsari, about 60% selected the CPN-UML candidate as the best performer but the other two candidates each DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 15 had 20% of respondents select them as the best performer. In Kanchanpur, 30% of respondents named the CPN-UML candidate, 30% named the NC candidate, and 25% named the Naya Shakti Party candidate as the best performer. The results of the instrumental variables analysis are summarized in Figures 4-12. Because the number of days for which we were able to hold the screening/discussion sessions and the content of the debate differ across constituencies (7 in Kanchanpur, 5 in Sunsari, and 3 in Jhapa Districts), we analyze each constituency separately. All models use whether the signup date was on or before November 27 as the instrument for attendance and include no covariates. Figures 4, 7, and 10 pool men and women together for Kanchanpur, Sunsari, and Jhapa, respectively. Recall that in the updated methodology, all respondents who attend an event (regardless of whether they are assigned to a discussion or what type of discussion they are assigned to) are considered “treated.” In each figure, the estimated average treatment effect is accompanied by 90% (thick lines) and 95% (thin lines) confidence intervals. The estimate is marked by a dark blue triangle if it is statistically significant at the 95% level, by a light blue square if it is statistically significant at the 90% level, and by a hollow circle if the effect is not statistically significant at the 90% level. For clarity, Figures 5, 8, and 11 highlight only the outcomes for which there are substantive differences in average effects between men and women for each district. The red triangles and their 90% and 95% confidence intervals are for women; the blue squares and their 90% and 95% confidence intervals are for men. Similarly, for clarity, Figures 6, 9, and 12 highlight only the outcomes for which there are substantive differences in average effects between those respondents who supported the Nepali Congress in the local elections and those who did not. The dark blue triangles and their 90% and 95% confidence intervals are for those who supported the NC in the local elections; the light blue squares and their 90% and 95% confidence intervals are for those who did not. The following sections summarize the results for each outcome category by district. POSITIVE IMPACTS ON EFFICACY AND ENGAGEMENT IN DISCUSSION In Jhapa, the screening/discussions increased participants’ sense that they are informed and can understand politics (efficacy) and the frequency with which they engage in political discussions with family, friends, neighbors, and members of their community, with effects concentrated among the men. The screening/discussions also reduced how frequently they believed that they had to be careful about what they said in discussions around men or women. In Sunsari, the screening/discussions increased female participants’ sense of external efficacy and reduced how frequently women believed that they had to be careful about what they said in discussions around men or women. However, the positive effect on the frequency with which participants engage in political discussions with family, friends, neighbors, and members of their community appears to be concentrated among men and those who did not support the NC in the local elections. In Kanchanpur, the screening/discussions increased men’s sense that they are informed and can understand politics (efficacy), but decreased that of women’s. The screening/discussion had no effect on the frequency with which participants reported discussing politics overall. However, the screening/discussants appear to have increased efficacy and interest in politics among those who did not for the NC in the local elections, but had no effect on those who did. CHANGES TO ISSUES AND POLICY ORIENTATION, WITH DIFFERENCES BY GENDER In Jhapa, the screening/discussions also changed the issues that were raised as participants’ top two concerns. Participants became less likely to note destitution/poverty as one of their top 2 national issues and infrastructure/roads as one of their top 2 local issues, and became more likely to note education as DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 16 one of their top 2 local issues. In addition, the screening/discussions changed the general attitudes of male participants on the role of government, but not of female participants. Men became more accepting of income inequality with no limits to earning and more supportive of paying fees for medical services in order to increase standards of medical service delivery. In Jhapa, which is a CPN stronghold but did not have the candidate from that party taking part in the debate, more than 80 percent of respondents selected the NC candidate as the best debater, and the NC candidate was able to sway respondents’ views. About half of respondents who did not share the same view as the NC candidate during the baseline (pre-debate screening) survey on the role of government and limits to income inequality shifted their views closer to the NC candidate’s views after the debate screening. In Sunsari, similar to Jhapa, the screening/discussions changed the issues that were raised as participants’ top two concerns. Male participants became more likely to note unemployment as one of their top 2 national issues and both male and female participants became less likely to identify infrastructure/roads as one of their top 2 local issues. But in contrast to Jhapa, the screening/discussions made participants more likely to say that the government (as opposed to individuals) should be mainly responsible for the wellbeing of citizens and that it is better to be able to visit clinics and get medicines for free, even if this means that standards of medical service delivery cannot be raised. This is consistent with participants responding to the UML candidate’s position on this issue. The UML candidate differed from the other two candidates on this question, and among those respondents who did not have the same opinion as the UML candidate on the baseline survey, about one-third shifted their views closer to that of the UML candidate. Unlike Sunsari, but more consistent with the effects in Jhapa, the screening/discussions in Kanchanpur made female participants less likely to say that the government (as opposed to individuals) should mainly be responsible for the wellbeing of citizens (statistically significant at the 90% level). In Kanchanpur, those who voted for the NC in the local elections were unmoved in their issue orientations by the screening/discussions, while those who had not voted for the NC were swayed more. Taken together, these results suggest that participants respond to the substance of the debate and the arguments made by the candidates. Respondents’ assessments of who was the best performer in the debate were unrelated to their baseline issue orientations (𝜒𝜒2 test), which suggests that participants generally had an open mind in the screening/discussion sessions.1 In all three constituencies, the average effect of the screening/discussions on issue orientation and priorities for those who had and those who had not voted for the NC in the local elections appear to be very similar. GENDERED IMPACT ON VIEWS OF THE POLITICAL PARTIES, BUT LITTLE EFFECT ON ELECTION BEHAVIOR In Jhapa, the candidate from the NC participated in the debate, while the candidate from the CPN-MC did not, and we find that the screening/discussions helped the NC. Respondents reported a more 1 A 𝜒𝜒2 contingency table test on which party’s candidate the respondents felt was the best performer in the debate and what local issues were among the two most important to the respondents at baseline produces a p-value of 0.78 with df=330; the simulation-based p-value is 0.54. A 𝜒𝜒2 contingency table test on which party’s candidate the respondents felt was the best performer in the debate and what national issues were among the two most important to the respondents at baseline produces a p-value of 0.31 with df=364; the simulation-based p-value is 0.35. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 17 positive feeling towards the candidate and more trust in the party, were more likely to report contact with the NC during the campaign, and were more likely to consider voting for the party. The screening/discussions had no effect on participants’ consideration for voting for one of the minor parties, but made participants less likely to consider voting for the CPN-MC candidate for the Leftist Alliance. These effects are similar for those who did and did not vote for the NC in the local elections. In Sunsari, it was the candidate from the CPN-UML who participated and the candidate from the NC who did not participate in the debate. Here the screening/discussions decreased men’s trust in and feelings towards the CPN-UML, and appear to have moved men’s and women’s opinions of the NC in different directions. Participants became more likely to say that supporters of the winning party are more likely to benefit from patronage than non-supporters. This effect was concentrated among men. Also, we find that those who did not vote for the NC in the local elections became less trusting of the CPN-UML candidate and felt more positive towards the (absent) NC candidate, while those who voted for the NC in the local elections became more trusting of the CPN-UML candidate. The screening/discussions seem to have moderated these more personal orientations towards the parties, unlike the policy orientations, which were very similar for these two groups. In Kanchanpur, candidates from both major parties/alliances along with six minor party candidates participated in the debate, and the respondents became more likely to consider voting for one of the minor parties. The screening/discussions also pushed men and women in different directions with respect to the two major parties. Although the effects are not statistically significant, we find that the screening/discussions pushed women towards more favorable views of the NC and pushed men towards more favorable views of the CPN-UML. And similar to the results in Sunsari, more personal evaluations such as trust and general feeling towards the major parties were somewhat moderated by the screening/discussions. Those who supported the NC in local elections became more positive towards the CPN-UML, while those who did not increased their trust in the NC. Figure 4: `Jhapa: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; all outcomes, all respondents (men and women together). DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 18 Figure 5: Jhapa: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. Figure 6: Jhapa: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 19 Figure 7: Sunsari: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; all outcomes, all respondents (men and women together). Figure 8: Sunsari: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 20 Figure 9: Sunsari: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only Figure 10: Kanchanpur: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; all outcomes, all respondents (men and women together). DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 21 Figure 11: Kanchanpur: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. Figure 12: Kanchanpur: Effect size estimates, with 90% (thick) and 95% (thin) confidence intervals; selected outcomes only. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 22 IMPACT OF DEBATE SCREENING AND TYPES OF DISCUSSION GROUPS Although unexpected developments cut into our sample size, we can still compare outcomes across randomized treatment arms with the understanding that all results should be taken as suggestive. Because of low statistical power the results in this section are neither conclusively positive nor null. Figure 13 summarizes the randomized design amongst the attendees. The design allows for two interesting comparisons: • The comparison of the two dark grey groups – debate screening followed by single-gender discussion as compared with single-gender discussion only – allows us to examine the impact of the debate screening, conditional on participation in a single-gender discussion group. • The comparison of the dark blue, dark grey, and light grey groups in the bottom row – mixed-gender discussion, single-gender discussion, and no discussion following the debate screening – allow us to examine the impact of different types of discussion groups, conditional on watching a debate screening. Figure 13: Summary of Debate and Discussion Design; attendees (red) were randomized into four groups DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 23 Table 2 (see next page) summarizes the number of men and women assigned to the different treatment groups among those who attended the sessions. These participants constitute the study sample for the analyses in this section. Note that all analyses are cross-sectional comparisons across randomly-assigned treatment groups. We find some interesting impacts of the debate screening and discussion type among the attendees. First, the most striking effect of discussion type is on the content of the discussions themselves. Reading and coding the discussions by the topics raised by participants themselves, women-only discussion groups (both following and not following the debate screenings) consistently raise employment opportunities for women as a major issue. This issue does not come up in the men-only or mixed￾gender discussion groups. It is also only in women-only discussion groups that participants advocate for voting for a female candidate because she is female. In other respects, the discussion topics are fairly similar across different groups. All types of discussion regularly raise roads/infrastructure, unemployment, and agricultural development. Most discussions also raise corruption and frustration with politicians coming to ask for voters’ support ahead of the election and then being unavailable after the election. Some discussions raise the question of whether they should vote for the party or an individual based upon the individual’s character, and group discussions vary on whether quotas/reservations are good or necessary for Nepal. Table 2: Randomized treatment assignment among attendees MEN WOMEN NUMBER OF DAYS Jhapa Total 82 81 3 Single-gender discussion only 21 20 Debate only 21 20 Debate + Single-gender discussion 20 21 Debate + Mixed-gender discussion 20 20 Sunsari Total 160 188 5 Single-gender discussion only 39 46 Debate only 40 47 Debate + Single-gender discussion 41 48 Debate + Mixed-gender discussion 40 47 Kanchanpur Total 210 236 7 Single-gender discussion only 53 59 Debate only 52 59 Debate + Single-gender discussion 53 60 Debate + Mixed-gender discussion 52 58 All Total 904 1010 15 Second, we often find effects in different directions for men and women, particularly immediately after discussions with many of these effects attenuated by the endline survey. Conditional on debate screening, men participating in a mixed discussion group were more likely to support for the NC and more willing to vote for a minor party compared to men participating in a single-gender discussion DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 24 group. However, women participating in a mixed discussion group were less likely to support for the NC and less willing to vote for a minor party as compared with participating in a single-gender discussion group. In Jhapa, participating in mixed or single-gender discussions after a debate screening had little effect on male respondents’ evaluations of whether candidates from minor parties are qualified. But women were more likely to evaluate minor-party candidates as qualified, after participating in a mixed-gender discussion group than in a women-only discussion group following a debate screening. Moreover, in Sunsari and Kanchanpur, mixed-gender discussion groups (relative to single-gender discussion groups) had different effects for men’s and women’s assessments of the qualifications and trustworthiness of Nepali Congress candidates. While men were more positive about the NC candidate with a mixed￾gender discussion, women were more negative. We see no such gendered differences with respect to the CPN-UML candidates in these two districts. The gendered nature of the reaction to the NC candidate invites further exploration. However, these differential impacts do not translate into differences on our main outcomes. We find no statistically significant effects on men’s or women’s efficacy, political participation, or expectations of MPs’ performance in office. Almost all respondents indicate support for gender and social inclusion issues at baseline, and we find no effects of the different treatment arms on their support at endline. The influence of strong debate performers on respondents’ attitudes towards the role of government and inequality is also not different by treatment arm. We find a strong secular trend of increasing support for candidates from the Leftist Alliance (CPN-MC and CPN-UML in our constituencies) over time (Figures 15 and 17), but no statistically significant differences across treatment arms on respondents’ choices at endline. Figures 14–17 show these differences for the respondents’ plan to vote for (at baseline) or self-reported vote (at endline) for the Nepali Congress or the Leftist Alliance in the federal election. In each of these figures, the y-axis is the proportion of the treatment group supporting the NC (Figures 14 and 16) or the leftist candidates (Figures 15 and 17). Within each subfigure, on the x-axis are the randomized treatment groups. The three left subfigures summarize male respondents and the three right subfigures summarize female respondents. Within each group of three subfigures, the left subfigure reports differences at baseline, the center subfigure reports differences after the discussion, and the right subfigure reports differences on the endline survey. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 25 Figure 14: Vote for Nepali Congress, for men and women, pooled across all districts; comparison of single-gender discussion only with debate plus single-gender discussion (Data: baseline survey, post-discussion survey, endline survey) Figure 15: Vote for Communist Party (MC or UML), for men and women, pooled across all districts; comparison of single-gender discussion only with debate plus single-gender discussion (Data: baseline survey, post-discussion survey, endline survey) DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 26 Figure 16: Vote for Nepali Congress, for men and women, pooled across all districts; comparison of single-gender discussion, mixed-gender discussion, and no-discussion groups; all groups watched debate (Data: baseline survey, post-discussion survey, endline survey) Figure 17: Vote for Communist Party (MC or UML), for men and women, pooled across all districts; comparison of single-gender discussion, mixed-gender discussion, and no￾discussion groups; all groups watched debate (Data: baseline survey, post-discussion survey, endline survey) DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 27 SUMMARY AND FUTURE PROGRAMMING Despite the unexpected difficulties with the EC and consistent evidence of heterogeneity in the debate screenings’ effects, it is important to note that most citizen and candidate participants had very positive responses to participating in the debates and discussion activities. Participants were attentive to the debate screenings, reflecting the dearth of information about the candidates and of engagement between candidates and citizens who are not members of their parties. Discussion transcripts suggest respectful, thoughtful conversation with broad participation. The correspondence between changes in participants’ views on the role of government and some policies and the substantive content of the debates suggests that debates and discussions may push citizens towards a more policy-oriented political discourse. Although the effects of these screenings/discussions were modest, there are potentially very significant up-sides and little down-side to expanding this programming for the 2022 elections. From an impact evaluation perspective, an important consideration is how the particular way in which the sample was generated qualifies the applicability of the study’s findings to potential future beneficiaries should the programming be extended to new areas or new populations through changes in the eligibility criteria. Future support for debates from USAID is likely to be directed again to the Mission’s priority zones, and specifically within those priority zones, to areas where political and other considerations make it more likely that the implementing partner will be able to organize debates. For the evaluation team, it is difficult to predict where it will be feasible to organize debates in the future, and how those areas will differ from those in 2017, since this may be affected by the candidates’ largely positive experiences with the debates and changes to the media landscape, as well as the larger changes in Nepalese politics and society. This study also specifically excluded politically competitive constituencies where the debate might affect the winner of the parliamentary election, but a future expansion may include more competitive constituencies. Since close races may induce candidates to sharpen their arguments and to attract more attention from voters, debates in those constituencies may have stronger effects. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 28 REFERENCES Bidwell, Kelly, Katherine Casey, and Rachel Glennerster. 2016. “Debates: Voting and Expenditure Responses to Political Communication.” Brierley, Sarah, Eric Kramon, and George Ofosu. 2018. “The Moderating Effect of Debates on Political Attitudes.” Freedom House. 2017. “Freedom in the World 2017: Nepal.” https://freedomhouse.org/report/freedom-world/2017/nepal. Hatlebakk, Magnus. 2017. “Son Preference, Number of Children, Education and Occupational Choice in Rural Nepal.” Review of Development Economics 21 (1): 1–20. International Crisis Group. 2016. “Nepal’s Divisive New Constitution: An Existential Crisis.” Asia Report, no. 276 (April). K.C., Luna, Gemma Van Der Haar, and Dorothea Hilhorst. 2017. “Changing Gender Role: Women’s Livelihoods, Conflict and Post-Conflict Security in Nepal.” Journal of Asian Security and International Affairs 4 (2): 175–95. Mendelberg, Tali, Christopher F. Karpowitz, and Nicholas Goedert. 2014. “Does Descriptive Representation Facilitate Women’s Distinctive Voice? How Gender Composition and Decision Rules Affect Deliberation.” American Journal of Political Science 58 (2): 291–306. Mendelberg, Tali, Christopher Karpowitz, and J. Baxter Oliphant. 2014. “Gender Inequality in Deliberation: Unpacking the Black Box of Interaction.” Perspectives on Politics 12 (1): 18–44. Panta, Smrittee K., and Bernadette P. Resurrección. 2014. “Gender and Caste Relations Amidst a Changing Political Situation in Nepal.” Gender, Technology and Development 18 (2): 219–47. Platas, Melina, and Pia Raffler. 2017. “Meet the Candidates: Information and Accountability in Ugandan Elections.” Population, Education, & Health Research Center. 2016. Nepal Population Report 2016. Kathmandu, Nepal: Government of Nepal Ministry of Population; Environment. Preece, Jessica Robinson. 2016. “Mind the Gender Gap: An Experiment on the Influence of Self-Efficacy on Political Interest.” Politics & Gender 12: 198–217. World Development Indicators 2016. 2016. Washington, D.C.: World Bank. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 29 APPENDIX A. SUMMARY ANALYSIS DISCUSSION AND ACCESS TO INFORMATION Figure A1 presents histograms summarizing responses from male and female respondents’ to the following three questions on the baseline survey: Top row: How frequently do you discuss political matters with friends, family, neighbors, and members of your community? (From left to right: Never, a few times a year, once a month, at least once a week, every day). Middle row: How often do you listen to the radio? (From left to right: Not at all, less than once a week, at least once a week, almost every day). Bottom row: How often do you watch television? (From left to right: Not at all, less than once a week, at least once a week, almost every day) Note that the distribution for men is left-skewed and the distribution for women is right-skewed, indicating significant gender disparities in engagement in political discussion, particularly in Jhapa. There also appears to be more gender disparity in frequency of listening to the radio than in frequency of watching television. Figure A2 presents histograms of the responses on the endline survey. The gender disparity in political discussion remains and appears to be greater for listening to the radio than on the baseline. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 30 Figure A 1: Baseline: Participation in political discussion and access to information Top row: How frequently do you discuss political matters with friends, family, neighbors, and members of your community? (From left to right: Never, a few times a year, once a month, at least once a week, every day). Middle row: How often do you listen to the radio? (From left to right: Not at all, less than once a week, at least once a week, almost every day). Bottom row: How often do you watch television? (From left to right: Not at all, less than once a week, at least once a week, almost every day) DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 31 Figure A 2: Endline: Participation in political discussion and access to information. Top row: How frequently do you discuss political matters with friends, family, neighbors, and members of your community? (From left to right: Never, a few times a year, once a month, at least once a week, every day). Middle row: How often do you listen to the radio? (Not at all, less than once a week, at least once a week, almost every day). Bottom row: How often do you watch television? (Not at all, less than once a week, at least once a week, almost every day). WILLINGNESS TO CONSIDER MINOR-PARTY CANDIDATES AFTER DISCUSSION AND AT ENDLINE Figure A3 presents what proportion of respondents in Jhapa who were assigned to one of the treatment groups that included discussion would consider voting for a minor-party candidate. Minor party is defined as neither the Nepali Congress nor the Communist parties. The first panel on the far left DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 32 presents responses for male respondents on the post-discussion survey by treatment group. Disc_single refers to single-gender discussion only (no debate screening), DD_single refers to single-gender discussion following debate screening, and DD_mixed refers to mixed-gender discussion following debate screening. The second panel summarizes the responses for male respondents on the endline survey by treatment group. The third and fourth panels summarize the responses for female respondents on the post-discussion and endline surveys, respectively. Both male and female respondents lose their willingness to consider voting for minor-party candidates between the post-discussion survey and the endline survey. Figures A4 and A5 present the same for Sunsari and Kanchanpur, respectively. The overall willingness to consider voting for a minor-party candidate is greater in Sunsari than in Jhapa. However, like in Jhapa, respondents lose their willingness to consider minor-party candidates between the post-discussion survey and the endline survey. By contrast, respondents in Kanchanpur were on average more likely to maintain their willingness to support minor-party candidates between the post-discussion survey and the endline survey. Figure A 3: Jhapa: proportion of discussion participants (by respondent gender and treatment group) willing to consider voting for a minor party, after the discussion and at endline The left two panels present male respondents; the right two panels present female respondents. Within each panel, Disc_single refers to single-gender discussion only (no debate screening), DebDisc_single refers to single-gender discussion following debate screening, and DebDisc_mixed refers to mixed￾gender discussion following debate screening. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 33 Figure A 4: Sunsari: proportion of discussion participants (by respondent gender and treatment group) willing to consider voting for a minor party, after the discussion and at endline The left two panels present male respondents; the right two panels present female respondents. Within each panel, Disc_single refers to single-gender discussion only (no debate screening), DebDisc_single refers to single-gender discussion following debate screening, and DebDisc_mixed refers to mixed￾gender discussion following debate screening. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 34 Figure A 5: Kanchanpur: proportion of discussion participants (by respondent gender and treatment group) willing to consider voting for a minor party, after the discussion and at endline The left two panels present male respondents; the right two panels present female respondents. Within each panel, Disc_single refers to single-gender discussion only (no debate screening), DebDisc_single refers to single-gender discussion following debate screening, and DebDisc_mixed refers to mixed￾gender discussion following debate screening. POST-DEBATE SCREENING ASSESSMENT OF WHETHER CANDIDATE IS QUALIFIED Figures A6 through A8 present male and female respondents’ average assessment after watching the debate of whether the candidate is qualified for office on a 5-point scale (1,2,3,4,5) for Jhapa, Sunsari, and Kanchanpur, respectively. This excludes respondents who were assigned to single-gender discussion only. Each panel refers to a different party. In Jhapa, from left to right, these are Communist Party of Nepal-Maoist Centre (CPN-MC), Mongol National Organization (MN), Nepali Congress (NC), and Sanghiya Samajbadi Forum Nepal (SSFN). In Sunsari, from left to right, these are Communist Party of Nepal-Unified Marxist Leninist (CPN-UML), NC, Rastriya Janata Party Nepal (RJPN), and SSFN. In Kanchanpur, from left to right, these are Bibeksheel Sajha Party (BSP), CPN-UML, NC, Nepal Dalit Party (NDP), Naya Shakti Party (NSP), Rastriya Janamorcha (RJM), RJPN, and SSFN. We find significant gender gaps in assessments of the candidates. Most prominently, the CPN-MC candidate was assessed far more favorably by men than by women in Jhapa, while the reverse held true in Sunsari. In Kanchanpur, we find that female respondents generally assessed minor-party candidates as higher quality than did male respondents. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 35 Figure A 6: Jhapa: rating of whether candidate is qualified after watching the debate, before discussion (by respondent gender); each panel is a different party DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 36 Figure A 7: Sunsari: rating of whether candidate is qualified after watching the debate, before discussion (by respondent gender); each panel is a different party Figure A 8: Kanchanpur: rating of whether candidate is qualified after watching the debate, before discussion (by respondent gender); each panel is a different party DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 37 POST-DISCUSSION ASSESSMENT OF THE TRUSTWORTHINESS OF CANDIDATES Figures A9 and A10 present how male and female respondents in Jhapa and Sunsari, respectively, assigned to three different treatment groups assessed the trustworthiness of candidates from different parties after the discussion. The four panels on the left summarize the responses of male respondents, each for a different party; the four panels on the right summarize the responses of the female respondents, each for a different party. Within each panel, Disc_single refers to single-gender discussion only (no debate screening), DD_single refers to single-gender discussion following debate screening, and DD_mixed refers to mixed-gender discussion following debate screening. Because of the number of candidates, Kanchanpur is split over Figures A11 (female respondents only) and A12 (male respondents only). In general, the candidates of the major parties (CPN-UML, CPN-MC, and NC) have an advantage over the less-known minor-party candidates. In Sunsari, there are significant differences between men and women in their assessments of the trustworthiness of the major candidates. Women find the CPN candidate more trustworthy than do men, and women find the NC candidate less trustworthy after watching the debate, while men do not. There are fewer differences between men and women in Kanchanpur, where Figure A1 indicated that there is a smaller gender disparity in participating in political discussion than in Jhapa or Sunsari. Figure A 9: Jhapa: Trustworthiness of candidates after discussion (by respondent gender and treatment group) The left four panels summarize responses of male respondents for each of four parties; the right four panels summarize responses of female respondents for each of four parties. Within each panel, Disc_single refers to single-gender discussion only (no debate screening), DD_single refers to single￾gender discussion following debate screening, and DD_mixed refers to mixed-gender discussion following debate screening. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 38 Figure A 10: Sunsari: Trustworthiness of candidates after discussion (by respondent gender and treatment group) The left four panels summarize responses of male respondents for each of four parties; the right four panels summarize responses of female respondents for each of four parties. Within each panel, Disc_single refers to single-gender discussion only (no debate screening), DD_single refers to single￾gender discussion following debate screening, and DD_mixed refers to mixed-gender discussion following debate screening. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 39 Figure A 11: Kanchanpur (female respondents only): Trustworthiness of candidates after discussion (by treatment group) Each panel is a different party. Within each panel, Disc_single refers to single-gender discussion only (no debate screening), DD_single refers to single-gender discussion following debate screening, and DD_mixed refers to mixed-gender discussion following debate screening. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 40 Figure A 12: Kanchanpur (male respondents only): Trustworthiness of candidates after discussion (by treatment group) Each panel is a different party. Within each panel, Disc_single refers to single-gender discussion only (no debate screening), DebDisc_single refers to single-gender discussion following debate screening, and DebDisc_mixed refers to mixed-gender discussion following debate screening. DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 41 APPENDIX B. PLANNED DEBATE QUESTIONS Each question was to be asked of each candidate at the debate, with a time limit of 1 minute. 1. The Economy and Unemployment − 1,400 to 1,800 youths are leaving Nepal every day to work abroad. Often under dire conditions. These youths are forced to leave Nepal because there are not enough jobs. What will you do to create more employment opportunities for youths in your constituency? Please be as specific as possible − It is estimated that more than 25% of Nepal's GDP/wealth (not everyone will understand what GDP is) comes from remittances of Nepali's working and living abroad. The third highest percentage in the world. In other words, Nepal's economy is heavily dependent on the money Nepali's are sending home from abroad. Much of that money is taxed by the government of Nepal. Please tell us how you will invest the money the government makes of remittances in your constituency? − With a lack of investment from government in the economy many Nepali's have started small businesses and shops themselves. There are many rules and regulations that make it difficult and expensive for Nepali's to start a business; getting permits, paying taxes and filing the right documents. What will you do to make it easier to start local businesses in your constituency? − A major obstacle to economic development in Nepal is the widespread corruption. Money that should be spend on developing local communities is often being pocketed by individuals for their personal gain. What will you do to curtail corruption in your constituency? 2. Local Development − What do you think is the most important local development issues for voters in your constituency and why? What will you do to solve that issue if you get elected? − Educating children is one of the best ways to improve people's lives. However, in Nepal many schools are in a bad condition, with old buildings, not enough teaching materials, overcrowded classes and teachers that have little training, leave early or do not show up at all. This is harming the future opportunities of children in Nepal. As a result many, often expensive, private school have arisen. But not everyone can afford to send their children there. What will you do to improve the education of public schools in your constituency? − A majority of households in Nepal do not have access to clean toilets (sanitation facilities) or safe drinking water? What will you do to improve access to clean toilets and safe drinking water for your voters? 3. Infrastructure − Many roads are unpaved or damaged, making travel for ordinary citizens difficult (or impossible during the monsoon). This does not only make it difficult for people to travel for work, to family or to buy food but also harms the economy. What will you do the coming years to improve these conditions? We would like you to specify which roads in your constituency you would build first and why would you chose those? DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 42 4. Inclusion − The new constitution of Nepal sets quotas for the political participation of women and people from marginalized communities. These groups have traditionally been excluded from politics and important decision making processes that could improve their lives. Do you support these quotas and can you give an example of what you have personally done to improve the inclusion of women and marginalized groups in your community and/or political party? − 37% of girls in Nepal marry before age 18 and 10 percent are married by age 15, in spite of the fact that the minimum age of marriage under Nepali law is 20 years of age. Data shows that Nepal has the third highest rate of child marriage in Asia, after Bangladesh and India. Do you agree that this law should be enforced stricter and what will you do to make sure this practice is banned in Nepal? DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 43 APPENDIX C. PARTICIPATING PARTIES (CANDIDATES) Parties (Candidates) with Debate Participants Bolded KANCHANPUR SUNSARI JHAPA Nepali Congress (Ramesh Lekhak) Nepali Congress (Bijay Kumar Gacchad) Nepali Congress (Biswo Prakash Sharma) Communist Party Nepal-- UML (Dipak Bhatta) Communist Party Nepal-- UML (Bhagbati Chaudhary) Communist Party Nepal--MC (Surendar Kumar Karki) Sanghiya Samajwaadi Forum Nepal (Mohan Bahadur Shahi) Sanghiya Samajwaadi Forum Nepal (Padam Kumar Adhikari) Sanghiya Samajwaadi Forum Nepal (Bhakta Bahadur Limbu) Bibekshil Sajha Party (Deependra Karki) Raastriya Janamukti Party (Bisweshwor Yadav) Raastriya Janamukti Party (Dan Bahadur Tamang) Nepal Dalit Party (Mahesh Tiruwa) Green Party Nepal (Santosh Timilsina) Mongol National Org. (Santosh Tamang) Raastriya Janata Party Nepal (Rubi Jairu) Raastriya Janata Party Nepal (Manish Parsad Rauniyar) Ekikrit Rastriya Prajatantra Party--Rastrabaadi Naya Shakti Party (Janak Singh Saud) Independent (Krishna Kumar Sanyasi) Sanghiya Loktantrik Rastriya Manch Rastriya Janamorcha (Lal Bahadur Kunwar) Independent (Bimal Kumar Paudel) Independent (Hira Singh Bist) Independent (Mo. Nasirudin Miya) DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 44 APPENDIX D. RANDOMIZATION AT SIGN-IN Example of sign-in sheet (Jhapa men, first session) DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 45 APPENDIX E. RESPONDENT CHARACTERISTICS (by district and session date) DISTRICT JHAPA SUNSARI KANCHANPUR Session on or before Nov 27 Yes No Diff Yes No Diff Yes No Diff Speaks Nepali 0.98 0.99 0.97 0.97 0.99 0.99 Female 0.52 0.51 0.51 0.51 0.50 0.51 Age 43.06 42.46 39.38 38.45 38.95 37.36 Hindu 0.92 0.87 * 0.91 0.93 0.99 1.00 Married 0.84 0.84 0.87 0.84 0.78 0.80 Can neither read nor write (%) 0.12 0.13 0.25 0.26 0.11 0.08 Can read and write 0.85 0.85 0.69 0.68 0.87 0.89 Head of household 0.62 0.61 0.59 0.49 ** 0.56 0.59 Read newspaper (freq, 1-4) 2.22 2.16 1.95 1.97 2.45 2.15 ** Listen to radio (freq, 1-4) 2.99 3.14 2.50 2.44 3.32 3.44 Watch TV (freq, 1-4) 3.68 3.72 3.57 3.57 3.60 3.60 Trust media (1-4) 3.30 3.23 3.24 3.24 3.15 3.10 Education (1-5) 2.66 2.50 2.19 2.42 ** 3.04 2.85 Political interest (0-3) 2.12 2.08 2.29 2.25 2.34 2.37 Political discussion (freq, 1-4) 2.11 2.04 2.38 2.23 2.40 2.37 Voted in local election 0.85 0.83 0.79 0.78 0.74 0.73 Voted for NC in local election 0.51 0.41 ** 0.43 0.35 * 0.53 0.51 Voted for CPN-UML in local election 0.41 0.49 * 0.39 0.41 0.43 0.37 Agree: government bears the main responsibility for the well-being of the people (1-4) 1.67 1.68 1.66 1.55 * 1.71 1.83 Agree: government should place limits people on how much rich people can earn (1-4) 1.99 2.01 2.03 1.90 1.91 1.97 Agree: better to raise health care standards, even if we have to pay medical fees (1-4) 3.01 3.04 2.84 2.97 2.98 2.97 Plan to vote for Democratic Alliance (0,1) 0.47 0.36 ** 0.36 0.25 ** 0.39 0.37 Plan to vote for Leftist Alliance (0,1) 0.33 0.43 ** 0.33 0.30 0.30 0.30 DRG-LER – Contract No. GS-10F-0033M/AID-OAA-M-13-00013 USAID.gov Impact Evaluation of USAID/Nepal’s Policy Dialogue Activity (Niti Sambad) Page | 46 DISTRICT JHAPA SUNSARI KANCHANPUR Session on or before Nov 27 Yes No Diff Yes No Diff Yes No Diff Efficacy: external (1-5) 3.20 3.10 3.29 3.34 3.48 3.43 Efficacy: informed (1-5) 3.35 3.27 3.34 3.35 3.64 3.60 Efficacy: internal (0,1) 0.40 0.37 0.47 0.47 0.46 0.48 Plan to vote (0,1) 0.97 0.99 0.93 0.89 0.90 0.91 Feel towards NC (1-5) 3.78 3.68 3.88 3.83 4.00 3.95 Feel towards CPN (1-5) 3.14 3.37 ** 3.90 3.95 3.94 3.96 Unemployment in top 2 national issues (0,1) 0.53 0.53 0.45 0.48 0.64 0.66 Poverty/destitution in top 2 national issues (0,1) 0.25 0.27 0.27 0.34 ** 0.22 0.22 Infrastructure/roads in top 2 local issues (0,1) 0.50 0.54 0.46 0.56 ** 0.46 0.49 Education in top 2 local issues (0,1) 0.17 0.20 0.25 0.22 0.20 0.17 Note: * indicates difference in means is statistically significant at the 5% level, ** at the 1% level