October 2014 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Tomas Bridle, Julia Rizvi, William Cartier, Shantanu Majumder, AKM Saifullah, and Naim Mostofa for Social Impact, Inc. Insert high-quality photograph representing the project being evaluated to replace this instruction and fill this text box. Mid-term Performance Evaluation of the Democratic Participation and Reform Program Cover Photo: “In demand of at least one formalin-free bazaar in Barisal.” Democratic Participation and Reform Program Youth Fellows Alumni Association organizes a human chain to advocate for food safety at the local level in Barisal Division. (Photo Credit: Democracy International) MID-TERM PERFORMANCE EVALUATION OF THE DEMOCRATIC PARTICIPATION AND REFORM PROGRAM October 25, 2014 USAID Contract AID-OAA-I-10-00003 Task Order AID-388-TO-12-00001 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program ii CONTENTS Acronyms................................................................................................................................................................................................. iii Executive Summary................................................................................................................................................................................ iv The Development Problem and USAID’S Response ............................................................................................................... iv Evaluation Purpose and Evaluation Questions........................................................................................................................... iv Evaluation Methodology and Limitations ..................................................................................................................................... v Findings and Conclusions................................................................................................................................................................. v Recommendations ............................................................................................................................................................................ vi Introduction..............................................................................................................................................................................................1 The Development Problem and USAID’s Response......................................................................................................................2 The Development Problem.............................................................................................................................................................2 USAID’s Response.............................................................................................................................................................................2 Evaluation Purpose and Questions.....................................................................................................................................................4 Evaluation Purpose ............................................................................................................................................................................4 Evaluation Questions.........................................................................................................................................................................4 Evaluation Methodology and Limitations ..........................................................................................................................................5 Data Collection Methods.................................................................................................................................................................5 Limitations............................................................................................................................................................................................7 Findings and Conclusions......................................................................................................................................................................9 Relevance .............................................................................................................................................................................................9 Effectiveness ......................................................................................................................................................................................11 Sustainability ......................................................................................................................................................................................26 Recommendations ................................................................................................................................................................................28 A. Relevance..................................................................................................................................................................................28 B. Effectiveness.............................................................................................................................................................................28 C. Sustainability.............................................................................................................................................................................30 Lessons Learned....................................................................................................................................................................................31 Annexes...................................................................................................................................................................................................32 Annex I. Evaluation Statement of Work ....................................................................................................................................33 Annex II. DPR Project Results Framework...............................................................................................................................42 Annex III: Map of Evaluation Sites................................................................................................................................................43 Annex IV. Documents Reviewed .................................................................................................................................................44 Annex V. Persons Interviewed .....................................................................................................................................................47 Annex VI. Evaluation Matrix..........................................................................................................................................................51 Annex VII. Focus Group Discussion Protocols........................................................................................................................59 Annex VIII. Mini-Survey Protocols...............................................................................................................................................68 Annex IX. Mini-Survey of District Leaders ...............................................................................................................................69 Annex X. Disclosure of any Conflicts of Interest....................................................................................................................72 TABLES Table 1: DPR Project Objectives and Intermediate Results ........................................................................................................1 Table 2: Data Collection Sites.............................................................................................................................................................5 Table 3: Evaluation Key Informant Statistics by Data Collection Method ...............................................................................6 Table 4: DPR Surveys and Research Tools ....................................................................................................................................15 Table 5: Overview of Mini-Survey Results .....................................................................................................................................20 Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program iii ACRONYMS AL Awami League BDGPE Bangladesh Democracy and Governance Program Evaluations BEC Bangladesh Election Commission BNP Bangladesh Nationalist Party CATSS Computer Assisted Telephone Survey System CDCS Country Development and Cooperation Strategy COP Chief of Party CRI Center for Research and Information DCOP Deputy Chief of Party DFID Department for International Development DG Democracy and Governance DGRC District Grassroots Representative Conference DI Democracy International, Inc. DPR Democratic Participation and Reform EWG Election Working Group F Foreign Assistance Framework FGD Focus Group Discussion ICT Information and Communications Technology IFES International Foundation for Electoral Systems IR Intermediate Result IRI International Republican Institute JEC Justification for Exception to Competition JP Jatiya Party KII Key Informant Interview MP Member of Parliament NDI National Democratic Institute NGO Non-Governmental Organization NJSJ Narir Joye Shobar Joy PMP Performance Management Plan PRODIP Promoting Democratic Institutions and Practices PROGATI Promoting Governance Accountability, Transparency, and Integrity RFA Request for Applications RMG Ready-Made Garment RPO Representation of the People Order SBO Statistically-Based Observation SI Social Impact, Inc. SMS Short Message Service SPM Senior Program Manager TAF The Asia Foundation USAID United States Agency for International Development USG United States Government WYC Women and Youth Center Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program iv EXECUTIVE SUMMARY This document reports on a mid-term performance evaluation of the United States Agency for International Development (USAID)/Bangladesh “Democratic Participation and Reform” (DPR) program implemented by Democracy International (DI) from April 2011 to April 2016. DPR aims to increase participation and leadership by women and youth in political parties; develop the capacity of parties to conduct and utilize research; and enhance parties’ internal communication, promote organizational best practices, and ensure compliance with electoral law. THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE DPR was designed to address a number of interlocking development opportunities and challenges in Bangladesh—a country with a mixed history of democratic governance. The DPR Request for Applications (RFA) notes that Bangladesh democracy “has been interlaced with military rule, political polarization, weak institutions of governance, and pervasive corruption. Partisan deadlock over electoral issues led to the cancellation of the 2007 parliamentary election and the establishment of a military￾backed caretaker government, which ruled the country under a state of emergency for nearly two years.” In contrast, the parliamentary election of December 29, 2008 was seen as competitive, well administered, and participatory. In the wake of these elections, USAID saw an opportunity to support Bangladesh in seeking to consolidate and sustain the democratic gains and to strengthen the governance institutions and processes needed for economic and development progress. The DPR RFA directed applicants that the program “expected to focus development activities at the local level and primarily target the membership base and mid-level structure of beneficiary political parties.” USAID’s 2011-2016 Country Development and Cooperation Strategy (CDCS) calls for programs to support increased citizen engagement, which will “generate increased demand for democratic governance and better services, which in turn, will culminate in improved accountability and responsiveness to citizen needs.” Consistent with that strategy, DPR supports the objective of “Governing Justly and Democratically” and Intermediate Result (IR) 1.1: “Strengthened Political Processes” under USAID/Bangladesh’s Development Objective 1: “Citizens’ Confidence in Democratic Institutions.” Specifically, DPR seeks to 1) expand youth and women’s leadership in political parties; 2) improve information access and use of information and research by parties; and 3) strengthen parties’ responsiveness and support internal reform. EVALUATION PURPOSE AND EVALUATION QUESTIONS The objective of this mid-term evaluation is to assess DPR’s technical and programmatic validity; its effectiveness; and the sustainability of its interventions. It seeks to answer the following questions, in order of priority: 1. Are DPR’s objectives still relevant to the current development circumstances after the January 2014 parliamentary election? Is it logical for USAID to remain engaged in a project with political parties? 2. Has DPR been successful in achieving its objectives? Is DPR on course to meet set objectives? 3. Are DPR implementation tools and management efficient in meeting project objectives? 4. How effectively were women and youth engaged by DPR? Would alternative approaches have greater impact? Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program v 5. Has DPR’s monitoring and evaluation (M&E) system been effectively used to track, monitor, and report results, and what changes are required to make it more effective? How effectively has DPR coordinated with other USAID/Bangladesh programs and contributed to other USAID/Bangladesh objectives? 6. Is DPR ensuring sustainability of program activities beyond USAID support? What obstacles undermine sustainability and what measures should be taken to address them? EVALUATION METHODOLOGY AND LIMITATIONS The evaluation employed a mixed methods design: document review, key informant interviews (KIIs), focus group discussions (FGDs), and a mini-survey. The findings, conclusions, and recommendations are based upon data collected from a total of 163 key informants (105 males, 58 females). The evaluation team conducted fieldwork in Dhaka, Chittagong, Khulna, and Rangpur to achieve a purposeful sampling of sites with varying intensity of DPR project activity and to represent diversity in party support. The evaluation team sought to mitigate the limitations of qualitative methods related to recall bias, response bias, and selection bias. There were specific limitations, the first being the presence of other donor programs, which limits judgments about attribution. Also, due to time and budget constraints, the team collected data in four of the seven divisions in which DPR works. The evaluation team needed to adjust activities in response to travel restrictions associated with hartals, cancelling some of the scheduled FGDs and KIIs. Finally, there were limitations in assessing the achievement of DPR results against stated objectives because of weaknesses in DPR’s M&E system and reporting. FINDINGS AND CONCLUSIONS Relevance of DPR to the current context. Looking at the state of affairs among political parties, the evaluation finds that DPR activity objectives are still relevant. With closing of democratic spaces, political party reform offers the possibility of enhancing policy responsiveness of the Awami League (AL) and strengthening the democratic opposition of Bangladesh Nationalist Party (BNP) and Jatiya Party (JP). The DPR logic of focusing on sub-national levels has generated interest and enthusiasm. Success in enhancing grassroots participation. DPR has engaged youth through a fellowship program and supports their work with district party leadership. The fellowship program increased fellows’ motivation to participate in politics in a constructive way and developed their advocacy skills. The events fellows organized in turn raised awareness among citizens about their political rights and the political responsibilities of their elected officials. The number of fellows is relatively small (101 youth graduated from the program), but DPR also reached youth party activists through other means. According to the September 2014 PMP update, DI conducted a total of 854 events at the Women and Youth Centers (WYCs) with a total of 12,447 young political leader participants. There have been more convincing results in engaging women in party leadership; DPR has maintained M&E data showing notable progress in increasing women’s participation in district committees. Success in improving information access and utilization. DPR developed a range of high-quality tools to further this objective. There is evidence of the parties taking up research, but it is not well documented. Overall, party demand for these tools has been quite limited. Success in enhancing internal reform. DPR has been successful in engaging party leaders and promoting democratic procedures for choosing committee members in districts where it has worked, but most party committee procedures are still undemocratic and non-transparent. The ongoing district councils will be an opportunity to test DPR’s impact in this component. Effectiveness of management. DPR has a robust implementation approach through the regional WYCs, which successfully engage local political party leaders. While there is an effective management structure, Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program vi the evaluation team questions the efficiency of having four full-time expatriate staff through to project close-out. Effectiveness of DPR’s M&E system. While the evaluation team found evidence that the project is meeting its objectives, DPR’s major failing is an extremely weak M&E system. This failure constitutes a major missed opportunity to demonstrate project impact to USAID and other stakeholders. There is no evidence of sustained collaboration with other USAID programs. Sustainability of DPR impacts. DPR’s implicit impact model is to create “momentum” for reform by the example of the positive changes in targeted district party structures. This is consistent with the project design; however, DPR lacks a clear vision of the specific program strategies that will carry forward this momentum within and beyond the targeted districts. Other sustainability strategies in the Year 4 Work Plan are less important in this sense. RECOMMENDATIONS 1. USAID should continue its support for DPR and plan for continuation of Democracy and Governance (DG) programs that engage political parties in Bangladesh. Parties continue to be the prime political actors, and their actions impact all priority USG concerns. 2. Future political party programs should incorporate aspects of the DPR approach. Party programs in Bangladesh should maintain the focus on district and sub-district party structures, as well as continue to emphasize youth and women’s political participation and use that as a tool for broader opening of party structures. 3. USAID should act quickly in designing a follow-on program to ensure there is no gap in program activity. A gap in programming could undermine DPR’s hard-won accomplishments and complicate future program start-up. 4. DPR should continue and expand its work to increase women’s participation in party committees. This work should include support to women’s membership in the sub-district committees. DPR should revive its women candidate training program where there are opportunities in district and local elections and by-elections. 5. DPR should continue to implement the youth fellowship program and support the Alumni Association. In order to leverage impact, the program should also be limited to the districts where DPR is active. 6. DPR should increase resources for research at the district level to help party leaders better understand priorities in districts and articulate strategies and messages that address local concerns, especially in municipalities and city corporations. 7. DPR should review its methodology for national-level polling to ensure it provides quality information. DPR should conduct a thorough technical review of its survey research and ensure quality and consistency in data collection, survey instrument design, and use of results. 8. USAID should continue to support regional offices, modeled after DI’s WYCs, under future party programs. These offices will ensure the program’s ability to reach grassroots party units and activities. Future implementing partners should demonstrate capacity to manage this regional operating structure. 9. DPR should increasingly promote and train local staff to take on more responsibilities. Where feasible, DPR should plan a transition from expatriate to local component leads over the remaining life of project, freeing up budget resources for program activities. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program vii 10. DPR should continue to emphasize a strong role for women in the fellowship program and to support other project elements related to women’s participation. At the same time, DPR should conduct more mixed-gender activities with political party members. 11. USAID should work with DI to improve the design and use of the PMP, work plans, and progress reports—particularly with respect to baseline measurements and targets. DPR should enhance its use of survey research to populate PMP indicators and ensure that the PMP includes an indicator to assess democratic procedures in party committees. 12. USAID should more actively identify potential synergies and lead efforts to coordinate between DPR and USG programs in other sectors, such as labor rights and economic policy reform. There may be opportunities in policy areas such as health, women’s rights, and other USG priority areas. DI’s strong cooperation with the British Department for International Development (DFID) could also be an opportunity for engagement in other policy areas and development objectives. 13. DPR should strengthen its sustainability strategies for intra-party reforms in sub-national structures. While maintaining and deepening its focus on a limited number of “treatment” districts, DPR should extend its impact to “non-treatment” districts. Other sustainability issues are less of a priority and should be carefully assessed. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 1 INTRODUCTION Throughout Bangladesh’s political history, the country’s long-established political parties have invested much authority in their leaders but struggled over how best to encourage participation and incorporate input from their large and diverse memberships. Political parties have come to recognize a need to increase their meaningful interactions with voters between elections and to develop inclusive strategic visions for the future. The United States Agency for International Development (USAID) in Bangladesh, in partnership with the British Department for International Development (DFID), supports a five-year program entitled “Democratic Participation and Reform in Bangladesh” (DPR) to strengthen political processes and broaden political participation. The DPR cooperative agreement began on April 11, 2011 and is scheduled through April 10, 2016. Program funding is USD 12,364,850. Implemented by Democracy International (DI), DPR aims to increase opportunities for participation and leadership by women and youth in political parties; develop the capacity of parties to conduct and utilize research in their decision-making; and respond to the needs of political parties for stronger internal communication, organizational best practices, and compliance with electoral law. Gender and youth engagement is intended to be incorporated throughout all DPR objectives to ensure that women and youth benefit from and participate in all DPR activities. See Annex II for the DPR Results Framework and Table 1 for DPR project objectives and intermediate results (IRs). Table 1: DPR Project Objectives and Intermediate Results Objective 1: Enhance Grassroots Participation. IR 1.1 Leadership opportunities for youth are expanded. IR 1.2 Representation of women in political parties is increased. Objective 2: Improve Information Access and Utilization. IR 2.1 The political parties’ capacity to conduct and utilize research is strengthened. IR 2.2 Relevant research tools are developed to enable their use of existing and to-be-developed information resources. Objective 3: Improve the Environment for Responsive Politics. IR 3.1 Pathways of coordination, communication, and dialogue within the parties are improved. IR 3.2 Internal reform initiatives are supported. This document reports on the results of a mid-term performance evaluation of DPR, conducted by Social Impact, Inc. (SI) through the Bangladesh Democracy and Governance Program Evaluations (BDGPE) project. In what follows, the evaluation team lays out the development problem and USAID’s response; evaluation methodology and limitations; main findings and conclusions; lessons learned; and recommendations for USAID and DI going forward. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 2 THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE THE DEVELOPMENT PROBLEM Bangladesh’s political parties are the dominant force in democratic discourse, competition, and governance. The parties effectively control the national government institutions that make and implement economic and development policies. They are intimately connected with powerful national economic groups and with local economic elites. Their internal practices at the local level effectively determine the extent and forms of engagement between citizens and local government. Powerful individuals use their influence in the local party bodies—and the parties’ influence over local government—to shape policies and delivery of services. USAID’s analysis determined that engagement with political parties was critical to achieving U.S. Government (USG) governance, economic, and development objectives in Bangladesh. However, political parties in Bangladesh are characterized by dynastic leadership, undemocratic internal practices, and a history of resorting to violence to win elections and maintain power. These characteristics inhibit their ability to build democratic institutions, implement effective development policies, and focus on citizen priorities. Women and youth remain sidelined and underrepresented in internal party decision-making and activities. USAID concluded that while there was limited opportunity for systemic change in party operations at the national level, there had been some incremental progress toward greater internal democracy and openness at the district and sub-district, or upazila, levels of party organizations. USAID also noted that parties were not implementing the electoral law amendment issued in 2008 by the Bangladesh Election Commission (BEC) requiring 33 percent women’s participation in executive committee party positions at all levels by 2020—and that advocacy around its implementation could serve as a lever for change at the parties’ district and sub-district levels. USAID’S RESPONSE USAID’s 2011-2016 Country Development and Cooperation Strategy (CDCS) for Bangladesh states that, “promotion of democracy and governance (DG)… is critical to advancing Bangladesh’s development in each of the sectors of focus in USAID/Bangladesh’s CDCS.” The CDCS calls for programs to support increased citizen engagement, which will “generate increased demand for democratic governance and better services, which in turn, will culminate in improved accountability and responsiveness to citizen needs.” USAID/Bangladesh designed the DPR program to contribute to the achievement of the USG strategic objective “Governing Justly and Democratically” and IR 1.1: “Strengthened Political Processes” under USAID/Bangladesh’s Development Objective 1: “Citizens’ Confidence in Democratic Institutions.” DPR was designed to address a number of interlocking development opportunities and challenges in Bangladesh—a country with a mixed history of democratic governance. The DPR Request for Applications (RFA) notes that Bangladesh democracy “has been interlaced with military rule, political polarization, weak institutions of governance, and pervasive corruption. Partisan deadlock over electoral issues led to the cancellation of the 2007 parliamentary election and the establishment of a military￾backed caretaker government, which ruled the country under a state of emergency for nearly two years.” In contrast, the parliamentary election of December 29, 2008 was seen as competitive, well administered, and participatory. In the wake of these elections, USAID saw an opportunity to support Bangladesh in seeking to consolidate and sustain the democratic gains and to strengthen the governance Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 3 institutions and processes needed for economic and development progress. The DPR RFA directed applicants that the program “expected to focus development activities at the local level and primarily target the membership base and mid-level structure of beneficiary political parties.” Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 4 EVALUATION PURPOSE AND QUESTIONS EVALUATION PURPOSE The objective of this mid-term evaluation is to assess the technical and programmatic validity of the DPR project, assess DI’s performance to date in achieving results against targets, determine whether the project is on course to meet set objectives, and assess sustainability aspects of the project. The audience includes USAID/Bangladesh, U.S. Embassy Dhaka, DFID, USAID Asia Bureau, and other bilateral and multi-lateral donors or development partners focused on DG. EVALUATION QUESTIONS The evaluation sought to answer the following evaluation questions, in order of priority: Relevance 1. To what extent are DPR’s objectives still relevant to the current development circumstances, particularly after the January 2014 parliamentary election? To what extent is it logical for USAID to remain engaged in a project with political parties given the current state of affairs among the major parties? Effectiveness 2. To what extent has DPR been successful in achieving its objectives? To what extent is DPR on course to meet set objectives? 3. To what extent are DPR implementation tools and management structure efficient (in terms of cost effectiveness and program flexibility) in meeting project objectives? [USAID clarified that “cost-effectiveness” refers to the overall value and sustainability of its investment in assistance for political parties.] 4. How effectively were women and youth engaged by DPR’s interventions in the targeted areas? Are there any alternative approaches identified for greater impact? 5. To what extent has DPR’s monitoring and evaluation (M&E) system been effectively used to track, monitor, and report results attributable to DPR activities and what changes are required to make the performance management system more effective? How effectively has DPR coordinated with other USAID/Bangladesh programs (DG, Economic Growth, Population Health Nutrition and Education, Food Disaster, Humanitarian Assistance) in achieving DPR objectives and contributing to other USAID/Bangladesh objectives? Sustainability 6. How much progress is DPR making in ensuring sustainability of program activities beyond USAID support? Are there any obstacles that undermine the goal of sustainability, and what measures should be taken to enhance sustainability? Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 5 EVALUATION METHODOLOGY AND LIMITATIONS The evaluation team conducted fieldwork from September 9-23, 2014 in four divisions: Dhaka, Chittagong, Khulna, and Rangpur. Fieldwork locations were selected in consultation with USAID/Bangladesh and DI in order to achieve a purposeful sampling of sites with varying longevity and levels of DPR project activity as well as to represent a diversity of allegiances to Bangladesh’s major political parties. All team members participated in data collection in Dhaka and Chittagong. USAID/Bangladesh staff observed data collection in Chittagong. In order to complete evaluation activities in the allocated time, members divided into two sub-teams to conduct simultaneous fieldwork in Khulna and Rangpur. See Annex III for a map of data collection sites and Table 2 for the distribution of team members at each site. The evaluation team presented its initial findings, conclusions, and recommendations in separate out-briefs for USAID/Bangladesh and DI on September 24. Table 2: Data Collection Sites Chittagong September 12-15 Khulna September 16-17 Rangpur September 16-18  Team Leader: Tomas Bridle  Team Members: William Cartier, Julia Rizvi  National Team Members: Shantanu Majumder, Naim Mostofa  Team Leader: Tomas Bridle  National Team Members: Shantanu Majumder, Naim Mostofa  Team Member: Julia Rizvi  National Team Member: AKM Saifullah DATA COLLECTION METHODS The evaluation team employed a mixed methods design including four data collection methods: document review, key informant interviews (KIIs), focus group discussions (FGDs), and a mini-survey. The team’s findings, conclusions, and recommendations are based upon data collected from a total of 163 key informants (105 males, 58 females) as shown in Table 3. See Annex V for a list of key informants and Annex VI for an evaluation matrix.  Document Review: The team reviewed relevant documents produced by USAID/Bangladesh, including the 2011-2016 CDCS and RFA for the DPR program; documents produced by the DPR project, including annual work plans, the approved Performance Management Plan (PMP), quarterly progress reports, and monitoring data collected by DI in September 2014; and surveys, assessments, and research on political parties conducted by DI or its partners through the DPR project. See Annex IV for a list of documents reviewed by the evaluation team. The team’s findings and conclusions are informed by analysis of both quantitative and qualitative data in the documents.  Key Informant Interviews: The team interviewed a total of 51 key informants (31 male, 20 female) on an individual basis or in groups to maximize efficiency—depending on circumstances, appropriateness, and availability of resources. Target groups included: o Donors: USAID and DFID staff o Implementers: DI staff at headquarters, the Dhaka Field Office, and Women and Youth Centers (WYCs) in Chittagong, Khulna, and Rangpur Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 6 o Beneficiaries: alumni of the Developing Young Leaders Fellowship program, male and female members of the major political parties: Bangladesh Awami League (AL), Bangladesh Nationalist Party (BNP), Jatiya Party (JP) 1 o External Actors: political party members who were not direct beneficiaries of DPR project activities, representatives from other USAID/Bangladesh DG partners Table 3: Evaluation Key Informant Statistics by Data Collection Method Division Key Informant Interviews Focus Group Discussions Mini-Survey Subtotal Total Chittagong Males 7 13 10 30 51 Females 4 17 21 Dhaka Males 17 N/A N/A 17 25 Females 8 8 Khulna Males 1 9 10 20 35 Females 2 13 15 Rangpur Males 2 22 10 34 48 Females 6 8 14 United States Males 4 N/A N/A 4 4 Females 0 0 Total 51 82 30 M: 105 163 F: 58  Focus Group Discussions: The team scheduled six FGDs with DPR beneficiaries at each data collection site outside Dhaka: three FGDs with male political party leaders, and three FGDs with female party members. Due to hartals, or strikes, that took place during fieldwork, the team conducted only 13 of the 18 scheduled FGDs. Participants included 82 representatives (44 male, 38 female) from each of the three major political parties in each of the three data collection sites, and they were disaggregated by sex and political party to promote open dialogue. Participants were selected at random from a list provided by DI of district, upazila, and city corporation leaders who had participated in DPR activities in each division. FGDs were organized in accordance with the USAID Technical Note on Focus Group Interviews Version 1.0 (November 2013) and facilitated by National Team Members. See Annex VII for FGD protocols.  Mini-Survey: The team used a mini-survey to obtain data about the parties’ intentions and actions to organize democratic councils to elect party leadership at the district level. Participants in the mini-survey included 30 district leaders—all male2—from each of the three major political parties in each of the three data collection sites outside Dhaka. The evaluation team selected respondents randomly from the DPR project participant list provided by DI. The mini-survey was conducted by National Team Members via phone and consisted of closed-ended questions. The team used quantitative data produced through the mini-survey to triangulate qualitative data obtained through KIIs and FGDs. See Annex VIII for the mini-survey protocol and Annex IX for detailed findings. 1 After consultation with USAID and DI, the evaluation team concluded that it was not feasible to conduct data collection with members of Jamaat-e-Islami, the main Islamist party in Bangladesh. DPR worked with party members (especially with youth fellows) when possible, but legal restrictions on Jamaat-e-Islami’s participation in politics made it difficult to conduct trainings, conferences, and other activities on a regular basis. 2 The evaluation team attempted to reach the limited number of female district vice presidents in the DI database, but they were not available for participation in the mini-survey. Mini-survey statistics are reflective of the limited participation of women in district-level political positions. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 7 LIMITATIONS Attribution USAID/Bangladesh has provided assistance to political parties and the electoral process in Bangladesh since 2003. The fact that multiple donors and implementers have been actively working in the DG arena presents challenges for making judgments about attribution. Given that this limitation was understood by the evaluation team from the outset, this report focuses less on attribution and more on implementation, approach, and contributions of the DPR project. Generalizability Due to time and budget constraints, the team conducted data collection in only four of the seven divisions in which the DPR project has been implemented. Therefore, the team is somewhat limited in its ability to generalize findings beyond a sampling of respondents in four divisions to project beneficiaries in all seven divisions. However, the evaluation team contends that the sample size does not weaken overall evaluation findings. Availability of Data The evaluation team encountered limitations in assessing the achievement of DPR results against stated objectives because: 1) DI did not establish baseline measures or targets for its PMP indicators; 2) DI did not systematically collect data to populate all PMP indicators; and 3) quarterly progress reports do not refer to PMP indicators. The evaluation team reviewed the PMP indicators before fieldwork began, but DI provided partial PMP monitoring data only after fieldwork had been initiated. Without monitoring data that corresponds with each PMP indicator, it is not possible for the evaluation team to determine some of the outcomes—and even some of the outputs—of the DPR project. The evaluation team needed to adjust activities in Khulna, Rangpur, and Dhaka in response to travel restrictions associated with hartals. Sub-teams cancelled five of their scheduled FGDs in Khulna and Rangpur due to hartals called by Jamaat-e-Islami and BNP. However, the team was able to conduct at least one FGD with each of the major political parties in those sites. Hartals caused the evaluation team to cancel several KIIs and DI to cancel a youth fellows graduation ceremony and alumni conference in Dhaka that the evaluation team planned to observe. Despite these constraints, team members spoke with youth fellows in Chittagong, Khulna, and Rangpur who represented all major political parties as well as via phone with the president of the Alumni Association who is a member of Jamaat-e-Islami. Overall, the evaluation team does not consider that these logistical constraints have substantially weakened the generalizability of findings. Potential Bias The evaluation team is aware of several bias-related risks for data analysis:  First, recall bias is a common evaluation problem. On occasion, DPR project beneficiaries responded to questions posed by the evaluation team with answers that blended their experiences into a composite memory. Party members participated in more than one DPR activity or in similar activities conducted by other implementers and did not always distinguish them as separate activities. The fact that party events facilitated by DI were not branded as such was important for effective project implementation and party ownership, but it resulted in lack of awareness about DI’s role among some respondents.  Second, the evaluation team may have encountered response bias. For example, youth fellows may have provided positive remarks about the program—not only because their experiences were positive, but also because they would like to attend such trainings in the future. Political Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 8 party representatives consistently relayed that financial and technical assistance from donors is welcome, such as for the District Grassroots Representative Conferences (DGRCs), and they may have believed that negative evaluation findings could result in reduced assistance from DI or USAID.  Third, selection bias is an inherent risk when implementers help to facilitate contact with project beneficiaries. The evaluation team worked closely with DI staff based in the Dhaka Field Office and WYCs to organize FGDs and KIIs with DPR beneficiaries. There is a risk that DI may have selected the most active, responsive, or engaged beneficiaries. To mitigate the risk of selection bias, the evaluation team randomly identified individuals for participation in FGDs from the initial sample of party leaders provided by DI. In addition, the evaluation team conducted the mini-survey independently, without assistance from DI. The evaluation team triangulated data provided by DPR project beneficiaries with perspectives provided by external actors. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 9 FINDINGS AND CONCLUSIONS RELEVANCE Findings Current Development Circumstances and State of Affairs among Major Political Parties The 2014 elections had a dramatic impact on politics and government in Bangladesh. The main feature of this impact is that AL now effectively controls every political and government institution, including Parliament, security forces, the judiciary, and “independent” electoral, anti-corruption, and media commissions. Independent civil society and media are being muzzled or taken over. There has been strikingly little civic opposition to these moves; for example, a group of lawyers associated with BNP was the only entity to protest the law that allows Parliament to impeach any judge with a vote by one third of Members of Parliament (MPs). Opponents of AL face politically-motivated tax or other criminal charges; threats to indict and possibly imprison BNP leader Khaleda Zia (which would bar her from participating in elections) have escalated. AL shows no sign of relenting, and evaluators frequently heard FGD and KII participants say, in one political expert’s words, “this is the worst time for democracy since the founding of Bangladesh.” The post-election landscape has had different impacts for the major political parties. For BNP, the party’s failure to participate in the election and subsequent exclusion from power has led to sharp internal divisions, frustration among mid-level leaders, and disorganization. Evaluators observed one FGD with BNP party leaders devolve into a shouting match between participants representing two factions of the party. Several BNP leaders said that they faced or feared violence as a result of their political activities and that it is increasingly difficult to organize peaceful party activities. In Chittagong and Khulna divisions, BNP members cited the protection offered by an international organization as one of the benefits of working with DI. Bangladeshi experts interviewed by the evaluation team said that many BNP members disagreed with the decision to boycott the election. The evaluation team’s mini￾survey with party leaders found that BNP respondents supported the decision to boycott but were frustrated by the party’s failure to organize itself since then. In KIIs and FGDs, BNP members appeared uncomfortable when discussing the decision, but none was outright critical of the leadership’s decision to boycott. 3 While BNP has lost all national power, polling by DI and others makes it clear that the party maintains support from a sizable number of voters. BNP did well in local elections held in February and March 2014 until AL took control over the process and ensured that BNP-supported candidates would be eliminated. Evaluators found a number of signs that the ruling AL party is actively pursuing a strategy to consolidate its power at the local level—as well as signs that AL national leaders seek to consolidate their power within the party. At a party conference in September 2014, AL leader Sheikh Hasina announced the formation of seven teams that will “tour the country to inject pace in the activities of the organization. Through the council sessions they must have to pick new leadership and reorganize the party.” The teams will “ensure smooth holding of the council sessions,” and party leaders were instructed to “accelerate the holding of pending council sessions at the grassroots level.”4 All but one of the AL mini- 3 See Annex IX for mini-survey findings. Respondents may have been more open to criticize their national leaders during the mini-survey, given that questions were posed by National Team Members via phone on a one-on-one basis—as opposed to the in-person group setting entailed by FGDs. 4 Jakir, A. (2014, September 21). AL leaders for tough stance against BNP. The Independent [Dhaka]. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 10 survey respondents were aware of a deadline to hold council sessions by the end of 2014 and were planning to comply. At the same time, mini-survey findings reflect growing tensions within AL and resentment among local leaders toward control exerted by the national leadership. The 2014 parliamentary election also affected AL’s involvement with the DPR project. DI staff reported that even before the election, AL had been the party least eager to engage with DI. This appears to have become slightly more pronounced after the election. In at least one FGD, AL participants said they discussed requesting support from DI to conduct councils but decided that it was preferable to conduct these activities on their own. However, evaluators did not find any reluctance among women and youth from AL to participate in DPR activities. JP is technically the parliamentary opposition but is riven by the split between its founder Hussain Muhammad Ershad and his wife, with the main parliamentary group effectively acting as a subsidiary of AL. DPR worked extensively with JP, especially in the first year of the project. DI staff described JP as the “early adopter” of DPR. JP had “less to lose” than the bigger parties, fewer internal factions among leadership, and was less suspicious of international organizations. DPR activities with JP enabled the larger parties to see the benefits of working with DI. Uncertainty about JP’s future was evident in the FGDs, KIIs, and mini-survey. JP participants not surprisingly seem to be waiting for some direction and clarity from their leadership. DPR Program Logic and Project Objectives The DPR RFA articulates a notable break in USAID/Bangladesh’s approach to political party development. In contrast to past programs, DPR was to “focus development activities at the local level and primarily target the membership base and mid-level structure of beneficiary political parties.” This choice was based on the assumptions that 1) political parties are a key—if very imperfect—link between citizens and government and 2) national-level political leaders are resistant to change. Evaluators found that these assumptions continue to be valid even after the 2014 parliamentary election. At the local level, political parties will continue to have significant control over local government bodies—both elected and administrative. AL’s post-election monopoly on power will only increase that control. The extent to which political parties are representative, responsive, and accountable will strongly affect the ability of local governments to provide services and meet citizens’ basic needs. Even in the closing political space, parties remain key entry points for citizen engagement and critical to long-term progress toward better governance. DPR’s approach also represents an important bridge between “supply side” programs, which are intended to strengthen governance institutions, and “demand side” programs that strengthen citizen input and pressure on those institutions. The DPR Year 1 Work Plan articulates another important change in approach: “unlike prior efforts, DI will not require all the parties to sit together unless there is willingness and a clear reason to do so. DI’s relationship with the parties will be one of technical advisor and service provider, whereby DI will always maintain a crucial level of respect for the parties’ strategic interests.” Evaluators found that this approach had been an important asset for DPR, but DPR’s emphasis on working with individual parties means that DI worked more with some parties than others. Namely, DPR conducted more activities with JP than with the other two parties; and then between BNP and AL, DPR had more activities with BNP than AL. DI staff explained that this was in part because JP was initially more interested than the larger parties in DPR support. FGD and KII participants expressed interest in some cross-party work in the wake of elections, and the Youth Fellows Alumni Association offers an opportunity and vehicle to engage in that work at an appropriate level. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 11 Conclusions  DPR program objectives are still relevant to the current development circumstances in Bangladesh, particularly after the 2014 parliamentary election. Engaging political parties through projects like DPR is not only logical but also critical to the success of DG-related objectives as well as other priority development objectives. For example, protection of labor rights in the garment sector will require policy reforms and greater engagement by government institutions in regulation of the industry. These changes will be sustainable only if there are more democratic and responsive political parties representing citizen interests. The current state of affairs within and among political parties does not affect the analysis USAID offered in the DPR RFA that “because of the role a political party plays in a centralized winner-take-all system, government and state institutions can only be as effective and democratic as the party in power.  DPR’s approach to focus on grassroots leaders rather than national-level leaders was innovative, effective, and met the development needs. Engaging mid-level party structures in Bangladesh’s districts and sub-districts, or upazilas, continues to be a valid and constructive approach.  The 2014 parliamentary election shifted Bangladesh’s political landscape and requires some new approaches for work with political parties. o Despite its shrunken role in politics, JP continues to present an opportunity for DPR. On internal party democracy, ground has shifted but not to all negative. o BNP hopes to regain lost ground but is in tremendous internal disarray. Party leaders continue to perceive DPR as a useful resource. The extent to which BNP participates in the next election will shape its political future and the future of politics in Bangladesh. o AL is trying to build its local branches but is somewhat suspicious of DPR and USG efforts. Still, it makes sense to support democracy at the grassroots level within AL. o Opportunities for direct inter-party work (outside of the inter-party workshops and town hall meetings organized by the Youth Fellows Alumni Association) are limited and would likely be viewed with great suspicion, especially by AL. EFFECTIVENESS Achievement of Objectives Findings Objective 1: Enhance Grassroots Participation Developing Young Leaders Fellowship Program5 As of September 2014, a total of 101 participants (42 male, 59 female) completed the fellowship program. AL, BNP, and JP participated fully in the program. Participation by Jamaat-e-Islami members was more limited due to the ban on party activities. DPR modified the structure of the program after the first year to broaden the regional distribution of fellows. In each district, committee presidents from each party nominate a number of candidates; DPR then chooses the fellows from among those candidates on the basis of curriculum vitaes and interviews. 5 The Developing Young Leaders Fellowship Program is listed under Objective 1 in the DPR Year 1 Work Plan. Subsequently, the program is reported under Objective 3. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 12 Evaluators asked key informants whether the nomination process was subject to nepotism and favoritism. DI staff stated that involvement of parties in the recruitment process was necessary to ensure buy-in from the party presidents and full engagement of fellows in party work; the parties would not trust or work with someone who they did not know or did not trust. Fellows interviewed agreed that nepotism and favoritism were an issue and noted that party presidents sometimes nominated relatives or associates who were not qualified, but they stated that in all but a few cases the DPR screening and interviews assured that those ultimately awarded the fellowship were in fact qualified and enthusiastic. External actors who observed or interacted with fellows agreed with this assessment. Fellows interviewed by the evaluation team almost universally gave positive assessments of the program structure, quality of trainings, and support from DPR and the WYCs. All of the 12 fellows interviewed described improved knowledge and abilities in political activities as a result of participation in the program. Fellows frequently stated that they were not familiar with their own party constitutions or with the 2008 electoral law amendment on women’s participation in party decision-making structures and learned about both subjects from DPR. In Chittagong, a fellow said, "Before, I had no idea about how to use a press conference to make an advocacy campaign successful; DI has given me confidence to organize events and prepared me to face the media." A fellow in Jessore stated: “We learned how to raise our voices, how to organize people, how to bring people into politics.” Questions about negative aspects of the program elicited only comments about relatively insignificant aspects of some of the trainings and practical problems like getting party leaders to take ownership of their supervisory role over fellows. After intensive training in Dhaka, fellows complete three-month rotations with leaders in party committees and secretariats at various levels. Fellows enthusiastically described the activities they conducted in their districts as part of the fellowship and the impact of these activities on their communities. Fellows held conferences or workshops for party leaders and activists on women’s participation, the content of the party constitution, and related topics. They helped organize party events, including the DPR-sponsored DGRCs, and helped create or maintain district-level databases of party members and leaders. The most substantive projects were advocacy campaigns around a local issue like waste clean-up or health services; one fellow interviewed claimed he had conducted five such campaigns. Evaluators were not able to observe any of the projects, but fellows provided detailed descriptions of how they were implemented. The evaluation team found that the advocacy projects did appear to succeed in bringing more citizen voices into the political process. One fellow said she learned from the program that “everyone has a right to be heard—not only the leaders, but everyone.” She credited DPR for teaching her how to reach people, how to organize them, how to help them identify issues, and how to reach political leaders who can make changes. She also demonstrated to leaders that they could benefit from engagement with citizens, for example by holding press conferences to highlight their accomplishments on issues important to citizens. Another fellow engaged 467 citizens to get town officials to clean up a “stinky pile of trash” located near a hospital. He organized a press conference—with DPR funding—in which local officials were able to show they had done something useful for their community. Again, this demonstrated to citizens that their campaigns could be effective and underscored to officials that engaging in substantive projects was beneficial. The campaign was so successful that local officials allocated 10 Lakh Taka, or more than USD 12,000, to fund a biofuel plant that converts garbage into energy; the fellow and the citizens group are following up to make sure the officials keep this promise. DI staff and fellows interviewed by the evaluation team asserted that the fellowship program has a long￾term effect on the organization and operations of parties and politics in Bangladesh overall. For example, the fellowship program has worked closely with other DPR activities that support increased women's Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 13 participation in committees. A fellow reported that the AL committee in her district is on track to meet the requirement that 33 percent of its 71 members be women and noted that she was “the key person in making that happen.” Fellows have also developed personal relationships across party lines that help reduce enmity between parties. Fellows noted that DPR training in Dhaka and activities conducted through the Youth Fellows Alumni Association provide “a unique experience to work together.” At the same time, some key informants reported that concrete impacts within party operations have not yet materialized. In Rangpur, a fellow said that he had not seen a noticeable change; “transparency is still a big question for political parties.” Another said: “We very much want this program to continue for 20-25 years. This is how long it takes for young leaders to become the senior leaders, to go to the top where we will be able to make decisions.” The same fellow articulated a view that was expressed by others about the need to have a larger number of fellows in order to achieve a critical mass: “The number of participants should be increased in order to have a big impact… to be effective we need to be in more touch with senior politicians. But senior politicians don’t listen to us. The number is important here. If the number of ‘us’ is increased, only then can we make a difference.” Women’s Participation in Party Committees DPR approached the objective of increasing the number of women in political party committees through a combination of “top-down” and “bottom-up” interventions. In its top-down interventions, DPR met with party leaders at the district level—and to a lesser extent, national level—to educate them about the 2008 amendment to The Representation of the People Order (RPO) that requires all party committees to be composed of 33 percent women by 2020. At the district level, WYC staff used the DGRCs as a tool to commit district party presidents to raising the number of women on the committees, usually to 10 percent of members. The DGRCs were also a venue for training on the RPO requirements and the value of women party activists and how to engage them in the political process. At the same time, DPR launched the bottom-up Narir Joye Shobar Joy (NJSJ) “When Women Win, We All Win” campaign that convenes women activists to lobby for a more effective presence in politics, including on the committees. DPR succeeded in increasing the number of women on committees in target districts. According to monitoring data reported by DI in September 2014: “Through April 2014, DPR worked with 272 grassroots committees of AL, BNP, and JP. Before DPR initiatives, these committees altogether had 581 women members. After DPR activities, these committees altogether have 3,216 women members, a remarkable 454 percent increase of women membership including 2,635 women into decision-making bodies of the major political parties.” FGD and KII participants also reported that DPR had concentrated its efforts on increasing the number and participation of women in committees. Both male and female FGD participants reported that DPR activities—including formal and informal meetings, trainings, DGRCs, and events organized by youth fellows—significantly increased awareness and understanding of the RPO amendment. In most cases, key informants credited DPR for increasing the number of women on committees. For example, a BNP district party leader in Chittagong said that his party had committed at a DGRC to have 11 percent women (of 101 members) on its district committee, and the party met that commitment. One of the 11 women on the committee is a vice president and the other is a secretary. He said that, “In 20 or 40 years, there has never been a woman on this committee. Now for the first time there are women on the committee and two are in the leadership.” Also in Chittagong, a JP FGD participant commented that, before DPR, women had been satisfied with being party members and were not recognized for their Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 14 contributions to the party; since then, JP has formed new committees in which women constitute between 3 and 13 percent, with a number holding leadership positions. Moreover, both male and female participants in FGDs and KIIs reported an increase in the quality of women's participation on the committees. In Khulna, FGD participants stated that, “DI has only been active for a couple of years, but it has succeeded in giving the message to leaders that women should be involved in politics—really impressive for a short time.” They said that the number of women is increasing and that “the women who participated in DPR activities are more vocal than before. They are very ambitious and sincere in what they want to do.” In at least four FGDs and KIIs, male party members and senior leaders noted—without prompting—that DPR helped them better appreciate that engaging women in grassroots committees was not only a requirement but also had real political value and was vital to their future electoral and political prospects. One district president said that his party had always had one or two women (often relatives of party leaders) on sub-district committees. He now recognizes that having several competent women on each committee is more effective because they form a critical mass that attracts other women to the party. Evaluators noted that, in these cases, men were talking about sub-district level committees, not the more powerful district-level committees. Both male and female FGD and KII participants consistently made two comments related to DPR activities that promote women’s participation in party committees. First, they expressed interest in more training and other activities in which men and women collaborate in order to model expectations for work within their parties. Second, they recommended that activities reach women at the grassroots levels of the party: upazila, union, and ward. There may be some element in this of desire to restrict women from more powerful, higher-level committees. But evaluators also found that women’s engagement is more realistic at the sub-district level, that they could have more impact at that level, and that they need experience and practice at those levels before they can be effective at higher levels. Women’s Participation as Candidates As with the effort to increase the number of women on party committees, the NJSJ strategy to increase the number of women candidates nominated in the January 2014 parliamentary election had both “top￾down” and “bottom-up” elements. The top-down component involved educating male party leaders about the value of women candidates, informing them about qualified women who were available to run as candidates, and organizing events at which women had the opportunity to present themselves to party leaders. The bottom-up component involved trainings for potential women candidates. A key informant who was on track to be a BNP candidate until the boycott emphasized that DPR's strong political connections at the national and district level were just as important as the trainings she received. She also said that DI’s requirement that she conduct a signature campaign to demonstrate her base of support (she got more than 3,000 signatures from men and women) and events to showcase her organizational abilities helped her demonstrate her skills and political capital to party leaders. DI staff did not consider DPR activities in this area to be a success, and evaluators found a number of reasons why this effort did not succeed. First, most obviously, the election boycott meant that there was no candidate from BNP in many districts. Even before the boycott, women faced an uphill struggle to overcome the volume of money and political capital that male candidates deployed to secure places on the ballot. A number of women identified another obstacle: "A women must be on a committee in order to be nominated and run; otherwise, no one knows you… it is important that the [committee] know you and see that you are working. Otherwise, [committee members] just support who they always support." Indeed, this challenge provides rationale for DPR activities that advocate for enhancing women’s roles in committees. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 15 Objective 2: Improve Information Access and Utilization Research at the Central Level The evaluation team reviewed a wide range of tools developed by DPR as described in Table 4. Evaluators found some limited anecdotal evidence that DPR activities increased parties’ capacities to conduct and analyze political data and that parties were using that data and research (their own or DPR’s) to make strategic decisions. Table 4: DPR Surveys and Research Tools Public Opinion Surveys: As of August 2014, DPR conducted 19 surveys: nine surveys of public opinion; four surveys for political parties; one thematic survey on the ready-made garment (RMG) sector issues; two surveys to inform a civil society campaign on political violence; and three surveys of political party leaders in 2014 to generate data for the PMP. DPR has also been using polling to confirm that the NJSJ campaign message is effective and reaching the target audience. Media Monitor: DPR developed an automated tool for tracking media mentions of specific political personalities and issues across the media in Bangladesh, in order to give parties a better understanding of how they were being perceived and covered in the media. Electoral Results Database: DPR analyzed election results and census data to produce a “non-web relational database of election results for the 1991, 1996, and 2001 elections” that matched district-level election results with survey data. The purpose was to enable parties and other governance stakeholders to accurately map the constituencies and carefully craft party programs based on their level of support. The data is available at Computer Assisted Telephone Survey System (CATSS): DPR developed “a fully automated computer￾based telephone interview tool which enables researchers to create customized random number generation, instrument design, interview and data processing. With CATSS, parties and other potential researchers will be able to generate an adequate sampling system to implement mobile-phone-based surveys in Bangladesh.” The use of CATSS considerably reduces time and costs of fieldwork, data processing, and analysis. Compared to face-to-face surveys, CATSS showed small differences in results. Causality and attribution are very difficult to establish in this context, but there are some indications that after DI presented BNP with poll findings showing that BNP-organized hartals were having a very negative effect on BNP’s popularity, those hartals stopped for several months. Representatives of two other international organizations who closely follow election violence in Bangladesh said they believe that DPR polls and interaction with BNP helped to reduce the number of hartals. BNP continues to engage with DPR, though according to DI staff, BNP is skittish about using polls: the party agrees to some questions and then pulls back because it is afraid that the questions asked will leak to the press and give opportunity for criticism. DI believes that BNP’s decision to boycott the 2014 elections will be an incentive to make greater use of surveys and research, a theory echoed by other international observers.6 This was (anecdotally) confirmed in some BNP FGDs. In Khulna, a participant stated that research is important because “we have been wondering what has happened in the elections and with voters. At the present time, we are not able to formulate our policy based on research.” A BNP leader who worked with DI on research generally gave DPR a positive evaluation, saying that DI held extensive meetings with a wide range of individuals within the party, solicited opinions about what questions to ask in the surveys and included party input, and was “doing very good work.” He said that some results of the research DPR conducted for BNP were “a real eye-opener” but said he could not talk about the specifics of these “sensitive issues.” He admitted that the research cell within BNP had not been very effective before the election, but that there is now recognition within the party that research is needed 6 Some DI staff commented that the election could make BNP more reluctant to use polling data, since it will only reveal the extent of the party’s mistake and loss of popularity. In fact, different factions within BNP could hold these competing views. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 16 and there is a three-year plan to build up the research team within the party. He expressed hope that DPR would provide assistance in building that research capacity within BNP. Of the major parties, AL has by far the most active research structure.7 The Center for Research and Information (CRI), a non-governmental organization (NGO) closely identified with AL, serves as the party’s de facto think tank and research wing. DI staff stated that there are 30 people employed at CRI.8 CRI’s very professional-looking website and Facebook page feature an article entitled “DI & CRI discuss public opinion research”9 that describes two “shared learning sessions” in which DI and CRI “exchanged their experiences of political research” which “in recent times has become an important fixture of political activism in Bangladesh.” DI asserted that the combination of its polling and recommendations that emerged from local conferences helped convince AL to move away from the traditional reliance on personalities and the independence movement in its party platforms and election manifestos and instead highlight specific issues facing voters. According to DI, AL’s election television advertising—the first time parties had used television advertising in a campaign—was “straight from DPR polls.” The evaluation team was not able to corroborate that statement. DI stated that JP used DPR’s electoral data in the party’s negotiations with AL over allocation of parliamentary seats to demonstrate the depth of JP’s support in certain regions and that this pushed AL to concede more parliamentary seats to JP. Evaluators were not able to confirm this. DI also asserted that the combination of its polling and the results of local conferences helped convince JP to include more substantive issues facing voters in its party platforms and election manifestos. A JP leader who had worked with DPR had a positive evaluation of DI’s work with the party. He described that, beginning in early 2013, DI actively responded to the party’s needs and suggestions. He stated that as the party learned more about polling methodology, it was able to raise questions about technical aspects of the surveys, such as the impact of the order of questions on results. By mid-2013, JP was “very happy” with the work done by DPR. He also commented that there has been a gradual increase in the use of political surveys and research. “Five or ten years ago no one was doing that. They would have laughed at the idea that it was useful…. Now we are more open to the idea.” He and others emphasized the slow generational change within the parties: the older generation of leaders had little use or interest in surveys and data. His younger generation, which has spent time in the West and observed how data is used, is much more open to research and will use it when they come into leadership positions. Evaluators did not find evidence that these examples are signs of a noticeable, systematic increase in parties’ use of polling to make decisions10 11 or that such a change would happen in the near future. For example, while CATSS is an innovative and cost-effective data collection tool that would in principle allow political parties to do rapid analysis of public opinion with respect to current and evolving issues, 7 Several sources suggested that Jamaat-e-Islami conducts and uses some kind of research; evaluators were not able to confirm this. 8 DPR’s point person at CRI did not respond to the evaluation team’s request for an interview. 9 DI & CRI discuss public opinion research | Centre for Research and Information. (n.d.). Retrieved from http://cri.org.bd/2014/05/15/di-and-cri￾a-mutual-journey-towards-political-research/ 10 Evaluators are confident in this finding but also note there is a significant methodological issue: political leaders are always going to be loath to admit that they relied on someone other than their own judgment for their decisions, especially on an American organization. Along the same lines, DPR’s PMP notes that “it is difficult to establish when internal policy debates are driven by research. It could be possible to use a media survey as a proxy since direct measurement is not practical.” 11 DPR’s polling of workers in the RMG sector showed that the majority of those workers were opposed to any political party activity in their organizations and activism. When presented to the parties, this finding apparently restrained the parties—especially AL—from trying to engage with (and probably try to control) the labor movement. A representative of an international NGO that works closely with labor unions in Bangladesh who was not involved in the design or implementation of the poll could not speak to the motivations of the parties but stated that the survey’s findings were “very useful” to them. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 17 the party representatives interviewed made no mention of CATSS. Nor did DPR management explain their strategy for marketing this tool to the parties. Evaluators were not able to confirm what kind of “research” AL is doing, who has access to the findings, or how the data is being used. CRI’s extensive website describes numerous activities but makes no reference to research other than the activity with DPR. An AL leader who is familiar with the party’s research was dismissive of DPR’s work. In BNP, there appears to be a disjointed, on-again-off-again approach to research. DI staff described BNP as unable to commit the resources needed for a good polling operation. One senior BNP leader suggested that Khaleda Zia’s son Tarique commissioned extensive political research, but the evaluation team was unable to confirm this. Research does not appear to have played any role for JP and is unlikely to do so in the party’s chaotic state. A key informant from JP said of the research provided by DPR, “we weren’t in a position to use this information at all… We may know what people want and need, but we are not in any position to meet those needs.” While parties were lukewarm about the value of DPR’s surveys, donors and international NGOs were eager consumers of the data. DI conducted numerous presentations and briefings for USAID, the U.S. Embassy and the Ambassador, and international NGOs. At the Embassy’s request, the Ambassador’s name was included in DPR’s media monitoring. In evaluators’ experience, some degree of information exchange between USAID programs and presentations for international agencies is very common and has benefits for both sides. Subjectively, evaluators found that the extent of that interaction in this case exceeded that norm. Research at the Local Level An emphasis on research work at the local level is not evident in DPR work plans or activities, despite the RFA requirement “to develop appropriate and sustainable research capabilities at the local party branch level.” DPR Year 1 and Year 2 Work Plans do not discuss this requirement. DPR’s September 2014 PMP update reflects an absence of work at the local level and offers little data or explanation. Under Indicator 20, “Number of local party branches receiving DPR assistance to articulate platform and policy agendas effectively,” DPR responded that “This type of assistance would have constituted campaign assistance and as such would not have been possible or appropriate to deliver.” Under Indicator 21, “Cumulative increase in total number of political party units and offices as well as local party branches that use DPR supported public opinion polling and training and designing voter outreach strategies” DPR responded, “As written this metric has not been practical to measure or track.” However, FGDs demonstrated an interest in research at the local level. Party leaders in all FGDs agreed that “political research is an important tool to win elections.” Participants were also able to articulate reasons why data is important to shape platforms and win elections and noted that it was missing. In Khulna, BNP FGD participants complained that “there is a position in each committee titled as Research Coordinator,” but “it is not working properly and there is no real research work.” Other FGDs echoed this assessment. Research was a component of the youth fellowship program, and a female fellow in Chittagong used the DI training to conduct an informal survey of the needs and political opinions of indigenous people living around Chittagong; the issues they raised became part of the party’s election platform. Survey Implementation and Methodological Issues Evaluators noted a number of technical issues with DI’s survey methodology and implementation, which may be limiting its value for both parties and donors. A review of the four national public opinion surveys shows that DPR focused on a broad range of variables: voting preferences and intentions; Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 18 patterns of switch voting; perceptions of whether the country is on the right/wrong track; perceptions of improvement/worsening of individuals’ situations; assessment of performance of incumbent and previous governments; identification of policy issues; media access and use; and a range of topical political issues, including preferences for a caretaker government mechanism to oversee elections, perceived legitimacy of the 2014 elections, and preferences for new elections. The public surveys differ in emphasis, with respect to both topical questions around emerging political issues and questions about the “core issues.” There is considerable variation in how public survey results are presented, making it difficult to understand trends. DI was not consistent in framing some survey questions: questions about preferences for women leaders in political parties changed between surveys; several surveys refer to “confidence” in institutions, and another referred to “favorability.” Statistically-Based Observation (SBO) 12 In anticipation of parliamentary elections, USAID provided DI with a follow-on extension to conduct an SBO in all 300 electoral constituencies in Bangladesh. The extension directs DPR to develop a consortium of two to five research organizations, train NGOs in SBO procedures, and provide resources to political parties to conduct their own SBOs in the future. The SBO would use a short message service (SMS) monitoring and reporting transmission system developed by DPR, along with an Internet-based relational database system and an Internet-based reporting system to detect, report, and deter electoral fraud. Unlike other election monitoring that relied on paper reporting, Internet, and faxes, the monitoring results would be available in real time and through widely available SMS technology. To implement the SBO, DPR formed a consortium of three research firms and cooperated with the Election Working Group (EWG), an existing group composed of 29 Bangladeshi NGOs organized by The Asia Foundation (TAF). DPR recruited and trained more than 600 district and constituency coordinators. TAF expressed satisfaction with DPR’s role and support to trainings on the relatively complex body of technology required to implement the SBO. TAF noted, too, that some local NGOs in the EWG were initially reluctant to participate because the SBO would be cheaper than traditional monitoring methods, which would reduce the size of their grants to conduct the monitoring. Following USAID and USG guidance, DPR did not implement the SBO for the January 2014 parliamentary election. However, DPR did deploy the SBO system in 10 upazila elections in spring 2014. DPR’s 2014 Q2 progress report asserts that the SBO system enabled “EWG to produce more objective observation and more timely reporting of the observation” and that “the adoption of the SBO approach has transformed the group into more methodical and has broadened the adoption of scientific approach in the political process in Bangladesh.” To support the SBO, DPR also conducted 17 party polling agent trainings with a total of 2,716 participants throughout Bangladesh. A BNP leader interviewed reported that these trainings were an effective response to the needs communicated by the party to DPR. 12 In May 2013, USAID/Bangladesh issued a contract modification and increased the award to enable DI to conduct an SBO of the January 2014 parliamentary election. This activity was not included in the DPR Results Framework, and no indicators were developed to measure its effectiveness. The extension included funding for a new “Parties in Parliament” program to improve parliamentarians’ accountability to citizens and party cohesion in Parliament. The “Parties in Parliament” activity was dropped after the elections. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 19 Objective 3: Improve the Environment for Responsive Politics13 Party District Conference Series and Councils The core of DPR’s efforts under this objective is assistance to parties in conducting DGRCs. The format of the events is based on parliamentary procedure and starts with a plenary session that includes speeches by local leaders or MPs and the chairman of the conference, who is nominated by the district president. Next, attendees form sub-committees to work on the process of electing party committee members and national conference delegates; developing information and communications technology (ICT) between the grassroots and party leaders; selecting women committee members to meet the 33 percent threshold as mandated by law; committing to a roadmap for party councils at all levels; and reviewing the candidate selection criteria for national and sub-national elections. By June 2013, DI and three major parties organized more than 200 DGRCs in all seven divisions with a total participation of more than 40,000 elected grassroots committee presidents and general secretaries. Evaluators found generally positive impressions of the DGRCs during KIIs and FGDs with party leaders who participated in them. FGDs reported a “mostly positive” impression of the events (five out of seven with recorded answers); four of seven FGDs said there were “some” positive changes as a result of DGRCs, and one FGD said those changes were significant. For example, FGD participants in Chittagong said that, as a result of the DGRCs, “many developments have taken place. Before, the party was almost dormant and there were no activities… with the intervention of DI, party activities have become more vibrant. Our party activities have increased; this is due to the intervention of DI.” However, the evaluation team notes the potential of recall bias and confusion among party leaders about what was supported by DI versus organized by the party itself—confusion that may also reflect successful ownership and uptake of the events. One issue frequently raised in FGDs and KIIs was the parties’ failure to hold the councils in which members and leaders are formally elected. A FGD participant in Rangpur said, “Our leaders do not practice democracy. Sometimes we try to plan councils, and the leader at the higher level stops our activities midstream because he wants to remain in power by protecting his cronies in the lower-level committees.” Evaluators noted an interesting dynamic on this issue during a FGD with BNP leaders (including the presidents) from two districts. In one district, DPR had not been active; in the other, DPR had a longer history of relationships and activities. FGD participants from the district where DPR was active said that they held their council immediately after the DGRC. These participants were informed about the RPO requirements and the party constitution, and they were vocal about the lack of communication from the national party and the control exerted by the national party over their activities. By contrast, FGD participants from the district with less DPR activity appeared to know little about the RPO and did not have much to say about the participation of women and youth in their committees. They described their main challenge as “bringing new faces into the committee… in about 5-10 years.” They were not concerned about the lack of councils to elect new members and leaders. Recognizing that the process of holding councils is critical to party development in Bangladesh, the evaluation team designed and conducted a mini-survey that reached 30 party leaders from across the three parties. The survey confirmed that parties had not followed requirements to hold councils and that the parties had very different approaches to future councils. See Table 5 for a summary of results and Annex IX for detailed results. 13 The DPR RFA explains that “the thrust of interventions under this objective is to capitalize on progress made through Objectives 2 and 3 to replicate, expand and consolidate gains at a national level. A window of opportunity created under Result 3.2 will make it possible to support specific reform initiatives adopted by political parties at a national level.” Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 20 Table 5: Overview of Mini-Survey Results Past District Councils Future District Councils AL Of nine respondents: one said the last council was in 1996, four said before 2005, and four said between 2011 and 2013. All said councils had been determined exclusively or to some extent “by selection” or “by consensus,” with only four saying some had been “by election.” AL is preparing for the next district-level councils sooner than its rivals. Nine of 10 interviewed said they would have councils before December 2014; one said 2016. BNP Four of nine respondents reported that district councils were held in 2009; five reported that councils had been held since 2009. No BNP respondents knew when their next councils would take place. Several said they were too busy organizing anti-government agitation to be holding councils. (There may be more activity at the sub-district level. A BNP district president said that 14 sub-district councils are planned between September and December 2014.) JP The most recent district councils were held in 2013, though they were very insignificant in number. One respondent said there had been no council in his district committee for 17 years; in 2013, a new committee was declared by the central committee with no council or even a discussion among the district leaders. JP is planning to hold district councils by the end of 2014. However, respondents did not make any reference to any directives from the central level. DI stated repeatedly that DPR is shifting focus from the DGRCs to councils. DPR’s 2014 Q3 progress report says, “One of the most important outcomes of DI’s Party Conferencing Series has been the demand generated within the party grassroots for more regular and meaningful internal party elections…. One of the primary purposes of the work with more than 40,000 grassroots leaders through the previous conferences was to train them on internal democracy and inculcate a culture of accountability.” Evaluators saw some signs in FGDs and KIIs that DPR activities were increasing pressure to hold councils, but there are also other fundamental political forces driving this pressure. A frequent comment in FGDs was that DGRCs should have more grassroots participation. In Chittagong, BNP participants complained that “only their [upazila and ward] presidents and secretaries could attend the conferences” and said there would not be any change in party operations and communication without broader participation. According to the Year 4 Work Plan, DPR’s emphasis will “reach downward, into the upazila, union, ward, and village level as preparation for national party conferences. In the initial phase of the conferencing, DI will facilitate more than 200 events at all levels.” Web Portal and Technological Support DPR’s work to develop party web portals is intended, according to the Year 4 Work Plan, to “enable parties to overcome space and communications barriers, effectively collaborate, and have more inclusive and democratic intra-party processes.” DI staff noted that Chairman Ershad of JP once used the SMS bulk messaging service embedded in the web portal, and told evaluators that AL was using the portal to communicate party events and activities. Evaluators found no independent corroboration, nor was there further mention in the DPR progress reports. DPR worked extensively with the parties to develop databases of members and leaders. DI uses SMS technology to register participants in the DGRCs and populate the database with this information. Several youth fellows mentioned that part of their work was to complete and clean up their party’s district database. Evaluators relied on these databases to randomly select FGD participants and to contact party leaders for the mini-survey. Evaluators found that the quality of the list was spotty with some incorrect names and numbers, but the team also recognizes Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 21 that knowing who is in your party and being able to contact them is fundamental to political organizing—and that even a spotty list is a substantial improvement over no list at all. Conclusions Objective 1: Enhance Grassroots Participation  Overall, DPR met the objectives and IRs for enhancing grassroots participation by women and youth in politics. In terms of outputs, the increase in women’s participation in political parties was more significant than youth participation because the latter is measured only by the number of fellowship program alumni.  Youth increased their skills as political organizers and enhanced their political prominence and influence as a result of their participation in DPR activities. Youth fellows affected party organization, women’s participation, and other aspects of political culture—especially citizen engagement in advocacy campaigns at the local level.  DPR interventions increased the quality and quantity of women's participation in political party committees. Men and women in political parties are more aware of the legal requirements—and practical incentives—for women’s participation in political life in Bangladesh. This recognition of self-interest among male party leaders is ultimately more sustainable than training or legal requirements that promote women’s participation.  DPR was not able to significantly enhance the number of women candidates in the January 2014 parliamentary election, but this was largely the result of factors beyond DPR’s control such as BNP’s election boycott. Objective 2: Improve Information Access and Utilization  DPR achieved only some of the desired results toward improving information access and use. DI developed a number of innovative tools for improving the quality of data about politics in Bangladesh, and these tools may have some effect in coming years. DPR may have had some positive effect on the capacity of some individuals within parties to understand the value of research and data. However, there are significant obstacles to uptake of this data and institutionalization of these skills. Failure to achieve results under this objective was influenced by the underlying political context.  DPR met the objectives of the SBO to the extent possible, given the unforeseeable decision not to monitor the election as planned. However, transferring the capacity to conduct an SBO to political parties or civil society is ambitious and unlikely to be achieved without additional effort and resources.  DPR’s most significant shortfall under this objective is the failure to focus as USAID intended on the local party level structures, to help them develop greater awareness of how research can help to shape more substantive and responsive party programs. Objective 3: Improve the Environment for Responsive Politics  DGRCs had some success in increasing party transparency and responsiveness at the district and sub-district level, for example in increasing awareness of the RPO amendment, introducing some new faces and voices of women and youth into party organizations, increasing communication between districts and sub-district committees, and encouraging some party organizations to hold election councils for committees. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 22  Progress at the district and sub-district levels has not translated into change at the national level. Evaluators concluded that DPR’s work to develop party web portals is unlikely to yield significant concrete results in the near future but holds some promise if there is a complementary evolution in the political party culture. Management Structure Findings DI did not provide the evaluation team with a formal organizational chart. Instead, the Deputy Chief of Party (DCOP) provided details on DPR’s management structure and responsibilities, which involve four program management components: 1) grassroots outreach with political parties to promote women and youth involvement, led by the DCOP and supported by a Bangladeshi Senior Program Manager (SPM); 2) support for political party events and activities in the regions, led by a Bangladeshi SPM; 3) fellows program led by an expatriate SPM; and 4) a research component led by an expatriate Director of Research. The DCOP has a large portfolio, responsible for the grassroots program supported by the SPM and three staff in Dhaka plus more than 20 full-time WYC staff outside Dhaka. The DCOP also supervises an administration and finance office, supported by a Bangladeshi Director of Operations. The relatively small youth fellowship program—involving recruitment, placement, and support to fellows—is managed by a full-time expatriate SPM supported by three Bangladeshi staff. The Director of Research supervises a Bangladeshi SPM and four full-time staff, as well as manages temporary staff required by the CATSS and SBO activities. USAID and DFID staff expressed satisfaction with DPR management. Both highlighted the value of the field office’s ability to operate independently of DI headquarters. One donor program manager said, “DI has been a very good partner… very responsive… one of the best I’ve worked with.” By the end of Year 1, DI established WYCs in the capital cities of each of the seven administrative divisions, giving the DPR project a broad reach. Each WYC is composed of a Coordinator, Deputy Coordinator, and Office Assistant. While the 2011 Q4 progress report describes WYCs as “a space for youth and women to discuss their participation in politics and governance,” the evaluation team found that, in practical terms, the WYCs function as DPR’s regional offices to support the entire range of DPR activities. Ongoing activities are planned and budgeted in Dhaka, then implemented by the WYCs. There are quarterly meetings with all WYC Coordinators and Deputies for program planning and capacity development; the Chief of Party (COP), DCOP, and SPMs visit the WYCs regularly to support planning and execution. WYC staff interviewed by the evaluation team were motivated and articulate about DPR objectives. KII and FGD participants were positive in their assessment of the WYCs. All but one FGD expressed the opinion that WYC staff were respectful, knowledgeable, and met their commitments.14 Flexibility: Evaluators found a number of examples of DPR flexibility in response to changes in context and need. Most fundamentally, DPR shifted its priorities to include the SBO for the 2014 elections in response to USAID interest in election monitoring and then cancelled that work when the situation changed. At the implementation level, DPR adapted several activities in response to lessons learned. For example, the structure of the youth fellowship was changed to reflect lessons learned in the first class of the program. 14 Evaluators noted the relatively low turnover and long tenures among both expatriate and Bangladeshi staff. The COP and Director of Research who recently departed were with the DPR project since its inception. A third expatriate has been with the project for two years. Local staff had similarly long tenures; most key staff and WYC Coordinators had been with the project since inception. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 23 Cost Control and Savings: 15 DPR reported that WYC operations are relatively inexpensive with rents, salaries, and other operating expenses at approximately USD 2,000-3000 per month. DPR uses a web-based system to manage those costs; WYC Coordinators request approval for expenditures from relevant SPMs in Dhaka. Requests are then approved by the DCOP, who also reviews the reconciliation after the expenditure. Activities are included in program budgets and not part of the WYC budgets. DI staff were able to give the evaluation team examples of cost savings. Many implementing partners traditionally employ international consultants on short-term assignments to work with local survey organizations. 16 In contrast, DI staff developed and implemented the software used in CATSS and the SBO. The CATSS telephone operators are DPR employees, and DI trained them to implement the surveys and polling. Evaluators did not compare the costs associated with these different approaches but, based on previous experience, suspect that the DPR approach may be less expensive. Yet, on the surface, this approach may contribute less to local capacity. DI has four full-time expatriate staff in Bangladesh. This is the largest number of any of the USAID￾funded DG programs evaluated by BDGPE and must be a significant share of overall expenses. The large number of expatriates was clearly necessary at early stages in the design of programs, building of relationships, and establishment of approaches, activities, procedures, and reporting; those functions should be less central to the project after three years of operations. It is clear that the expatriate staff brought a higher degree of technical knowledge in areas like research and political organization than was available in Bangladesh, that they brought deep personal experience with politics in the U.S. and other countries, and that they were less likely than many Bangladeshis to be perceived as biased in favor of one party. Evaluators did not ask KII and FGD participants specifically about the expatriate staff but repeatedly heard that no local organization would be able to do the work that DPR has done. Conclusions  DPR’s management structure is appropriate for its project objectives and activities, which involve thousands of participants across all seven divisions and half of Bangladesh’s districts.  DPR met the target of establishing and effectively managing WYCs in all divisions. The WYC management structure is functional and efficient, delivers a large volume of activities, and enables DPR to successfully target party units at the grassroots level.  DPR appears to have an effective structure for managing and controlling costs, and several innovative programs represent significant cost savings.  There is reason to examine the necessity and cost-effectiveness of maintaining four full-time expatriate staff on the program for its duration. Engagement of Women and Youth Findings DI has pursued a holistic strategy for promoting women’s political participation by integrating the objective throughout multiple aspects of the DPR project. All DI and WYC staff interviewed by the evaluation team spoke substantively about DPR goals and activities related to women’s engagement in 15 Evaluators did not conduct an audit of DPR. These findings and conclusions are based on interviews and evaluator observations. 16 See https://my.vanderbilt.edu/seligson/files/2013/12/Improving-the-Quality-of-Survey-Research-in-PS-March-2005.pdf for a detailed discussion of the problems associated with this approach. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 24 politics. In addition, many project beneficiaries—both men and women—who participated in KIIs and FGDs identified advocacy for women’s political advancement as the most visible component of the DPR project. A majority of male FGD participants were familiar with the NJSJ campaign, and many spoke enthusiastically about positive attitudinal shifts among party leaders about the role of women in politics. The evaluation team found that DPR engaged both men and women to champion increased opportunities for women within political parties. Through document review and KIIs, the evaluation team found that while the fellowship program has engaged a relatively small number of youth, participants significantly increased their skills as well as received promotions and recognition from party leaders. Fellows credited their enhanced standing in the community, increased political capital, and career development to the fellowship program. One fellow advanced from party member status to serve as both assistant joint secretary of women’s affairs in the party’s main committee in Chittagong and assistant joint secretary of the student wing at the central committee in Dhaka. Another fellow advanced from ward chairperson to thana secretary and is now executive committee member of the party’s city corporation branch. Both have aspirations to campaign in local elections. More than 50 percent of fellowship program graduates are women, and some overcame socio-cultural obstacles in order to participate. The evaluation team found that some families object to young women traveling independently to Dhaka for DI training or to residential offices of male party leaders for evening meetings. Some female fellows spoke to evaluators about the threat of sexual harassment by male party leaders and noted that DI provides training to prepare women to deal with unwanted advances. In the face of these challenges, the fellowship program increased women’s confidence and earned them respect from party leaders. One female fellow said that, “before the fellowship, I was shy to talk to the party leaders. But now I see that they are people like me and that they are supposed to listen to me. I see that I have a right to approach them. I am a changed person now.” Conclusions  DPR engaged women and youth in target areas, although the outputs (and impacts) appear to be more significant with respect to women. While the results of the fellowship program are very positive, it has only reached 101 graduates. DI’s activities with female party members are more expansive and the impacts reach farther.  DPR activities successfully engaged male party leaders and members to promote and facilitate women’s involvement in political parties at the local level. Both the NJSJ campaign and youth fellows supported by DPR promoted women as capable contributors and valuable assets to political parties and provided opportunities for men and women to collaborate within parties. Monitoring, Evaluation, and Coordination Findings Monitoring and Evaluation DI was required to design a PMP that according to the RFA “should be aligned with Foreign Assistance Framework (F) standard indicators and include performance measures that can be used to capture the overall expected results… The PMP will further serve as a management tool to assess and report progress toward achieving the targets.” Likewise, there is a requirement for DI to report on PMP indicators in its quarterly Performance Monitoring Reports, using the baseline measures and targets. To track DPR’s progress in engaging women and youth, disaggregated data on women and youth was intended to be collected whenever possible. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 25 Evaluators found that DPR’s PMP design and reporting is missing essential elements outlined in USAID’s technical guidance on PMP design and management.17 The PMP approved by USAID contains 46 distinct indicators—more than can be reasonably measured. The indicators do not differentiate between outputs (such as number of young party leaders trained), outcomes (such as increase in party member and public acceptance of youth and women in politics), and impacts (such as the increase in perceived influence of women in political parties). Furthermore, the PMP contains baseline measures or targets for only six of the 46 indicators. DPR conducted a number of activities designed to assess partner capacity and establish baselines, but these are not incorporated into any PMP reports. Progress reports and work plans make occasional but unsystematic reference to PMP indicators and data, for example on women’s participation in party committees. DPR’s Q2 2012 and Q3 2012 progress reports refer to a baseline survey of 600 political party leaders contracted with Nielsen, but DI staff in Dhaka were unable to explain what happened to this survey and results were not made available to the evaluation team. Some DPR public opinion and political party surveys include questions related to the PMP but not in a systematic way. For example, the first public opinion survey, conducted in June 2012, contains questions that respond to three PMP indicators (Indicators 10, 18, and one F Indicator). However, it does not include a question that responds to PMP Indicator 28 on perception of party responsiveness. Subsequent DPR public opinion surveys do not contain these same questions, or they frame them in a different manner, making comparison impossible. For example, one survey refers to “confidence in institutions,” while another refers to “responsiveness” of institutions. The CATSS survey conducted in July 2014, soon after the evaluation team’s request for PMP data, asked a number of questions related to the PMP, but there is no baseline data against which to track DPR’s progress toward targets on PMP Indicators 10, 13, 17, 18, 25, 32, and 33—the most important outcome indicators in the PMP. DFID staff expressed a different perspective, describing DI’s M&E as “really, really well done.” DFID appreciated the integration of M&E systems and project activities (in particular, through the research activities and CATSS), which they described as “a significant step forward that other [programs in Bangladesh] don’t have and haven’t done, not only in the DG portfolio but in other sectors as well.” A global, cross-sector evaluation of DFID programs’ integration of information technology and M&E highlighted DPR’s evaluation methods as innovative, cost-effective, and practical. Perhaps DFID’s comments reflect the copious amount of survey data that DPR collects. While DI has not reported specifically on the PMP data as required by the cooperative agreement, the evaluation team found that DPR has in fact collected a significant amount of data related to the project and political context through its surveys, CATSS, party database, analysis of committee membership, and other tools. In principle, these data collection efforts could have been used in relation to the PMP to produce valuable analysis of program impact. In other words, the shortcoming is not in data collection capacity. DPR collects and analyzes a substantial quantity of data that could be useful to USAID and DI staff. The shortcoming is in the planning, design, analysis, and presentation of data vis-à-vis DI’s PMP requirements. Collaboration The evaluation team found that cooperation between DPR and other USAID programs was very limited. 18 The one exception is DPR’s engagement with the International Foundation for Electoral 17 Performance Monitoring & Evaluation Tips - Preparing a Performance Management Plan (7). (2010). 18 For Objective 1 activities with youth, the RFA directs DPR to “feed into and benefit from” activities under other USAID/Bangladesh programs. Under Objective 2, the RFA directs DPR to use research resources and mechanisms developed by a variety of USAID programs, including Promoting Democratic Institutions and Practices (PRODIP), Promoting Governance Accountability, Transparency, and Integrity (PROGATI), and “other initiatives in health, education, economic growth, and food and disaster assistance.” Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 26 Systems (IFES) to provide training and other support to the Youth Fellows Alumni Association. This cooperation was successful and praised by both organizations. DPR provided SBO technology to the EWG, organized by TAF with USAID and other funding, and trained observers on its use before the election monitoring was cancelled. However, according to TAF personnel, this coordination was largely accidental: TAF heard from BEC sources that DI was working on some kind of parallel vote tabulation and approached DI to see what it was doing.19 Evaluators heard a similar story about DPR’s survey on the RMG sector; the USAID program to support labor rights in Bangladesh heard about the survey after it was completed. DI staff had occasional interaction with their counterparts at the National Democratic Institute (NDI) and the International Republican Institute (IRI), but there was no programmatic cooperation. This was true even in areas where the organizations conducted similar activities, with similar funding from USAID and DFID. For example, DI and TAF through the EWG, as well as NDI and IRI separately, trained and deployed a small number of election monitors to observe local elections. NDI and IRI staff said that they were not informed about DPR’s objectives, activities, and in many cases the results of polls and research conducted by DPR—and that this lack of knowledge undermined the possibility of cooperation. Outside of the DG sector, DI could not point to coordination with other USAID program implementers. Conclusions  DI’s M&E systems have not been effectively used to track, monitor, and report results attributable to DPR activities. Data collection itself is relatively robust, but data is presented in an ad hoc manner and does not appear consistently in progress reports. Other than data on women’s participation, DPR is unable to present quantitative data on its long-term impacts on the improvement of internal governance of political parties.  DI has not coordinated with other USG-funded programs to achieve DPR objectives or to contribute to other USAID/Bangladesh objectives. There are shortcomings and opportunities for improvement in this area for both DPR and USAID. SUSTAINABILITY Findings DI describes its model for sustainability and broader impact as “buzz and momentum.” For example, DGRCs raise expectations that the advancement of women’s roles in committees will be pursued across all upazilas in a district, not just in upazilas where DPR supports councils. The ability of DPR direct project beneficiaries to mix with other party members at DGRCs stimulates demand, and DI believes that this “buzz and momentum” will result in increased numbers of women in committees—even in upazilas where DPR is not working directly. While the concept is a good one, the evaluation team did not find evidence of a more structured approach, i.e., as a specific intervention to extend or deepen DPR’s impact. DI might consider, for example, a formal mechanism to ensure that party officials from “non-treatment” districts or upazilas observe DPR-sponsored councils to create or reinforce broader demand. As of now, this important and relevant concept lacks clearly defined goals, implementing mechanisms, and timelines. The upcoming party councils will be an important milestone in gauging the sustainability of any changes in parties’ internal governance practices. The evaluation team’s mini-survey results, however, are not encouraging in this respect. Established practices of “selection” and “consensus” continue to be prevalent in many council processes to date. While DPR’s emphasis on measuring changes in women’s 19 The Justification for Exception to Competition (JEC) memorandum prepared by USAID does not mention TAF or other programs. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 27 participation in committees is understandable, it is not clear whether DPR monitoring of councils will be able to distinguish between processes of “selection” and “election” of party committees. In its first three work plans, DPR did not include a strategy for sustainability of other project interventions. The Year 4 Work Plan emphasizes an “aggressive implementation schedule” and includes activities aimed at ensuring sustainability of some important achievements. This reflection on sustainability may be too late, given the lead time necessary for implementing the strategies below. DPR proposes the following sustainability strategies: 1. Encourage parties “to use the WYCs as on-demand resource centers.” However, both DI and USAID staff expressed doubts about the sustainability of the WYCs. In fact, KIIs with WYC staff indicate that they function more as regional offices for DPR than as resource centers for various political parties. The parties have no motivation to finance the WYCs, and there is no donor interest in sustaining them beyond the project period. 2. Establish “NJSJ as a sustainable and nonpartisan hub for women’s political leadership development.” DPR notes that it will establish a NJSJ advisory board of female political leaders; develop partnerships with universities, civic, and women’s organizations; and explore opportunities for sub-grant funding to partners. However, the Year 4 Work Plan does not go into further detail. 3. Engage with more party officials interested in public opinion research in order to increase the number of “data champions.” This is an integral part of DPR’s ongoing work with the political parties. DI staff noted that the success of these strategies depends on obtaining and reinforcing the commitment of national and regional party leaders. Other conditional factors are the parties’ internal organization, management capacity, and financial resources, which lie outside of the scope of DPR interventions. 4. Partner with local NGOs, university-based organizations, or media to ensure a sustainable effort for a Bangladesh Barometer. Again, no further details are provided in the Year 4 Work Plan. DPR recognizes that this is a long-term effort and will require a discussion about the institutional “home” for such a project. 5. Work with the parties to familiarize party offices at different levels with the party portal, database, and related communications tools. As with Strategy 3 above, this is an integral part of DPR’s ongoing work with the political parties. KIIs with party officials suggest that this will be very challenging. DPR does not indicate whether it will give priority to parties that show effective demand, rather than continuing to cajole all three major parties to use research. Conclusions  Overall, while DPR articulated strategies for the sustainability of specific activities, there are problems with formulation. There is insufficient substantive discussion of how “buzz and momentum” can be better sustained with DPR support.  DPR’s proposal to sustain the WYCs as resource centers for political parties is unrealistic. All evidence suggests that political parties will not take “ownership” of a multi￾party resource like the WYCs during or after the DPR project period.  DPR’s strategy to sustain the NJSJ network is problematic. It is not clear how giving a grant to NJSJ would contribute to sustainability.  It is unclear why a Bangladesh Barometer is needed or how it could be established within the DPR project period. The larger objective of promoting use of research by parties seems like a higher priority. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 28 RECOMMENDATIONS A. RELEVANCE 1. USAID should continue its support for DPR and plan for continuation of DG programs that engage political parties in Bangladesh. Decisions and actions taken by the political parties will continue to affect political stability, protection of human rights, capacity of local and national governments to deliver services, progress on women's participation, continued economic growth, and protection of labor rights, among other priority USG concerns. USAID’s engagement with political parties is not a guarantee of successful outcomes on these issues, but it does mean that USAID is acting as a positive force in Bangladesh’s political evolution rather than sitting on the sidelines. 2. Future political party programs should incorporate aspects of DPR’s approach, adapted to Bangladesh’s changing political context. Programs should continue DPR’s focus on sub-national party structures but intensify efforts to reach activists at upazila, union, and other grassroots levels. Programs should continue to emphasize women’s political participation and use that as a tool for broader opening of party structures. Programs should continue to emphasize intra-party activities to make parties more democratic, transparent, and responsive—and tailor those activities to the specific needs of the parties, i.e., working with AL on greater transparency and internal democracy and working with BNP and JP on their organizational capacity at the district level. 3. USAID should act quickly in designing a follow-on program to ensure there is no gap in program activity. A gap in programming could undermine DPR’s hard-won accomplishments and complicate future program start-up. Engagement with the parties will be particularly important in the coming years because of continued weak consensus among the parties on the rules of the game. By remaining engaged with the parties, USAID is helping to identify and strengthen individuals with the skills and political capital to act as stabilizing and constructive forces in a crisis. In its approach to DPR and any follow-on program, USAID/Bangladesh should draw on broader USAID experience with political party programs in similar contexts of closing political space. B. EFFECTIVENESS Achievement of Objectives 1. DPR and future party programs should intensify activities that engage district and sub-district party committees and activists. DI can consolidate DPR successes achieved thus far by focusing on a smaller number of districts and on work at lower levels of party organizations such as upazilas and wards. For example, DPR may choose to work in 30 of the 64 districts and municipalities—and to work in districts where DI believes that results are more likely. (However, if DI or USAID is planning a final evaluation, DPR should also select a group of districts with similar characteristics to monitor as points of comparison.) 2. DPR should continue and expand its work to increase women’s participation in party committees. This work should include support to women’s membership in the sub￾district committees where there appears to be more political openness and where women can acquire the skills and political capital needed to be effective at district and national levels. DPR should revive its women candidate training program where there are opportunities in district and local elections and by-elections. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 29 3. DPR should continue to implement the youth fellowship program and support the Alumni Association. In order to leverage impact, the program should be limited to the districts where DPR is active. DPR and USAID should ensure that the selection process remains fair and that the Alumni Association is not captured by any one party. 4. DPR should increase resources for research at the district level to help party leaders better understand priorities in districts. Evaluators found little interest in political research at the national level but indications that there is interest at the local level. This data should be used to help parties articulate strategies and messages that address local concerns, especially in municipalities and city corporations (which are autonomous in allocating their fiscal resources to local policy objectives). 5. DPR should review its methodology for national-level polling to ensure that it provides quality information. DPR should conduct a thorough review of its survey research to develop a technical document that will guide its survey design; discuss sampling design in CATSS versus face-to-face surveys; identify the main issues that will be addressed; establish a set of core questions to be included in each survey; and ensure that questions are framed identically from one survey to the next. Similarly, DPR should review other research and information technology tools, such as the media monitor and web portals, to ensure that they are relevant to current party needs. Management Structure 6. USAID should continue to support regional offices under future party programs. These offices will be key to the program’s ability to reach grassroots party units and to conduct activities at the sub-national level. Future implementing partners should demonstrate capacity to manage this regional operating structure. 7. DPR should increasingly promote and train local staff to take on more responsibilities. Where feasible, DPR should plan a transition from expatriate to local component leads over the remaining life of the project, freeing up budget resources for program activities. 8. DPR should review its internal organization and procedures to assign clear responsibilities for managing M&E activities. Activities should be carried out at the field level, not the home office. Women and Youth 9. DPR should increase the number of youth who participate in the fellowship program and continue to recruit qualified female candidates. As appropriate, DI should continue to help women overcome obstacles to their participation. 10. DPR should conduct more mixed-gender activities with political party members. Both male and female party members requested more activities that would provide opportunities for them to work together in pursuit of shared goals. Monitoring, Evaluation, and Coordination 11. USAID should work with DPR to ensure compliance with requirements on the design and use of PMPs, work plans, and progress reports—particularly with respect to baseline measurements and targets. DI and USAID should revise the DPR PMP to reduce the number of indicators and adjust select indicators to ensure relevance and reliability. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 30 For example, the PMP should include an indicator (and appropriate data collection methods) to assess parties’ procedures to choose committee members. 12. DPR should enhance its use of public opinion survey instruments to measure PMP targets and project results. DPR should ensure consistency of questions from one survey to the next and consistency in the framing and wording of questions. DPR should plan for follow-up applications of the public opinion surveys to include questions from the 2012 survey, as well as for an application of the political party member survey carried out in July 2014. These surveys should be conducted in the final months of the project, and results should be included in the final report to USAID. 13. USAID should more actively identify potential synergies and lead efforts to coordinate between DPR and USG programs in other sectors. For example, USAID could direct DPR and USAID health programs to collaborate on encouraging local party leaders to focus policy debate on how to improve the quality of health services. Or, DPR could work with USAID implementing partners to engage party leaders in debating how to better implement recent legislation on child marriage and domestic violence. C. SUSTAINABILITY 1. DPR should strengthen its sustainability strategies for intra-party reforms in sub￾national structures. While maintaining and deepening its focus on a limited number of “treatment” districts, DPR should extend its impact to “non-treatment” districts. For example, DPR could consider a formal mechanism whereby party members from “non-treatment” districts observe DPR-sponsored councils in “treatment” districts to create or reinforce demand for internal democracy. 2. USAID and DPR should prioritize and accelerate the development of clear strategies for sustainability of other select activities. DPR should accelerate the formation of a partnership to sustain the NJSJ network, and collaboration with a university would help avoid perceived political bias. USAID should recognize that some activities are not sustainable without donor assistance. For example, there is no incentive for parties or NGOs to devote any resources to maintaining WYCs. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 31 LESSONS LEARNED Evaluators identified a number of “lessons learned” that merit consideration in Bangladesh and in other similar development circumstances.  Even in conditions of closing democratic space and limits to electoral competition, USAID can maintain fruitful engagement with political parties outside of parliament through programs like DPR. In countries like Bangladesh—where political parties have deep organizational structures and are prime actors in political, social, and cultural spheres—this engagement offers opportunities for continued dialogue on democratic governance.  Like in Bangladesh, most political parties in developing countries maintain an explicit commitment to internal democracy, and many fail in delivering on this commitment. A program like DPR working directly with regional and local leaders to promote compliance with the formal party constitution and bylaws can get around the weak political will of national leaders.  Even in a very large polity like Bangladesh, it is possible to achieve wide coverage by working through the party structures (committees, councils) at the sub-national level, and directly with the respective party officials. There is no need to establish additional, USAID-funded groups or committees to support program implementation.  An international (i.e., non-Bangladeshi) organization has considerable advantages in working with political parties on issues such as internal democracy and responsiveness, owing to its perceived neutrality. Local organizations would not be given the same access and trust by the political parties. In a less polarized political environment, local organizations may have more opportunities for constructive relationships across various political parties.  The positive response of youth to opportunities for working with party leaders through the DPR fellowship program confirms that there is a large, untapped reservoir of talent and energy in the party grassroots. Working with this pool of party youth may be a way of focusing emerging leaders on party responsiveness and policy debate. By definition, this is a long-term strategy that would need to be sustained.  Showing entrenched political leaders that parties’ commitments to promoting women for important leadership roles will attract female voters and is a pragmatic approach that recognizes the overriding electoral imperative behind political party evolution can reinforce and complement other program interventions that emphasize women’s full and active participation in politics and government at all levels in Bangladesh.  Supplying political parties with opportunities for conducting opinion polling has its limits as a way of inducing parties to think more about policy issues and less on political spoils. Ruling parties in particular might be less open to the offer of research services, as they have larger resources at their disposal. After a trial period, USAID programs should focus on working with parties that show an effective demand for research services. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 32 ANNEXES Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 33 ANNEX I. EVALUATION STATEMENT OF WORK Scope of Work for the Democratic Participation and Reform Program External Mid-term Performance Evaluation USAID/Bangladesh Office of Democracy and Governance Program Identification Data Program Title : Democratic Participation and Reform (DPR) Program Number : Cooperative Agreement AID-388-A-11-00003 Program Dates : April 11, 2011 - April 10, 2016 Program Funding : $12,364,850 Implementing Organization: Democracy International Agreement Officer Technical Representative (AOR): Billy Woodward I. Background Throughout Bangladesh’s political history, the country’s long-established political parties have invested great authority in their leaders but struggled over how best to encourage participation and incorporate input from their large and diverse memberships. Political parties now recognize a need to increase their meaningful interactions with voters between elections and to develop inclusive strategic visions for the future. In partnership with Democracy International, USAID is implementing a five-year project entitled “Democratic Participation and Reform in Bangladesh” (DPR) to increase opportunities for participation in and leadership by women and youth in political parties, to develop the capacity of parties to conduct and utilize research in their decision-making, and to respond to the needs of political parties for stronger internal communication, organizational best practices, and compliance with electoral law. DPR program seeks to promote broader participation in Bangladeshi political parties, especially by women and youth, to strengthen the capacity of parties to access and utilize information, and to improve the environment for more responsive politics. This goal aligns with USAID’s mission goal in Bangladesh to increase citizens’ confidence in democratic institutions. This program will also bolster USG’s strategic objective, “Governing Justly and Democratically.” To achieve these results the program will conduct activities under three major program objectives. Objective 1: Enhance Grassroots Participation. Under this objective DI will increase the political engagement of women and youth in political parties and more broadly in national political activity. There are 2 intermediate results under this objective: 1.1. Leadership opportunities for youth are expanded, and 1.2. Representation of Women in political parties is increased. Throughout the period of the program DI will establish seven regional co-located Youth and Women centers, create a Developing Young Leaders Fellowship Program, launch an awareness campaign consisting of a series of town hall meetings to inform women and youth on ways to participate in the political process, and implement a Micro-Activities funding mechanism to help encourage and fund innovative projects designed by local youth and women’s groups which will foster the goals of the program. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 34 Objective 2: Improve Information Access and Utilization. The second objective of this program is to enable political parties to increase their capacity to access and utilize evidence/research based decision making. The intermediate results under this objective are: 2.1. The political parties’ capacity to conduct and utilize research is strengthened 2.2 Relevant research tools are developed to enable their use of existing and to-be-developed information resources. To strengthen the capacity of political parties to conduct and utilize research, the program will conduct a series of activities to help political parties appreciate the value of public opinion research, to develop effective polling and research strategies, and to incorporate research findings into policy formation and communications strategies. Activities under this objective will include the development of an Applied Research Program, which will include focus group discussions (FGD) and national and regional surveys, a targeted research training program, and an expanded training-of-trainers program following the upcoming elections. Objective 3: Improve the Environment for Responsive Politics. This objective aims to provide incentives for responsive politics in Bangladesh. The 2 intermediate results under this objective are: 3.1. Improved the pathways of coordination, communication, and dialogue within the parties 3.2. Internal reform initiatives are supported The program will conduct activities to assist parties in developing more sophisticated and strategic constituency initiatives and to encourage them to consider the strategic advantages of broader engagement. Activities to support this objective will include a national conference series, the creation of an intra-party web portal, and support to specific party requests for technical assistance through a Windows of Opportunity fund. Gender and Youth engagement, while also a specific program objective, will be incorporated into program activity throughout all program objectives. This approach will ensure that women and youth benefit from and participate in all DPR activities. To track the program’s progress in engaging women and youth, disaggregated data on women and youth will be collected whenever possible. II. Objectives of the Evaluation The objective of this mid-term performance evaluation is to assess the technical and programmatic validity, assess DPR implementers’ performance to date to achieve actual results against targeted results, find whether the program is on course to meet set objectives and assess sustainability aspects of the project. The audience for this mid-term performance evaluation includes USAID/Bangladesh, US Embassy state department in Bangladesh, UKAid, political parties of Bangladesh, the USAID Asia Bureau and, other bi￾lateral and multi-lateral donors working for democratic governance and other concern development partners. III. Evaluation Questions This Scope of Work is for a mid-term performance evaluation of the DPR program implementation from April 2011 to July 2014. The evaluation should review, analyze, and evaluate the DPR program using the following prioritized questions. The evaluation should make conclusions based on the findings, identify opportunities and make recommendations for improvement. In answering these questions, the Evaluation Team should assess both the performance of USAID and that of the implementing partner(s). Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 35 Relevance 1. To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh, particularly after the January parliamentary election? To what extent is it logical for USAID to remain engaged in project with Political Parties given the current state of affairs among the major parties? Effectiveness 2. To what extent has DPR been successful in achieving program objectives? To what extent the program is on course to meet set objectives? 3. To what extent DPR implementation tools and management structure is efficient (in terms of cost effectiveness and program flexibility) in meeting project objective? 4. How effectively were women and youth engaged as planned by DPR’s interventions in the targeted areas? Are there any alternative approaches identified for greater impact? 5. To what extent DPR’s M&E system have been effectively used to track, monitor, and report results attributable to DPR activities and what changes are required to make the performance management system more effective? How effectively has DPR coordinated with other USAID/Bangladesh programs, i.e., DG, Economic Growth, Population Health Nutrition and Education, Food Disaster and Humanitarian Assistance in achieving DPR objectives and contributing to other USAID/Bangladesh objectives? Sustainability 6. How much progress is DPR making in ensuring sustainability of program activities beyond USAID support? Are there any obstacles undermine the goal of sustainability and what measures should be taken to enhance sustainability? IV. Proposed Evaluation Methodology Efforts to determine the outcomes of the DPR Program will rely on a desk review; selected key informant interviews with stakeholders, including USAID personnel, implementer staff, media personnel, political party members and CSOs who were involved in the implementation of DPR; and focus group discussions (FGDs). The evaluation team is encouraged to use mixed methodologies and suggest alternative approaches during the planning stage. For example, given the sensitive nature of political issues, the evaluation team could suggest alternative or additional approaches that can maximize data collection and analysis. The evaluation methodology will include following methods, however the evaluation team is encouraged to propose new methods of data collection and analysis in the work plan: 1. Desktop Review of Key Documents and Initial Analysis The Evaluation Team shall review relevant USAID and sector specific documents, as well as key documents from USAID’s implementing partners and outside sources. The Evaluation Team will use this literature to develop an initial response to the questions listed in Section III above, and to set forth hypothesized cause-effect relationships that can be tested through field research and interviews. The Evaluation Team will also use the information from the desk review to design tools for conducting key informant interviews and focus group discussions. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 36 2. Conduct Interviews and Field Research: The Evaluation Team will conduct interviews with USAID/Bangladesh, relevant USAID/ Washington staff, program participants, implementing partners, sub￾contractors and sub-grantees, relevant GOB representatives, civil society representatives, the media, donors, stakeholders, and other relevant beneficiaries. The Team should create sampling frame to conduct interviews of stakeholders and field visits with 1-2 sub-grantees under each type of sub-grant. The Evaluation Team’s work plan should include an interview list and proposed field visits. 3. Conduct Focus Group Discussions The evaluation should include focus group discussions with a sample of the target beneficiaries and other stakeholders involved in the implementing of the DPR program. To measure clients’ satisfaction as well as program effectiveness the team should conduct focus group discussions and/or opinion polls for media, civil society, elected officials and local and regional government officials in targeted regions. The Team will build on the proposed methodology and provide more specific details on the evaluation methodology in the Evaluation Work Plan (see Deliverables below). The evaluation will be participatory in its design and implementation and the evaluation methodology will be finalized through further review and discussion between USAID/Bangladesh and the Evaluation Team. The methodology narrative should discuss the merits and limitations of the final evaluation methodology. In the final evaluation report, the evaluator should also detail limitations and how these limitations were addressed or how limitations were taken to account in proposing recommendations. The Evaluation Team will design appropriate tools for collecting data from various units of analysis. The tools will be shared with USAID during the evaluation and as part of the evaluation report. The information collected will be analyzed by the Evaluation Team to establish credible answers to the questions and provide major trends and issues. Existing Sources of Information USAID/Bangladesh DG Office will provide documents for the desk review that are not available from other sources and contact information for relevant interviewees. The list is not exhaustive and the Evaluation Team will be responsible for identifying and reviewing additional materials relevant to the evaluation. List of Documents (provided by USAID) 1. DPR program documents (RFA, Cooperative Agreement, work plans, etc.) 2. DPR PMP 3. DPR performance reports 4. DPR deliverables and research reports 5. DPR communications materials 6. DPR data bases of participants 7. DPR sub-grants and contracts for program delivery 8. Other relevant reports from DPs 9. Other relevant national level/government reports VI. Deliverables All deliverables are internal to USAID and the evaluation team unless otherwise instructed by USAID. Evaluation deliverables include: Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 37 Evaluation Team Planning Meeting(s) – essential in organizing the team’s efforts. During the meeting (s), the team will review and discuss the SOW in its entirety, clarify team members’ roles and responsibilities, work plan, develop data collection methods and instruments, review and clarify any logistical and administrative procedures for the assignment and prepare for the in-brief with USAID/Bangladesh; Work Plan – Complete a detailed work plan (including task timeline, methodology outlining approach to be used to answer each evaluation question, and describe in detail the team responsibilities, and the data analysis plan): the draft work plan will be submitted within 10 working days after commencement of the evaluation; the final work plan will be submitted within 3 days after the international team members’ arrival in Bangladesh. In-brief Meeting – An in-brief meeting with USAID/Bangladesh will be held within 2 working days of international team members’ arrival in Bangladesh; Evaluation Design Matrix – A table will be prepared that lists each evaluation question and the corresponding information sought, information sources, data collection sources, data analysis methods, and limitations. The matrix should be finalized and shared with USAID/Bangladesh before evaluation field work starts. It should also be included as an annex in the evaluation report. Data Collection Instruments – Data collection instruments will be developed and submitted to USAID/Bangladesh during the evaluation design phase prior to the commencement of the evaluation field work. The completed evaluation report should also include the data collection tools, instruments and list of people interviewed as an annex in the evaluation report. Weekly Updates - The Evaluation Team Leader (or his/her delegate) will brief the Bangladesh Democracy and Governance Programs Evaluation (BDGPE) COR on progress with the evaluation on a weekly basis, in person or by electronic communication. Any delays or complications must be quickly communicated to USAID/Bangladesh as early as possible to allow quick resolution and to minimize any disruptions to the evaluation. Emerging opportunities for the evaluation should also be discussed with USAID/Bangladesh. Debriefing with USAID – A PowerPoint presentation of initial findings, conclusions and preliminary recommendations will be made to USAID/Bangladesh before the international team members depart from Bangladesh. Debriefing with Partners - The team will present the major findings from the evaluation to USAID partners (as appropriate and as defined by USAID) through a PowerPoint presentation prior to the team’s departure from the country. The debriefing will include a discussion of achievements and activities only, with no recommendations for possible modifications to project approaches, results, or activities. The team will consider partner comments and incorporate them appropriately in drafting the final evaluation report. Draft Evaluation Report – The Evaluation team will analyze all data collected during the evaluation to prepare a draft Performance Evaluation Report and submit the report within 15 working days on after the departure of international team members from Bangladesh. The draft report must be of a high quality with well-constructed sentences, and no grammatical errors or typos. The report should answer ALL the evaluation questions and the structure of the report should make it clear how the evaluation questions were answered. The draft report must meet the criteria set forth under the final report section below. USAID will provide comments on the draft report within ten working days of submission. The Evaluation Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 38 Team will in turn revise the draft report into a final Performance Evaluation Report, fully reflecting USAID comments and suggestions. Final Report: The Evaluation Team will submit a final Performance Evaluation Report that incorporates Mission comments and suggestions no later than ten working days after USAID/Bangladesh provides written comments on the draft Performance Evaluation Report. The format of the final report is provided below. The final report must meet the following criteria to ensure the quality of the report:  The evaluation report must represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why.  Evaluation reports shall address all evaluation questions included in the scope of work.  The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Evaluation findings will assess outcomes and impact on males and females.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. The format of the final performance evaluation report should strike a balance between depth and length. The report will include a cover sheet, table of contents, table of figures and tables (as appropriate), glossary of terms (acronyms), executive summary, introduction, purpose of the evaluation, scope and methodology, findings, conclusions, lessons learned and recommendations. Where appropriate, the evaluation should utilize tables and graphs to link with data and other relevant information. The report should include, in the annex, any “Statement of Differences” by any team member or by USAID on any of the findings or recommendations. The report should not exceed 30 pages, excluding annexes. The report will be submitted in English, electronically in both word and PDF forms. The report will be disseminated within USAID. Upon instruction from USAID, Social Impact (SI) will submit (also electronically, in English) this report excluding any potentially procurement-sensitive information to Development Experience Clearinghouse (DEC) for dissemination among implementing partners, stakeholders, and the general public. The DEC submission must be within three months of USAID’s approval of the final report. All quantitative data, if gathered, must be (1) provided in an electronic file in easily readable format; (2) organized and fully documented for use by those not fully familiar with the project or the evaluation; (3) owned by USAID and made available to the public barring rare exceptions. A CD with all the data could be provided to the COR. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 39 The final report will be edited/formatted by Social Impact and provided to USAID/Bangladesh 15 working days after the Mission has reviewed the content and approved the final revised version of the report. VII. Team Composition/ Technical Qualifications and Experience Requirements for the Evaluation Team The Contractor will provide a team of specialists for the evaluation composed of experts in conducting assessments and evaluations of this nature. The team will include and balance several types of knowledge and experience related to program evaluation. Individual team members should have the technical qualifications as described below. The proposed team composition will include one team leader and two team members. USAID will select/approve the proposed candidates for each position based on the proposed methodology and the strength of the candidate(s). Team Leader (International): A mid-level social scientist/political scientist with an advanced degree in a relevant discipline and at least eight (8) years of experience. Experience in democratic participation programs and political parties programming in developing countries is required. Prior experience and ability to conduct evaluations, in particular on democratic participation issues, political party strengthening programs, and to write well in English is required. The team leader will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to USAID/Bangladesh. Ability to produce a high quality evaluation report in English is essential. Team Member (International): A mid-level evaluation specialist with an advanced degree in a relevant discipline and at least five (5) years of experience. Experience in design, management monitoring and evaluation of democratic participation programs in developing countries is required. Prior experience and ability to conduct evaluations, in particular on democratic participation issues, political party strengthening programs, and to write well in English is required. S/he will work with the national team member to manage focus group discussions (FGD) and other data collection methods, analyze findings and draft inputs for the evaluation report. Team Members (National): Team experience should include post-graduate level social science, law, economics, and/or political science experience. In-depth knowledge of issues relating to Bangladesh political development, political parties, and process of democratic participation and role of civil society in developing and/or transitional democracies is required. Experience in conducting evaluations or assessments is essential. Ability to conduct interviews and discussions and write well in English is essential. Bangladesh Democracy and Governance Programs’ Evaluation (BDGPE) will include Dhaka office based staff as a fourth team member for this Evaluation. The Team will be supported by interpreter/translators (as needed) through the auspices of the Bangladesh Democracy and Governance Program Evaluations (BDGPE) project. VIII. Conflict of Interest All evaluation Team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the program being evaluated. USAID/Bangladesh will provide the conflict of interest forms. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 40 IX. Scheduling and Logistics/Logistical Support and Government Furnished Property The proposed evaluation will be funded and implemented through the BDGPE project. Social Impact will be responsible for all offshore and in-country administrative and logistical support, including identification and fielding appropriate consultants. Social Impact support includes arranging and scheduling meetings, international and local travel, hotel bookings, working/office spaces, computers, printing, photocopying, arranging field visits, local travel, hotel and appointments with stakeholders. The Evaluation Team will be required to perform tasks in Dhaka, Bangladesh and also will travel to activity sites within the country. The evaluation Team should be able to make all logistic arrangements including the vehicle arrangements for travel within and outside Dhaka and should not expect any logistic support from the Mission. The Team should also make their own arrangement on space for Team meetings, and equipment support for producing the report. Schedule Task/ Deliverable Proposed Dates LOE Team Leader Team Member (Intl) Team Member (Natl) Review background documents & preparation work: Draft work plan submitted to USAID/Bangladesh by 9/6 8/22-9/6 3 3 3 Travel to Bangladesh by expat team members 9/5-9/6 2 2 Team Planning Meeting hosted by BDGPE 9/7 1 1 1 In-brief with USAID/Bangladesh 9/8 .5 .5 .5 Meet with Democracy International staff 9/8 .5 .5 .5 Submit Final Work Plan to USAID 9/10 COB Data collection 14 14 14 9/9-9/24 Analysis and product drafting in-country Evaluation Team submits annotated report outline and draft presentation for USAID/Bangladesh DG Team review; data collection continues after submission 9/22 - - - USAID provides comments (as needed) on report outline and draft presentation; team continues field work 9/23 2 2 2 Presentation and debrief with DG Team and USAID/Bangladesh 9/25 .5 .5 .5 Debrief meetings with key stakeholders 9/25 .5 .5 .5 Expat Team members depart Bangladesh 9/26-9/27 2 2 - Produce draft report to USAID 9/29-10/25 6 6 3 USAID and partners review draft and provide comments 10/26 – 11/6 - - - Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 41 X. Reporting Requirements The total pages, excluding references and annexes, should not be more than 30 pages. The following content (and suggested length) should be included in the report: Table of Contents Acronyms Executive Summary - concisely state the project purpose and background, key evaluation questions, methods, most salient findings and recommendations (2-3 pp.); 1. Introduction – country context, including a summary of any relevant history, demography, socio￾economic status etc. (1 pp.); 2. The Development Problem and USAID’s Response - brief overview of the development problem and USAID’s strategic response, including design and implementation of the PHR program and any previous USAID activities implemented in response to the problem, (2-3 pp.); 3. Purpose of the Evaluation - purpose, audience, and synopsis of task (1 pp.); 4. Evaluation Methodology - describe evaluation methods, including strengths, constraints and gaps (1 pp.); 5. Findings and Conclusions - describe and analyze findings for each objective area using graphs, figures and tables, as applicable, and also include data quality and reporting system that should present verification of spot checks, issues, and outcomes (12-15 pp.); 6. Lessons Learned - provide a brief of key technical and/or administrative lessons on what has worked, not worked, and why for future project or relevant program designs (2-3 pp.); 7. Recommendations – prioritized and numbered for each key question; should be separate from conclusions and be supported by clearly defined set of findings and conclusions. Include recommendations for future project implementation or relevant program designs and synergies with other USAID projects and other donor interventions as appropriate (3-4 pp). Annexes – to include statement of work, documents reviewed, bibliographical documentation, evaluation methods, data generated from the evaluation, tools used, interview lists, meetings, focus group discussions, surveys, and tables. Annexes should be succinct, pertinent and readable. Should also include if necessary, a statement of differences regarding significant unresolved difference of opinion by funders, implementers, or members of the evaluation team on any of the findings or recommendations. The report format should be restricted to Microsoft products and 12-point type font should be used throughout the body of the report, with page margins one inch top/bottom and left/right. BDGPE reviews draft comments, edits, finalizes, and submits to USAID 11/28 3 3 - TOTAL 35 35 25 Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 42 ANNEX II. DPR PROJECT RESULTS FRAMEWORK 2.2: Relevant research tools are developed Objective 2: Information Access and Utilization Improved Result 2.1: Capacity to Conduct and Utilize Research Strengthene d Objective 1: Grassroots Participation Enhanced Result 1.2: Women’s Representatio n in Political Parties Increased Result 1.1: Leadership Opportuniti es for Youth Expanded Citizens Confidence in Governance Institutions Increased Goal: Strengthened Political Processes Objective 3: Improved Environment for Responsive Politics Result 3.2: Reform Initiatives Supported Result 3.1: Pathways of Coordinatio n and Dialogue Improved Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 43 ANNEX III: MAP OF EVALUATION SITES Key Informant Interviews Focus Group Discussions [Evaluation Sub-Team] Key Informant Interviews Focus Group Discussions Key Informant Interviews Key Informant Interviews Focus Group Discussions [Evaluation Sub-Team] Site: Jessore Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 44 ANNEX IV. DOCUMENTS REVIEWED Donor Documents: 1. Action Memorandum. (n.d.). Democracy and Governance Office, USAID/Bangladesh. 2. Bangladesh Country Development Cooperation Strategy FY 2011 – FY 2016. (2011). USAID/Bangladesh. 3. Democratic Participation and Reform in Bangladesh (DPR) Request for Applications (USAID￾Bangladesh-388-10-003-RFA). (2011). USAID/Bangladesh. 4. Performance Monitoring & Evaluation Tips - Preparing a Performance Management Plan (7). (2010). Retrieved from USAID website: http://pdf.usaid.gov/pdf_docs/pnadw107.pdf DPR Project Documents: 5. DI performance indicators - September 2014 collection. (2014). Democracy International, Inc. 6. Implementation Plan – Year 1: Democratic Participation and Reform in Bangladesh (DPR). (2011). USAID/Bangladesh. 7. Implementation Plan – Year 2: Democratic Participation and Reform in Bangladesh (DPR). (2012). USAID/Bangladesh. 8. Implementation Plan – Year 3: Democratic Participation and Reform in Bangladesh (DPR). (2013). USAID/Bangladesh. 9. Implementation Plan – Year 4 (Revised): Democratic Participation and Reform in Bangladesh (DPR). (2014). USAID/Bangladesh. 10. Performance Management Plan (PMP): Democratic Participation and Reform in Bangladesh (DPR). (2011). USAID/Bangladesh. 11. Quarterly Report - Program Startup – Q4 2011: Democratic Participation and Reform in Bangladesh (DPR). (2012). USAID/Bangladesh. 12. Quarterly Report – Q1 2012: Democratic Participation and Reform in Bangladesh (DPR). (2012). USAID/Bangladesh. 13. Quarterly Report – Q2 2012: Democratic Participation and Reform in Bangladesh (DPR). (2012). USAID/Bangladesh. 14. Quarterly Report – Q3 2012: Democratic Participation and Reform in Bangladesh (DPR). (2012). USAID/Bangladesh. 15. Quarterly Report – Q4 2012: Democratic Participation and Reform in Bangladesh (DPR). (2012). USAID/Bangladesh. 16. Quarterly Report – Q1 2013: Democratic Participation and Reform in Bangladesh (DPR). (2013). USAID/Bangladesh. 17. Quarterly Report – Q2 2013: Democratic Participation and Reform in Bangladesh (DPR). (2013). USAID/Bangladesh. 18. Quarterly Report – Q3 2013: Democratic Participation and Reform in Bangladesh (DPR). (2013). USAID/Bangladesh. 19. Quarterly Report – Q4 2013: Democratic Participation and Reform in Bangladesh (DPR). (2013). USAID/Bangladesh. 20. Quarterly Report – Q1 2014: Democratic Participation and Reform in Bangladesh (DPR). (2014). USAID/Bangladesh. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 45 21. Quarterly Report – Q2 2014: Democratic Participation and Reform in Bangladesh (DPR). (2014). USAID/Bangladesh. 22. Quarterly Report – Q3 2014: Democratic Participation and Reform in Bangladesh (DPR). (2014). USAID/Bangladesh. 23. Sub Agreement between Democracy International, Inc. and The League of Women Voters of the United States. (n.d.). Democracy International, Inc. 24. Sub Contract Agreement between Democracy International, Inc. and Democracy Watch. (n.d.). Democracy International, Inc. 25. Sub Grant Agreement between Democracy International, Inc. and PUSPO-Bangladesh. (n.d.). Democracy International, Inc. DPR Surveys, Assessments and Research: 26. Ara, R. (2012). Study on Political Perception and Engagement of Young Wo/men in Bangladesh. Dhaka, Bangladesh: Democracy Watch. 27. Arguedas, Z. (2011). Democratic Participation and Reform in Bangladesh - Needs Assessment Trip: August 12-27, 2011. League of Women Voters of the United States. 28. Electorate Analysis, Nationwide Survey, Findings Presentation to Jatiya Party National Leadership [PowerPoint slides]. (2013). Democracy International, Inc. 29. Electorate Analysis RCC Survey, Findings Presentation to Jatiya Party National Leadership [PowerPoint slides]. (2012). Democracy International, Inc. 30. “Election: Context, Settings and Arrangements” Nationwide Survey, November 2013 – Summary of Findings [PowerPoint slides]. (2013). Democracy International, Inc. 31. Key Findings Presentation, Political Parties and Governance Nationwide Survey [PowerPoint slides]. (2012). Democracy International, Inc. 32. Nationwide Survey: Issues, Messages and Opinions of Voters – Summary of Findings [PowerPoint slides]. (2013). Democracy International, Inc. 33. PMP Survey of Party (AL, BNP, JP) Members & Leaders, July – August 2014 – Summary of Findings [PowerPoint slides]. (2014). Democracy International, Inc. 34. Political Parties and Governance Nationwide Survey – Preliminary Findings [PowerPoint slides]. (2012). Democracy International, Inc. 35. Political Parties and Governance Nationwide Survey April 2013 – Summary of Findings [PowerPoint slides]. (2013). Democracy International, Inc. 36. Political Parties and Governance Nationwide Survey July 2013 – Summary of Findings [PowerPoint slides]. (2013). Democracy International, Inc. 37. Political Parties Rapid Assessment – Baseline Assessment Report: Democratic Participation and Reform in Bangladesh (DPR). (2012). USAID/Bangladesh 38. “Post-Election Environment” Nationwide Survey, January 2014 - Summary of Findings [PowerPoint slides]. (2014). Democracy International, Inc. 39. “TK” Survey of Bangladesh RMG Workers, April 2014 – Summary of Findings [PowerPoint slides]. (2014). Democracy International, Inc. Secondary Research: 40. Ahmad, A. (2013). Bangladesh in 2012: Economic Growth, Political Under-development. Asian Survey, 53(1), 73-83. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 46 41. Ahmad, A. (2014). Bangladesh in 2013: Year of Confusions, Confrontations, Concerns. Asian Survey, 54(1), 190-198. 42. DI & CRI discuss public opinion research | Centre for Research and Information. (n.d.). Retrieved from http://cri.org.bd/2014/05/15/di-and-cri-a-mutual-journey-towards-political￾research/ 43. Green, A., Rizvi, J., Mostofa, N., & Ahmed, B. (2013). Mid-Term Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project. Retrieved from USAID website: http://pdf.usaid.gov/pdf_docs/pdacx655.pdf 44. Jakir, A. (2014, September 21). AL leaders for tough stance against BNP. The Independent [Dhaka]. 45. Majumder, S. (2011). What was wrong in Bangladesh before the army intervention in politics and state affairs in 2007? The Journal of Social Studies, 130, 1-14. 46. Riaz, A. (2014). Bangladesh's Failed Election. Journal of Democracy, 25(2), 119-130. 47. Seligson, M. A. (2005). Improving the Quality of Survey Research in Democratizing Countries. Retrieved from American Political Science Association website: https://my.vanderbilt.edu/seligson/files/2013/12/Improving-the-Quality-of-Survey-Research-in-PS￾March-2005.pdf Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 47 ANNEX V. PERSONS INTERVIEWED Chittagong Name Sex Position Affiliation Chanchala Gunewardena Female Associate Director, Career Development Center Asian University for Women S. M. Jafar Sadek Male Senior Officer, Career Development Center Asian University for Women Anonymous (5) Female Awami League Anonymous (6) Male Awami League Moslem Uddin Male President Awami League Suchitra Lala Munni Female DPR Youth Fellow Awami League Anonymous (4) Male Bangladesh Nationalist Party Anonymous (6) Female Bangladesh Nationalist Party Aslam Chowdhury Male Convener of Chittagong Bangladesh Nationalist Party Imtiaz Ahmed Male DPR Youth Fellow Bangladesh Nationalist Party Jafrul Islam Chowdhuri Male President of Chittagong South District Bangladesh Nationalist Party Fatema Begum Female Deputy Regional Coordinator, Chittagong Women and Youth Center Democracy International Sadrul Islam Male Regional Coordinator, Chittagong Women and Youth Center Democracy International Anonymous (3) Male Jatiya Party Anonymous (6) Female Jatiya Party Md. Solaiman Alam Seth Male President Jatiya Party Taslina Akhter Female DPR Youth Fellow Jatiya Party Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 48 Dhaka Name Sex Position Affiliation Kayser Kamal Male Member, Central Advisory Council Bangladesh Nationalist Party Ismail Zabihullah Male Research Coordinator Bangladesh Nationalist Party Farida Yesmin Female Vice President, Kustia District Committee; Member, Central Committee of Women's Wing Bangladesh Nationalist Party Sohrab Hasan Male Joint Editor Daily Prothom Alo Katie Croake Female Acting Chief of Party, Director of Grassroots Program Democracy International Haley Smith Female Political Party Specialist, Fellowship Program Democracy International Lipika Biswas Female Program Manager, Fellowship Program Democracy International Aminul Ehsan Male Senior Program Manager, Grassroots Program Democracy International Iqbal Mahmood Male Senior Program Manager, Political Parties Program Democracy International Golam Mostofa Male Senior Program Manager, Research and M&E Democracy International Helen Barnes Female Governance Advisor Department for International Development (DFID) Richard Butterworth Male Governance Team Leader Department for International Development (DFID) Alistair Legge Male Chief of Party International Foundation for Electoral Systems Silja Paasilinna Female Deputy Chief of Party International Foundation for Electoral Systems Frank Bonne Male Resident Country Director International Republican Institute (IRI) Abdul Basit Chowdhury Male President, DPR Youth Fellow Alumni Association Jamaat-e-Islami Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 49 Dhaka Name Sex Position Affiliation Nazmul Ahsan Kalimullah Male Chairman JANIPOP National Election Observation Council Boby Hazzaj Male Advisor to the Chairman Jatiya Party Masuda M. Rashid Chowdhury Female Presidium Member, Former Secretary of Women's Wing Jatiya Party Rishi Datta Male Resident Director National Democratic Institute (NDI) Alonzo Glenn Susson Male Country Program Director Solidarity Center Jerome Sayre Male Chief of Party, Strengthening Democratic Local Governance (SDLG) Program Tetra Tech ARD Hasan Mazumdar Male Country Representative The Asia Foundation Billy Woodward Male Democracy and Governance Officer USAID/Bangladesh Rejwana Farha Female Political Process Advisor USAID/Bangladesh Khulna Name Sex Position Affiliation Anonymous (8) Female Awami League Jesmin Sultana Female DPR Youth Fellow Awami League Anonymous (9) Male Bangladesh Nationalist Party Nadira Islam Female Regional Coordinator, Khulna Women and Youth Center Democracy International Anonymous (5) Female Jatiya Party Belal Hosen Male DPR Youth Fellow Jatiya Party Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 50 Rangpur Name Sex Position Affiliation Anonymous (10) Male Awami League Sabeena Yasmin Female DPR Youth Fellow Awami League Sonia Akter Female DPR Youth Fellow Awami League Anonymous (5) Male Bangladesh Nationalist Party Kajoli Begum Female DPR Youth Fellow Bangladesh Nationalist Party Md. Nasimul Alam Male DPR Youth Fellow Bangladesh Nationalist Party Md. Ali Ejad Male Regional Coordinator, Rangpur Women and Youth Center Democracy International Rehena Begum Female Deputy Regional Coordinator, Rangpur Women and Youth Center Democracy International Anonymous (7) Male Jatiya Party Anonymous (8) Female Jatiya Party Rasheda Begum Female DPR Youth Fellow Jatiya Party S. M. Maglubun Mantasa Female DPR Youth Fellow Jatiya Party United States Name Sex Position Affiliation Will Covey Male Deputy Director of Programs Democracy International David Dettman Male Former Chief of Party Democracy International Glenn Cowan Male Principal and Co￾Founder Democracy International Jeff Van Ness Male Democracy Specialist USAID Center for Excellence on Democracy, Human Rights, and Governance Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 51 ANNEX VI. EVALUATION MATRIX Political Party Leaders (FGD & KII) Women & Youth (FGD & KII) DPR Management & Staff (KII & Document Review) USAID, USG, other donors and implementers (KII) EQ 1: To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh, particularly after the January parliamentary election? To what extent is it logical for USAID to remain engaged in project with Political Parties given the current state of affairs among the major parties? EQ 1.1 (Relevance) - What engagement did you have with DPR and DPR activities? - What impact did the January parliamentary elections on your party and the political context in Bangladesh? - From your perspectives, are DPR objectives related to political parties still relevant to the development circumstances in Bangladesh, particularly after the parliamentary election and given the current state of affairs among the political parties? - From your perspective, to what extent is the current state of affairs among the major parties affected by the level of democracy and representation within the major parties? - What are the major challenges facing your party (or political parties) in the coming 1-2 years? - What engagement did you have with DPR and DPR activities? - What has been the impact of the January parliamentary elections on the participation of women and youth in politics and political parties? - From your perspectives, are DPR objectives related to greater engagement of women and youth still relevant to the development circumstances in Bangladesh, particularly after the parliamentary election and given the current state of affairs among the political parties? - What were the program objectives and sub-objectives? - How did the program contribute to USAID DO 1 (“Citizen confidence in governance institutions increased.”) and to the DO development hypothesis: “support for increased citizen engagement and improved governance will generate increased demand for democratic practices and better services, which in turn, will culminate in improved accountability and responsiveness to citizen needs, thereby increasing citizen confidence in governance institutions.” - From your perspective, how has the January election changed the political context in Bangladesh? - How do DPR objectives contribute to USAID DO 1 (“Citizen confidence in governance institutions increased.”) and to the DO development hypothesis: “support for increased citizen engagement and improved governance will generate increased demand for democratic practices and better services, which in turn, will culminate in improved accountability and responsiveness to citizen needs, thereby increasing citizen confidence in governance institutions.” EQ 2: To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh, particularly after the January parliamentary election? To what extent is it logical for USAID to remain engaged in project with Political Parties given the current state of affairs among the major parties? EQ 2.1 (Objective 1.2: Women) - Are you aware of DPR activities with women? Please describe which ones? - Are there women on the committee at your level? How many? - Are you aware of the RPO requirement for women - Did you participate in DPR programs with women? Please describe the concrete activities. - Which of those activities do you consider particularly successful? Which were not successful? - Describe DPR activities to increase women’s engagement. - Which of those activities do you consider particularly successful? Which were not successful? - To what do you attribute that success/failure? What lessons learned from that experience? Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 52 Political Party Leaders (FGD & KII) Women & Youth (FGD & KII) DPR Management & Staff (KII & Document Review) USAID, USG, other donors and implementers (KII) participation on committees? Do you have plans to increase to RPO requirement? What are the plans? - Are the women who are on the committees typically independent or do they get their positions through family connections? - What role do women and youth play in your party committee? Are they actively involved in discussions? How many women are on your committee? - Has the role or number of women or youth on committees changed in the last few years? - Did your candidate selection process include consideration of women? - Have DPR activities changed the type or level of engagement of women by your party? - How would your engagement in politics be different today if you had not been participated in DPR activities? - What role do women play on the political committees in your region? - To what do you attribute that success/failure? What lessons learned from that experience? - Have you worked with other donors or implementers? How are DPR’s activities different or the same, better or worse, than those other implementers? - Did you coordinate these activities with other donors/implementers? - PMPs from Objective 1.2 EQ 2.2 - (Objective 1.2: Youth) - Are you aware of DPR activities to engage youth such as the Youth Fellowship? Please describe what you know about those activities and your engagement with them - Did you participate in DPR youth activities? Please describe. - How were your selected for participation? - Please describe the training you received and the activities you conducted. - What impact did those activities have on the work and operations of the party? Did they make the party more open, representative and democratic? - In general, has representation of youth in parties and their engagement changed in the past 2- 3 years? Has it changed since the January election? Do you think it will change in the coming 2-3 years? For better or worse? - Describe DPR activities to increase youth engagement. - Which of those activities do you consider particularly successful? Which were not successful? - To what do you attribute that success/failure? What lessons learned from that experience? - Did you coordinate these activities with other donors/implementers? - PMPs from Objective 1.3 Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 53 Political Party Leaders (FGD & KII) Women & Youth (FGD & KII) DPR Management & Staff (KII & Document Review) USAID, USG, other donors and implementers (KII) EQ 2.3 - (Objective 2: Research) - Did you participate in or are you aware for DPR activities related to political research and data such as Surveys, Media monitor, automated telephone surveys, Bangladesh Barometer? Please describe how you participated or engaged in those activities. - If you participated in those activities, how would you evaluate their impact or success? How did you integrate them into the work of the party? - How, if at all, does your party use research, surveys or data produced by DPR or anyone else? - From your perspective, is it important for political parties to include public opinion surveys, data and research into their operations and campaigns? - Describe DPR activities related to research, data and surveys. - Which of those activities do you consider particularly successful? Which were not successful? - To what do you attribute that success/failure? What lessons learned from that experience? - Did you coordinate these activities with other donors/implementers? EQ 2.4 - (Objective 3: Representation) - Did you participate in the District Party conferences sponsored by DPR? What was your role in those events? What did those events accomplish? Did the conference produce a party platform or manifesto? Were there any other changes in the party operations or practices as a result of the conference? How would you evaluate the success or importance of the conference? - How are District & Upazilla committees formed and leaders selected? Are you aware of the party constitutions requirements that committee selection to go bottom up? Would you say that this is the case in your region? - How did DPR support the party conferences? How did parties change as a result of conferences? - Which of those activities do you consider particularly successful? Which were not successful? - To what do you attribute that success/failure? What lessons learned from that experience? - Did you coordinate these activities with other donors/implementers? Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 54 Political Party Leaders (FGD & KII) Women & Youth (FGD & KII) DPR Management & Staff (KII & Document Review) USAID, USG, other donors and implementers (KII) How does one become a member of the party committee at your level, or a committee leader (eg president, secretary, etc)? How does one become a member of a higher level party committee? Has this process changed in the past 2- 3 years? (For national KII also ask about central committee and leadership selection.) Are you satisfied with this process? If the process were different – more open and democratic – what impact would that have on party operations? Would it have an impact on relations between the parties? - How were candidates for parliamentary election selected? Are you aware for the RPO requirements for candidate selection? Were candidates proposed from the grassroots as required? Do you believe the best candidate for the party was selected? What role does the national party play in candidate selection? - What role did you and the party committee play in the election campaign? What strategies, tactics and methods did you employ to increase your vote and advocate for the candidate? (i.e Personalities? Policies? History? Family connections? Accusations of corruption against other parties?) What help did you get from the national level party? Was there any research or data Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 55 Political Party Leaders (FGD & KII) Women & Youth (FGD & KII) DPR Management & Staff (KII & Document Review) USAID, USG, other donors and implementers (KII) employed? How - if at all - are the views and needs of party members, grassroots organizations, citizens, etc taken into consideration by the party? Conferences? Counciling? Informal meetings? EQ 3: To what extent DPR implementation tools and management structure is efficient (in terms of cost effectiveness and program flexibility) in meeting project objective? EQ 3.1 (Management) - How effectively has DPR used WOO, sub-grants, etc. What changes has DPR made in its work plans and budget in response to the changing political environment? What is the % of the project budget spent in the field activities versus program management (DPR Dhaka office and DI HQ) spending? Do the individual WYCs have “cost center” budgets? Is there a budget template for the WYCs, with staffing, infrastructure and activity budgets? Do WYCs have their own operational plans and schedules? - Do DPR Component managers have their own indicators and targets? Are these from the PMP? How do they monitor and manage component performance? Do component managers have tools/formats for monitoring individual activities? Does DPR track responsiveness/attendance at training? What is the % of “no show”? - Has DPR evaluated the effectiveness of its training? Does Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 56 Political Party Leaders (FGD & KII) Women & Youth (FGD & KII) DPR Management & Staff (KII & Document Review) USAID, USG, other donors and implementers (KII) DPR use before/after evaluations for training events? EQ 4: How effectively were women and youth engaged as planned by DPR’s interventions in the targeted areas? Are there any alternative approaches identified for greater impact? EQ 4.1 (Women and Youth Activities) n/a - Did DPR engage and train the targeted quantity and distribution of youth? - Did the youth training program increase participant skills and capacity? Were youth more active in party management and decision-making? - Did DPR engage and train the targeted quantity and distribution of women?- Were women better represented in parties as a result of DPR programs? - Were there more women candidates in elections, and were they more successful, as a result of DPR? - What alternative approaches to women and youth programs could be identified for greater impact? EQ 5: To what extent DPR’s M&E system have been effectively used to track, monitor, and report results attributable to DPR activities and what changes are required to make the performance management system more effective? How effectively has DPR coordinated with other USAID/Bangladesh programs, i.e., DG, Economic Growth, Population Health Nutrition and Education, Food Disaster and Humanitarian Assistance in achieving DPR objectives and contributing to other USAID/Bangladesh objectives? EQ 5.1 ( M&E) - Is the PMP clear and well￾organized? Are the indicators clearly and comprehensively defined, along with data sources and data collection? Does the PMP include data reference sheets? Is the number of indicators adequate to reflect DPR project results, but manageable? Are the PMP indicators closely aligned with DPR project activities and objectives? Is there a results framework showing causal Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 57 Political Party Leaders (FGD & KII) Women & Youth (FGD & KII) DPR Management & Staff (KII & Document Review) USAID, USG, other donors and implementers (KII) relationships between the DPR outcomes? Are key DPR outputs, outcomes, and impacts reflected in the results framework? Do the selected indicators cover DPR’s major areas of project spending? Are the anticipated costs for implementing the PMP included in the DPR budget? - Is there enough information in the PMP to judge the quality of the indicators? Do the indicators reliably measure what is intended, and is their collection and use feasible? Are indicators sufficiently disaggregated to meet DPR project information needs? Was there a baseline data collection done before the DPR project started? Does the PMP generate information for DPR reports to USAID? Has DPR carried out focused evaluations of specific components/activities? Has the PMP helped DPR management to identify problems with project design or implementation? Examples? EQ 5.2 (Coordination) - What are the Task Order requirements for coordination? Has DPR presented specific proposals to other IPs for coordination? How is coordination defined in the annual work plans? Do the DPR work plans set out specific coordination activities with other USAID programs? How have these work plan activities been designed and negotiated with other IPs? Are there written Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 58 Political Party Leaders (FGD & KII) Women & Youth (FGD & KII) DPR Management & Staff (KII & Document Review) USAID, USG, other donors and implementers (KII) agreements, memos, etc? What role do USAID CORs/AORs play in fomenting this coordination? DG Office role? Do the other IPs include coordination in their respective work plans? Have the activities that required coordination been effectively carried out as per the work plans and schedules? EQ 6: How much progress is DPR making in ensuring sustainability of program activities beyond USAID support? Are there any obstacles undermine the goal of sustainability and what measures should be taken to enhance sustainability? EQ 6.1 (Sustainability) - What planning has taken place for the sustainability of the WYCs? - What measures have taken place for sustainability of research program, CATSS and Bangladesh barometer? - What obstacles undermine the goal of sustainability and what measures should be taken to enhance sustainability? Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 59 ANNEX VII. FOCUS GROUP DISCUSSION PROTOCOLS Introduction, Ground Rules, Confidentiality, and Informed Consent The moderator should first introduce herself or himself, then welcome and thank all of the focus group discussion (FGD) participants. After that, the moderator should explain the purpose of the FGD, how participants were selected, any potential benefits or risks to participating in the FGD, how long it will take, and whether there will be any compensation for participating. After discussing these logistical issues, the moderator also should address expectations, or ground rules, for the FGD. The ground rules will vary depending on the FGD, but in general they will include:  Everyone is encouraged to share their ideas, and the FGD is strengthened if everyone participates.  There are no wrong answers, and everyone’s perspective is equally valued.  The ideas shared during the FGD should not be shared outside the FGD with non-participants in order to respect participants’ privacy.  Disagreements about ideas can be valuable and productive, but personal attacks will not be tolerated. After establishing these ground rules, the moderator should ask if there are any questions or concerns participants have, and these issues should be addressed and consensus reached as a group before moving on. The question of confidentiality is also important to address. The moderator should clearly describe how the data collected will be used, including with whom it will be shared, and crucially, whether names or other personal or identifying information will be included with the data. Many times, the experiences and opinions shared during a FGD will include sensitive information, and participants may not feel comfortable sharing openly if they feel it could have negative consequences for them in the future. The moderator must be honest about how the data will be used, but should also reassure the participants that the data will be treated sensitively and that their privacy will be respected to the greatest degree possible given the needs and purposes of the evaluation. After providing this information, it is important to describe what will or will not happen if they choose not to participate. To ensure the data collected are reliable, participation in a FGD should be entirely voluntary and there should be no consequences for declining to participate. After informing participants of all of this information, the moderator must ask each member to confirm that they consent to participate. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 60 Political Party: Participants: X Division: Date: September X, 2014 Focus Group Discussion Protocol POLITICAL PARTY LEADERS 1. OVERALL IMPRESSION OF DPR 1.1 Please describe what you know about Democracy International and DPR. In which activities did you participate? [Note to moderator: We use “Democracy International” or “DI” to describe the project. Confirm that participants understand that terminology.] Prompts: * District Grassroots Representative Conference (DGRC)? * Trainings? * Meetings? * Other? Which activities do participants mention? 1.2 Overall, what was your impression of DI? Prompts: * Was the staff knowledgeable? * Did they have a respectful approach? * Did they meet their commitments? * Did they ask your input and opinions about activities? * Was the Women and Youth Center (WYC) a convenient place to meet? (a) Mostly positive. (b) Mixed. (c) Mostly negative. What do participants mention as positive or negative impressions? 1.3 How frequently and/or actively did you interact with DI/DPR/WYC staff? (a) Very actively participated (b) Some participation (c) No participation Comments about participation. 2. DISTRICT GRASSROOTS REPRESENTATIVE CONFERENCES 2.1 One of DPR’s activities was to work with parties at the local level to help organize the District Grassroots Representative Conferences. Are you aware of these conferences? Prompts: Events under the party conferencing series have five purposes: * to encourage internal party democracy at the grassroots level; * to increase communication between the central party leadership and locally-elected committee members; * to improve party organization at local levels; * to discuss party reform activities from the local level upwards; and * to provide a specific conduit of information between local and national party organizations. (a) Yes, they remember and can describe the activities in detail. (b) They remember some activities but cannot describe in detail. (c) No, they do not remember or cannot describe any activities. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 61 2.2 What was your impression of those conferences? Were the conferences any different from other conferences organized by the party alone? Prompts: * Different topics were discussed? * They were better organized than in the past? * We had more participants than in the past? * They were more open and there was more participation from the grassroots? * More women and youth participated and were more active than in past conferences? (a) Mostly positive. (b) Mixed. (c) Mostly negative. What do participants describe as positive or negative experiences? How were the DPR conferences different from other party events? 2.3 Do you feel that anything changed (improved) in your party as a result of the conference? Prompts: * Better understanding of issues facing citizens and voters? * Better campaign strategy for elections? * More women and youth participated and were active than in past conferences? (a) There were many positive changes. (b) There were some positive changes. (c) There were no changes. What changes do participants discuss? 2.4 In particular, would you say that the conferences made you and your party more responsive to the needs and interests of citizens? If yes, how did the party become more responsive? For example: * We included suggestions from the grassroots in our party manifesto and platform. * The suggestions were sent to the national party and included in the national platform. * There was discussion about specific local issues and how the party can address them. If no, why not? What were the obstacles? * It is not important. * We do not have the time or resources to do it. (a) Yes, the conferences made the party and the local branch more responsive. (b) There were only very small changes in party responsiveness to citizens. (c) No, the conferences had no effect on party responsiveness. What changes do participants discuss? 2.5. Would you say that the party conferences made you and your party more transparent in how decisions are made about party activities and leadership? If yes, how did they become more transparent? * We had votes and elections for party leaders. * We had open discussions about issues and concerns of the grassroots. * We incorporated suggestions from the grassroots. (a) Yes, the conferences made the party and the local branch more transparent. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 62 (b) There were only very small changes in party transparency. (c) No, the conferences had no effect on party transparency. What changes do participants discuss? 3. WOMEN’S PARTICIPATION 3.1 Do you remember any DI/DPR activities related to women? If yes, which activities? Prompts: * Do you know the Narir Joye Shobar Joy campaign? * Advocacy meetings with grassroots party leaders to develop concrete action plans to appoint women to main party committees * Roundtable meetings with women political leaders to develop advocacy plans for increased women’s party leadership and to expand the Narir Joye Shobar Joy network of qualified women to serve on party committees and elected positions * Media events to highlight and raise awareness of opportunities to strengthen and increase women’s leadership in Bangladesh * Campaign and leadership trainings to build and strengthen the leadership and political skills of grassroots women leaders (a) Yes, they remember and can describe activities in detail. (b) They remember some activities but cannot describe in detail. (c) No, they do not remember or cannot describe any activities with women. Which activities do they remember and describe in most detail? 3.2 How would you describe the impact of those activities on you, on your party branch, or on women? Can you identify one or two activities that were the most successful? Prompts: * Women were more active and better speakers at party events. * We got a better understanding of the party (i.e., RPO) requirements for women’s participation. * There were more women considered as candidates to be nominated for office. * The women who are on the committees have more substance and education. * Party conferences and discussions consider issues important to women and youth. (a) Very successful, with considerable impact on the party. (b) Somewhat successful. (c) Not at all successful. What do participants say was successful and had an impact on the party? 3.3 Should women have a larger role in politics or not? Why or why not? (a) Yes (b) Not sure. (c) No. 4. YOUTH ENGAGEMENT 4.1 DPR has a youth fellowship program for training young people from your party and helping them work with party committees. Those fellows also have an alumni association for former fellows. Have you heard of this program? Have you sponsored any fellows, or have you seen their work in your district? What is your impression of this work? Prompts: * The activities bring together participants from all four political parties and range from roundtable meetings, town halls, rallies, dialogue, media campaign, and advocacy campaigns. * What kinds of activities have you seen the fellows supporting? Examples: women and youth conferences, press conferences, roundtable discussions, seminars, town hall meetings * Alumni conduct local advocacy campaigns and events on political violence, political dialogue Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 63 (a) Yes, they know the fellowship program, and they say it was a success overall. (b) They know the program but don’t see it as a success. (c) They don’t know the program. How do participants describe their experience with the fellowship program? What activities are they aware of and can describe clearly? 4.2 Should youth have a larger role in politics or not? Why or why not? (a) Yes (b) Not sure. (c) No. 5. DISTRICT COMMITTEES AND COUNCILS 5.1 Has your district party held a committee council since the parliamentary election? If not, do you have plans to hold a council? (a) Yes, already held district council. (b) No, haven’t held council yet. Does anyone volunteer that DPR supported that process? 5.2 What about grassroots councils (i.e. sub-district/upazila, union, or municipality)? These can be organized by the district committees without involvement from the national party committee and can promote grassroots involvement in the party. Do you have plans to initiate organizing committees and new councils? Are you aware of any sub-district councils that have “expired” and should be renewed? (a) Yes, we are actively promoting new councils and committees at the sub-district level. (b) We are doing some work to promote grassroots committees, but it is not very active. (c) We are not actively promoting the sub-district councils. 5.3 Party councils are private events, but DPR has helped some party committees organize their councils and ensure that they are efficiently organized. Have you had any discussions with DPR about your district committees and councils? Have you received assistance from DPR on the councilling process? Prompts: * Has DPR offered any support or guidance? * What role has DPR played… or will play… in supporting? * For example: by providing a venue, providing material for voting. (a) No, we have not had any discussions or interaction with DPR. (b) Yes, we have talked with DPR about obstacles to councils and/or how DI can support our events, and this was helpful and/or promising. (c) Yes, DPR has supported our council and that support was helpful. (d) We talked to DPR about the councils or got assistance from DPR, but that was not useful to us. 5.4 Do you think political research is an important tool to win elections or not? (a) Yes (b) Not sure. (c) No. 5.5 Are you satisfied with the level of coordination and dialogue between local and national leaders in your party? (a) Yes (b) Not sure. (c) No. 5.6 Do you feel that your party leaders understand your concerns and take them seriously? (a) Yes (b) Not sure. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 64 (c) No. 6. CLOSING 6.1 Considering everything that we have discussed, what do you consider to be your greatest challenges and needs as a local branch of your party today? Open… 6.2 What other advice or suggestions do you have for how international donors such as USAID can support parties and democratic participation in Bangladesh now and in the coming 2-5 years? Open…. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 65 Political Party: Participants: X Division: Date: September X, 2014 Focus Group Discussion Protocol WOMEN PARTY MEMBERS 1. OVERALL IMPRESSION OF DPR 1.1 Please describe what you know about Democracy International (DI), DPR, and Women and Youth Centers (WYCs). In which activities did you participate? How frequently did you interact with DI/DPR/WYC staff? Prompts: * Narir Joye Shobar Joy campaign? * Advocacy meetings with grassroots party leaders to develop concrete action plans to appoint women to main party committees * Roundtable meetings with women political leaders to develop advocacy plans for increased women’s party leadership and to expand the Narir Joye Shobar Joy network of qualified women to serve on party committees and elected positions * Media events to highlight and raise awareness of opportunities to strengthen and increase women’s leadership in Bangladesh * Campaign and leadership trainings to build and strengthen the leadership and political skills of grassroots women leaders (a) Yes, very actively participated (b) Some participation (c) No participation Which activities do participants mention unprompted? Which activities after prompting? 1.2 Overall, what was your impression of DPR? Prompts: * Was the staff knowledgeable? * Did they have a respectful approach? * Did they meet their commitments? * Did they ask your input and opinions about activities? * Was the WYC a convenient place to meet? (a) Mostly positive (b) Mixed (c) Mostly negative What do participants mention as positive or negative impressions? Prompted versus unprompted? 1.3 What were the one or two most important and valuable things that you learned? (Unprompted) 1.4 Have you or your political party received training or technical assistance through programs implemented by other NGOs/donors? How was DI/DPR different, better, or worse? (a) DPR was better. (b) DPR was about the same as the others. (c) The others were better than DPR. (d) Did not participate in other programs. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 66 Note what participants say was better or worse. 1.5 How did you personally become involved in politics? What motivated you to become engaged in politics? And why with this party? (Unprompted) 2. ADVOCATING FOR WOMEN ON COMMITTEES 2.1 Have you participated in any of the DI/DPR activities to get more women on party committees? How actively did you participate? Could you describe those activities? (a) Yes, very actively participated (b) Some participation (c) No participation Look for unprompted recall of activities. 2.2 From your perspective, were those activities successful? Why or why not? (a) Yes, very successful. (b) Somewhat successful (c) Not successful. *Describe reasons given for not successful. 2.3 The electoral law (RPO) amendment issued in 2008 requires that women reach 33 percent participation in committees by 2020. Prompts: * Are district and grassroots leaders aware of the RPO amendment? * Are political parties sincerely trying to achieve that goal? * Is it possible for parties to do it on their own, or do they need help from programs like DPR? * Has DPR helped educate the party leaders about the RPO requirements? (a) Yes, DPR has helped parties achieve the RPO requirements. (b) Somewhat. (c) No, DPR has not helped parties be more active. 2.4 Did DPR activities change how you were treated by male party leaders? Did it help them acknowledge the importance of training and participation of women and youth? (a) Yes, there were changes in how men treated women. (b) Somewhat. (c) No, there were no changes. 3. PROMOTING WOMEN CANDIDATES FOR OFFICE 3.1 DPR conducted a number of activities to promote women running for and winning public office. Are you aware of and/or did you participate in those activities? (a) Yes, actively participated. (b) Aware but did not participate or only moderate participation. (c) Did not participate. 3.2 How would you evaluate the success of those efforts… at least before the election boycott? (a) Yes, very successful (b) Somewhat successful (c) Not successful 3.3 What obstacles exist to increasing the number of women in politics and as candidates in the elections? (Unprompted) Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 67 4. CLOSING 4.1 Considering all that we have discussed, what are your greatest challenges and needs as women in a local party organization today? Open…. 4.2 What other advice or suggestions do you have for how international donors such as USAID can support parties and democratic participation in Bangladesh now and in the coming 2-5 years? Open…. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 68 ANNEX VIII. MINI-SURVEY PROTOCOLS Mini-Survey Protocol POLITICAL PARTY LEADERS Hello. I am calling from Social Impact, an independent research company based in Washington, DC. We are working on behalf of USAID to collect data about the schedule of councils and committee elections for all political parties across Bangladesh. A partner of USAID, Democracy International, provided your name as the authority on this topic in your district. I have a few questions for you. Could I speak with you for five (5) minutes? Position: Date: September X, 2014 Political Party: District: Division: 1. In your district, when did your party hold the most recent district council? Month: Year: 2. What was the method of forming your district committee? ⃝Election ⃝Selection ⃝Consensus 3. Has your party set a date for the next district council? ⃝No ⃝Yes 4. If yes, what is the date? Month: Year: 5. How many upazilas are in your district? Number: 6. How many upazila councils have been held in your district? Number: Thank you for your time. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 69 ANNEX IX. MINI-SURVEY OF DISTRICT LEADERS During the Democratic Participation and Reform (DPR) evaluation, a telephone survey of district leaders was conducted by BDGPE, using a data base of district leaders provided by DPR. The aim of the survey was to collect information about the process of organizing district and upazila level councils; the formation of party committees during the councils; and the plans for holding the next round of councils in the districts and upazilas. The sample was made up of thirty district leaders from eighteen DPR program districts20 in the Chittagong, Rangpur, and Khulna Divisions from across three political parties, as shown in Table 1 below. This mini survey is not a representative sample of all the 64 districts, hence care should be taken in generalizing from these findings from the DPR districts. Nevertheless, they do provide some interesting insights that merit further analysis and discussion. Table 1: Party Affiliation of Respondents Party No. Awami League(AL) 11 Bangladesh Nationalist Party (BNP) 10 Jatiya Party (JP) 9 Total 30 Generally, it was easier to reach District Vice Presidents and General Secretaries than the District Presidents. This is reflected in Table 2: Table 2: Party position of the respondent in the District Committees Position No. President 3 Vice-President 13 General Secretary 9 Joint-Secretary 2 Organizing Secretary 1 Other Secretary 1 Joint Convenor 1 Total 30 Findings On the question of the date of the last council at the district level, it was found that councils are not a routine event for the parties, although the tenure of the committees is well articulated in the constitution of the respective parties. Only one-third of the Districts held District Councils in the last four years. 20 The districts were: 1) Chittagong Division: Banderban, Brahmanbaria, Chittagong, Chandpur, Cox’s Bazar, Feni, Khagrachhari, Laxmipur, and Noakhali; 2) Khulna Division: Chuadanga, Jenaidah, Magura, Meherpur, and Narail; and 3) Rangpur Division: Gaibandha, Kurigram, Lalmonirhat, and Rangpur. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 70 Table 3: Date of Last District Council Years No. 1996-2000 1 2001-2005 6 2006-2010 12 2011-2014 10 Couldn’t Say 1 Total 30 The purpose of the councils is to constitute the party committees. The procedure used for constituting the committees varies considerably. As Table 4 shows, only in five Districts were the committees established through direct voting of conference participants. The rest of the committees are selected by the central level political party leaders; through a negotiated consensus between central leaders and local leaders; or through a mixture of election, selection, and consensus (where the President, General Secretary, and Organizing Secretary are elected, but they subsequently choose the remainder of the committee members—either selecting them on their own, or selecting them through consensus with central leaders). Table 4: Procedures for Establishing Committees Mode of Committee formation No. Election 5 Selection by central leaders 6 Consensus between local and central leaders 10 Consensus plus selection 2 Consensus plus election + consensus 7 Total 30 The district committees of the three parties have tenures of either two or three years, which means that most are overdue to hold their councils. Only half of the respondents indicated that the next council had been scheduled. Table 5: Next District Council Scheduled? Response No. Yes 15 No 7 Don’t know 8 Total 30 With respect to the date of the next district council, according to the respondents, all were planned for this year—with most of them occurring in December. Table 6: Date of Next Council Date No. By October 2014 2 By November 2014 3 By December 2014 9 In 2016 1 Total 15 Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 71 As Table 7 shows, in the districts covered by the mini survey, there is a wide range in terms of the number of upazilas and municipalities in each district, from two to 16 upazilas/ municipalities. The district leaders reported that in most of the upazila committees, the tenure had expired. Table 7: Number of Upazilas in the District Committees No of upazilas/ muncipalities in the district No. of Districts 3 2 4 6 5 3 6 4 7 4 8 5 9 4 10 1 16 1 30 The district level leaders also reported that they had not received instructions from the central committee about holding upazila councils, which, according to the party constitutions, are supposed to be held before the district councils so the upazilas can send councilors for the district level councils. Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 72 ANNEX X. DISCLOSURE OF ANY CONFLICTS OF INTEREST Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 73 Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 74 Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 75 Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 76 Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 77 Mid-term Performance Evaluation of the Democratic Participation and Reform (DPR) Program 78 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523