March 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Andrew Green, Julia Rizvi, and Naim Mostofa of Social Impact and Bashir Ahmed of Research, Training and Management (RTM). Mid-Term Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Cover Photo: Social Impact evaluation team photo of a District Public Policy Forum organized by PRODIP democratic governance grantee, Sabalamby Unnayan Samity (SUS), in Dhaka, Bangladesh, on January 12, 2013. MID-TERM PERFORMANCE EVALUATION OF THE PROMOTING DEMOCRATIC INSTITUTIONS AND PRACTICES (PRODIP) PROJECT March 11, 2013 USAID Contract AID-OAA-I-10-00003 Task Order AID-388-TO-12-00001 DISCLAIMER The author‘s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ii Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project CONTENTS Acronyms ..........................................................................................................................................................iii Executive Summary .......................................................................................................................................... v Introduction ....................................................................................................................................................... 1 Development Problem and USAID Response ............................................................................................ 3 Evaluation Purpose and Evaluation Questions ........................................................................................... 8 Evaluation Methods and Limitations .......................................................................................................... 10 Findings and Conclusions ............................................................................................................................. 12 Recommendations and Lessons Learned ................................................................................................. 32 ANNEXES Annex I: Evaluation Statement of Work................................................................................................... 37 Annex II: Sources of Information ............................................................................................................... 49 Annex III: Evidence Matrix ........................................................................................................................... 55 Annex IV: Semi-Structured Interview Protocols .................................................................................... 62 Annex V: Mini-Survey Protocol .................................................................................................................. 76 Annex VI: Disclosure of Conflict of Interest for USAID Evaluation Team Members .................... 78 TABLES Table 1: Parliamentary Standing Committees Targeted by PRODIP Activities .................................. 5 Table 2: Distribution of Training (Women) ............................................................................................ 13 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project iii ACRONYMS APPG All Party Parliamentary Group ASUS Adibashi Sangskritik Unnayan Songstha BDAWL Bangladesh Alliance for Women Leadership BDGPE Bangladesh Democracy and Governance Performance Evaluations BEI Bangladesh Enterprise Institute BNP Bangladesh Nationalist Party BNPS Bangladesh Nari Progati Sangha (BNPS) BNWLA Bangladesh National Women Lawyer‘s Association BWCCI Bangladesh Women Chamber of Commerce and Industry CAPTA Citizen Alliance for Promotion of Transparency and Accountability CDCS Country Development Cooperation Strategy CEDPA Center for Development and Population Activities CIPE Center for International Private Enterprise CODEC Community Development Centre CSO Civil Society Organization DCOP Deputy Chief of Party DFID Department for International Development DG Democracy and Governance DPPF District Public Policy Forum EWG Election Working Group GPG Group on Political Governance (GPG) ICT Information Communication Technology IED Institute for Environment and Development IGS Institute of Governance Studies IID Institute of Informatics and Development (D.Net) IPD Improving Democracy through Parliamentary Development IPDS Indigenous Peoples Development Services JATRI Journalism Training and Research Initiative JIPA Japan International Cooperation Agency JS Jatya Sangsad KOICA Korea International Cooperation Agency LOI Leaders of Influence M&E Monitoring and Evaluation MP Member of Parliament MRDI Management and Resources Development Initiative NDI National Democratic Institute PMP Performance Monitoring Plan PRODIP Promoting Democratic Institutions and Practices PS Private Secretary PSA Public Service Announcement PTF Parliamentary Task Force RTM Research, Training, and Management SI Social Impact, Inc. SOW Statement of Work SPEMP Strengthening Public Expenditure Management SPO Strengthening Parliamentary Oversight Project SSI Semi-Structured Interview iv Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project SUNY/CID State University of New York/Center for International Development SUS Sabalamby Unnayan Samity TAF The Asia Foundation TI Transparency International UNDP United Nations Development Programme UNFPA United Nations Population Fund USAID U.S. Agency for International Development Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project v EXECUTIVE SUMMARY This evaluation report assesses the results, outcomes, and lessons learned at the mid-term of the five-year Promoting Democratic Institutions and Practices (PRODIP) project, implemented by The Asia Foundation (TAF) and State University of New York‘s Center for International Development (SUNY/CID), funded by the U.S. Agency for International Development (USAID) and Department for International Development (DFID). The evaluation covers PRODIP project activities conducted from April 2010 to January 2013. Democracy in Bangladesh has developed in fits and starts since the country‘s independence in 1971 from the ―two-nation‖ Pakistan of 1947. It is a unitary state with a parliamentary system that follows the Westminster model. One of the most populous countries, it is also one of the poorest, with large parts of the country vulnerable to typhoons and rising sea levels. The political history of Bangladesh is dominated by competition among the Awami League party, the Bangladesh Nationalist Party (BNP) and the military. Power has alternated between the Awami League, the BNP, the military and the military-related Jatiya Party, which was formed in the 1980s. Unrest and violence marked the years leading up to the 1991 elections, the two elections in 1996 and the 2001 elections. With regular elections scheduled to take place in October 2006, a crisis stemming from a deadlock over naming a caretaker government was resolved when President Iajuddin Ahmed assumed the role. However, unrelenting political unrest and military pressure forced Ahmed to resign in early 2007 and appoint a new caretaker. Elections for the Ninth Parliament held in December 2008 were won by the Awami League, although it needed to rely on some smaller parties to form a government. The 2008 elections brought promise of opportunities for political change. The rise of social organizations and watchdogs injected a fundamentally new element into political life in Bangladesh, candidates from the major parties spoke of the need for Parliament to be a more significant actor and many Members of Parliament (MPs) arrived for the Ninth Parliament ready to work on their commitments. International donors aimed to take advantage of this new fusion of political will for reform among MPs and civil society. PROGRAM BACKGROUND In April 2010, USAID awarded a five-year cooperative agreement with a total estimated cost of $23,210,783 to TAF to implement PRODIP, a project that aims to increase the effectiveness and responsiveness of legislative institutions and processes in Bangladesh through two simultaneous and mutually-reinforcing objectives. Component 1 addresses the ―supply side‖ to develop ―An Effective and Open Parliament,‖ and Component 2 focuses on the ―demand side‖ to achieve a ―More Constructive and Sustainable Role for Civil Society in Democratic Governance.‖ TAF maintains overall responsibility for the implementation of PRODIP and leads civil society strengthening interventions under Component 2. SUNY/CID leads the implementation of parliamentary activities under Component 1. vi Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project EVALUATION PURPOSE This mid-term performance evaluation has four purposes that incorporate both performance and process aspects: 1. Evaluate PRODIP‘s overall performance by assessing results against stated targets and indicators; 2. Assess the efficacy and results of PRODIP‘s implementation approaches and management structure in meeting its objectives; 3. Make recommendations to USAID/Bangladesh concerning possible programming changes or adjustments to the second half of PRODIP‘s implementation; and 4. Analyze interventions under Objective 2 (Civil Society) to determine if they are supporting the cooperative agreement‘s intent to strengthen legislative function and processes. The audience of this evaluation is USAID/Bangladesh, USAID/Washington, leaders of USAID Forward, TAF, SUNY/CID, DFID, United Nations Development Programme (UNDP) and other bilateral and multilateral donors to Bangladesh. USAID/Bangladesh plans to use the results of this evaluation to review the feasibility of the PRODIP work plan and to consider possible programming changes or adjustments. EVALUATION QUESTIONS The evaluation questions are grouped into seven areas: A. Results: To what extent has PRODIP been successful in achieving its program objectives? How well did the activities of each of PRODIP‘s two components contribute, in a coordinated way, to building a stronger parliamentary institution? B. Management & Administration: To what extent has PRODIP‘s performance management system provided useful data to support management decisions? To what extent have PRODIP management and stakeholders incorporated knowledge gained through PRODIP‘s monitoring and evaluation (M&E) systems into project management? How effectively have TAF and SUNY/CID worked together in implementing PRODIP and achieving programmatic success? To what extent did PRODIP coordinate with other legislative strengthening programs? C. Client Satisfaction: What are the levels and areas of project stakeholder satisfaction or dissatisfaction with PRODIP‘s cooperation and performance? This group includes: leaders and staffs of Parliament, the Office of Speaker of the Parliament, and Parliamentary Secretariat; civil society and citizens‘ organizations; USAID and other implementers of legislative programs. D. Program Opportunities: To what degree is PRODIP‘s approach a cost-effective means to increase citizen confidence in governance institutions? E. Program Theory & Design: How well did USAID incorporate Section 5.3.2 of the 2009 Democracy and Governance (DG) Sectoral Assessment into the PRODIP design? F. Sustainability: How sustainable are PRODIP activities beyond USAID support? G. Cross-Cutting Issues: How well were gender issues addressed by PRODIP? Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project vii EVALUATION METHODOLOGY AND LIMITATIONS The evaluation team examined both PRODIP‘s processes and performance to address the evaluation questions. Social Impact (SI) strongly believes in the use of mixed methods approaches to answering evaluation questions, a view that is reiterated in USAID‘s Evaluation Policy. Therefore, the evaluation team applied standard rapid appraisal techniques including semi-structured interviews (SSIs) of key informants, group interviews, a materials review and one mini-survey, providing both quantitative and qualitative evidence. The team used parallel analysis methods to analyze the evidence from interviews, project materials and the mini￾survey. There are some noteworthy limitations on comparison, and hence attribution. First, recall bias is a common evaluation problem. MPs or parliamentary staff may have responded to questions by blending multiple experiences into a composite memory. For example, target committee staff may have received training on legislative research, oversight functions and incorporating expert testimony. In answering evaluation questions, they may have been unable to distinguish between these separate trainings. Additionally, because some trainings took place some time ago, respondents were not able to provide the level of detail needed for the evaluation. Second, response bias is also a common evaluation problem. For example, a former participant may have given an interviewer positive remarks about a study trip because she or he would like to go on another trip in the future. The evaluation team fully expected that parliamentary leaders, MPs and staff may understand that a negative evaluation could mean the end of a project that has provided needed training and other benefits. Third, selection bias, in the form of contacts provided by the implementers, can mean that the evaluation team may have only heard from people with positive experiences. Finally, PRODIP‘s practice of building the District Public Policy Forums (DPPFs) on the prior Citizen Alliance for Promotion of Transparency and Accountability (CAPTA) committees and the ready availability of constituency MPs may have obscured attribution for key project activities. An effective approach used by the evaluation team to combat most of these forms of bias was to triangulate multiple sources of data, as is often accomplished through qualitative reliability matrices. The team combined information found in documents and interviews from multiple sources so that any one piece of biased data would not skew the analysis. The team also interviewed key informants who did not benefit directly from PRODIP and asked interviewees to provide specific examples of knowledge use. Social Impact‘s quality assurance process provided another review of the evaluation team‘s evidence and analysis. Nonetheless, the team was unable to overcome attribution problems for some aspects of DPPF and MP outreach activities. In addition, the team had limited evidence with which to evaluate a few of the project‘s activities. FINDINGS AND CONCLUSIONS Objective 1: An Effective and Open Parliament For Result 1, the evaluation team concluded that PRODIP training and study tours have produced a broad but shallow improvement in the lawmaking and oversight capacity of parliamentary personnel. Stakeholders in targeted committees and related Secretariat units are generally satisfied with PRODIP‘s assistance, but they view the supply of training as inadequate viii Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project and not addressing different skill levels. In addition, the training was not always aimed at the most appropriate people. Moreover, improvement is vulnerable to turnover in MPs, Private Secretaries (PSs) and non-permanent staff. The All Party Parliamentary Groups (APPGs), particularly the Women‘s Caucus, have been an appreciated, active, and successful activity facilitated by PRODIP. Limited parliamentary budgets, equipment and infrastructure affect committee activity, committee staff and APPGs alike, so much of the progress achieved under Result 1would be reversed in the absence of continued PRODIP activities. For Result 2, the evaluation team concluded that PRODIP assistance has produced a broad but shallow improvement in committees‘ capacity function. As with Result 1, there is general satisfaction with PRODIP‘s efforts among targeted committees and related Secretariat units, but the amount and content of training was not viewed as appropriate to demand. As with Result 1, this improvement is vulnerable to turnover by MPs, PSs and non-permanent staff. The ability of committees to act as a vital part of the legislative branch of government is limited by the nature of the legislative system, gaps in the Rules of Procedure, and limited parliamentary budgets, equipment and infrastructure, so much of the progress achieved under Result 2 would be reversed without PRODIP. For Result 3, the evaluation team concluded that the only real progress came from the public hearings activity, a successful but small set of events. Although parliamentary reports spurred ministry action, in many cases committees and APPGs had limited ability to follow-up or compel a response. MPs valued the public hearings and PRODIP‘s assistance in facilitating them. The MPs are interested in continuing this practice but cannot without PRODIP. Only one MP knew of the existence of the DPPFs, even among the subset of MPs we interviewed who had attended a DPPF event. The team‘s overall conclusion for Objective 1 was that targeted parts of Parliament have become slightly more open and effective in a number of ways, for example regarding committee procedures, legislative drafting, reporting and public hearings. In addition, PRODIP can justly claim credit for promoting skills development, APPGs, the Women‘s Caucus and public hearings. The gender inclusion element of this objective is noteworthy, as evidenced in the success of the Washington study tour, the ongoing Women‘s Caucus, and the public hearings on issues such as reproductive health and how climate change affects women. The progress PRODIP has made with stakeholders in Parliament is valued by those stakeholders, yet PRODIP activities are viewed as being not entirely coherent, not ambitious enough and not always aimed at appropriate participants. All of the positive developments, moreover, are quite vulnerable to turnover among MPs, PSs and non-permanent staff, and much of the developed capacity may not continue in the next Parliament. These activities would also likely not continue without PRODIP‘s sustained involvement and funding. Additionally, the connection between the two objectives of PRODIP is almost non-existent, as if they were inhabiting separate worlds. Objective 2: A More Constructive and Sustainable Role for Civil Society For Result 4, the team concluded that PRODIP has achieved important foundational results during the project period. In a relatively abbreviated time period (10-17 months), PRODIP grantees and PRODIP-supported DPPFs advanced public participation in the policymaking process and demonstrated initial progress toward potential policy successes. DPPF members Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project ix have provided formal input to legislation including the draft Forest Act and the Small Anthropological Groups Act. Advocacy issues including irrigation and intercity train service have been raised in Parliament as an outcome of public forums conducted with PRODIP support. Policymakers have expressed commitment to support action at the local level, such as blocking the establishment of a coal-fired power plant and devoting marketplace space to female entrepreneurs. In time, and with increased outreach to non-constituency MPs, PRODIP support for targeted advocacy efforts could lead to policy change at the national level. For PRODIP grantees with extensive advocacy experience in diverse issue areas, formal communication with MPs represents a new dimension in citizen engagement in policymaking. While some Civil Society Organization (CSOs) are conducting evidence-based research to support parliamentary oversight and direct democracy activities, PRODIP‘s technical assistance does not currently include training in this area. Grantees track outputs against monitoring and evaluation plans provided by TAF, but there is no evidence to suggest that knowledge gained through monitoring is used to inform grantees‘ project management. CSOs value the strategic oversight and constructive recommendations provided by TAF, and future technical support should continue to include internal governance and strategic planning with a focus on sustainability. In addition, CSOs should seek increased opportunities to discuss solutions to common challenges and to plan collaborative activities. For Result 5, the team concluded that PRODIP has made significant progress in this area during the project period. PRODIP partners regularly provide inputs to policy discussions, both through politically-unbiased research and through formal dialogue between DPPF members and constituent MPs. Recipients of TAF research grants envision their research being used to inform dialogue and support citizen participation in the policymaking process. Researchers interact with MPs, parliamentary standing committees and other PRODIP CSO grantees, but not necessarily with DPPF members. Steps could be taken to increase the use of policy research by relevant PRODIP grantees and to better coordinate among all parties. If implemented as planned, parliamentary watch activities conducted by the Institute of Informatics and Development (D.Net/IID) could serve as a unifying force and mechanism for information-sharing among PRODIP grantees and help to advance advocacy goals on the national stage. PRODIP democratic governance grantees mobilized the existing capacity of active citizen groups across Bangladesh to establish 140 public policy forums at the district, upazila and union levels. Gender inclusion was prioritized in the formation of DPPFs: One third of members are women, some women actively serve in leadership roles, and female DPPF members do not feel dominated by male colleagues in DPPF settings. With oversight and guidance from CSOs, DPPFs advocate for policy changes and monitor the implementation of laws at the local level. Yet, while MPs regularly attend DPPF events, familiarity with ―DPPFs‖ as formal public policy advocacy groups is almost non-existent among committee chairs, MPs and personal secretaries. With improved branding, dialogue with non-constituent MPs, continued citizen commitment, and internal capacity-building, DPPFs could prove to be a sustainable model for public policy advocacy at the national level. For Result 6, the team concluded that as of the mid-term of the project period, PRODIP is underperforming in this area. Thus far, PRODIP has yet to provide CSOs and the media with tools to educate the public on democratic processes and practices. All of PRODIP‘s proposed x Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project activities under this result have been initiated recently – most of them near or during the third year of the project. The PRODIP website remains in the preliminary stages of development, several public service announcements (PSAs) of uncertain effect have been produced and aired, and two grants have been awarded to build the capacity of the media. Overall, it is too early to assess the impact of the limited activities conducted by PRODIP under this result. For the evaluation aspects dealing with process issues, the team concluded that: B. Management & Administration: Significant management and coordination problems exist at all levels: within the program, with the program‘s stakeholders, between programs, with donors‘ stakeholders and between donors. D. Program Opportunities: A parliamentary program under the direction of USAID was and is the best option for US foreign assistance. E. Program Theory and Design: The PRODIP design incorporates Section 5.3.2 of the 2009 DG Sectoral Assessment well, and goes beyond it to add the civil society objective. RECOMMENDATIONS AND LESSONS LEARNED The team makes several result-level and program-level recommendations. Results Level Objective 1: An Effective and Open Parliament o Result 1  Expand the internship program to provide valued and effective resources to Parliament, in particular increasing the number of law interns and women interns that are both in very short supply.  Re-configure the training structure for MPs, seconded staff and permanent staff in Parliament to better target each type with more appropriate training. o Result 2  Devote more resources to expanding the activities of APPGs, which have proven to be popular and meaningful venues for policy discussions.  Eliminate support for party caucuses to avoid the appearance of partisanship. o Result 3  Reduce the number of activities in order to devote more resources to public hearings. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project xi Objective 2: A More Constructive and Sustainable Role for Civil Society o Result 4  Facilitate stronger communication and information sharing among grantees to increase the flow of advocacy innovations and progress.  Reconsider the format of periodic meetings to allow for synergistic discussions. o Result 5  Expand assistance to DPPFs to address sustainability issues.  Help DPPFs develop a stronger and more recognizable identity to improve their visibility and sense of purpose. o Result 6  Develop the PRODIP website to meet the needs of DPPFs, research partners and even parliamentary actors. Program-Level The two PRODIP objectives are largely stand-alone efforts. In the next few months, USAID should require PRODIP to develop a strategy and detailed work plan that specifically addresses collaboration. The strategy should identify specific activities in the two objectives that provide opportunities to combine the strengths of both in the service of the overall purpose of the program. The detailed work plan should then address specifically what will be done, by which specific PRODIP personnel, in a specific scheduled period of time, with a budget specific to each objective. If USAID is not satisfied with the strategy and work plan, it should consider separating the objectives into two stand-alone programs. Lessons Learned The evaluation team also draws USAID‘s attention to some lessons learned from this program‘s implementation that should inform future parliamentary strengthening efforts:  Parliamentary strengthening programs are by nature quite demand-driven, which places a premium on prior consultation with parliamentary and party leaders.  Increased parliamentary ownership and consultation reduces the volatility of program activities.  Strategic plans developed by Parliament should set the context for program activities in order to maximize stakeholder satisfaction and minimize later program design changes.  Implementers must coordinate with each other to plan new MP orientation after elections in order to avoid taxing the patience and energy of Parliament leaders and MPs.  Parliamentary programs must differentiate the needs of various parliamentary actors.  Parliamentary programs should bring parliamentary leaders into the design logic and progress of its activities. xii Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project  The executive-legislative relationship in Westminster systems places severe limits on the feasibility of some typical parliamentary strengthening activities, which has important implications for expectations of reform.  Public policy advocacy programs are distinct from traditional advocacy programs and, accordingly, should incorporate training and capacity building on related topics in order to better meet the needs of partners.  Public policy advocacy initiatives that seek results at the national level must be primarily issue-based rather than constituency-based in order to be more effective.  Strategic planning and effective branding can contribute to longer-term sustainability of citizen groups.  Adherence to a gender-balanced membership policy can help position women to participate in citizen groups as equal partners in public policy dialogues. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 1 INTRODUCTION Democracy in Bangladesh has developed in fits and starts since independence in 1971 from the ―two-nation‖ Pakistan of 1947. It is a unitary state with a parliamentary system following the Westminster model. One of the most populous countries, it is also one of the poorest, with large parts of the country vulnerable to typhoons and rising sea levels. The political history of Bangladesh is largely a story of competition between the Awami League party, the Bangladesh Nationalist Party (BNP), and the military. The Awami League, a party that traces its history back to the early years of post-colonial history of the two-nation Pakistan, has been one of the mainstays of the political party system, along with the BNP formed in the late 1970s. Power has alternated between the Awami League, the BNP, and the military, including the military-related Jatiya Party formed in the 1980s. Following political pressure in 1990, the military-led government resigned and the first ―caretaker‖ government was established extra-constitutionally to ease the transition to democratic rule. The 1991 elections resulted in a narrow victory for the BNP over the Awami League, and BNP formed a coalition government along with the Islamic Jamaat-e-Islami party. However, growing political unrest fueled by accusations of corruption and abuse of state resources against the government led to a boycott of Parliament by the Awami League, and ultimately early elections in February 1996 under the auspices of the BNP-led coalition government. When those elections were won decisively by the BNP due to an election boycott by the Awami League, an explosion of political turmoil and violence forced another round of elections in June 1996, administered by a newly-constitutional caretaker government. The Awami League won the June elections, although the party needed the support of some smaller parties to form a government. For the remainder of the 1990s, the ruling Awami League and the BNP were at a political impasse, characterized by large political strikes, state harassment, and parliamentary and election boycotts by the BNP. The caretaker government appointed in 2000 held elections in October 2001 that were won overwhelmingly by the BNP. Growing political unrest was again fueled by accusations of corruption and abuse of state resources, this time against the BNP government. With regular elections scheduled to take place in October 2006, a crisis stemming from a deadlock over naming a caretaker was resolved when President Iajuddin Ahmed assumed the role. Unrelenting political unrest and military pressure forced President Ahmed to resign from the caretaker government in early 2007, leading to the appointment of Fakhruddin Ahmed, the governor of Bangladesh Bank. Ahmed‘s caretaker government declared emergency law and engaged in mass arrests in an attempt to root out corruption and halt political violence. One particularly controversial move was the forced removal of Awami League and BNP leaders from their parties. A reconstructed and credible voter list was used for the elections to the Ninth Parliament held in December 2008, which were won by the Awami League, although it needed to rely on some smaller parties to form a government. The Awami League-led government has remained in power since the December 2008 elections, and elections to the Tenth Parliament are expected at the end of 2013. One continuing political issue, however, is whether a caretaker government will be appointed in the run-up to the elections. The Awami League has so far refused to agree to naming a caretaker government because memories of the repression of the last caretaker 2 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project government are still fresh, and the BNP supports the idea of a caretaker government because of concerns about abuse of state resources by its bitter rival. The 2008 elections held out the promise of opportunities for political change. The rise of social organizations and watchdogs injected a fundamentally new element into political life in Bangladesh. Candidates from the major parties spoke of the need for Parliament to be a more significant actor, and many Members of Parliament (MPs) arrived for the Ninth Parliament ready to work on their commitments. The fusion of political will for reform among both MPs and civil society was new, and the Promoting Democratic Institutions and Practices (PRODIP) project was intended to take advantage of that. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 3 DEVELOPMENT PROBLEM AND USAID RESPONSE Bangladesh‘s democratic development has been stunted by ineffective institutions and rampant corruption instigated and sustained by overly-centralized governance structures and adversarial party politics. These factors hamper the ability of policymakers to meet citizen demands, and service delivery is insufficient to promote economic growth and political stability. Gender disparities and the exclusion of women from decision-making processes also limit the country‘s advancement in various sectors. Bangladesh‘s prosperity and stability are important factors for U.S. foreign policy interests, especially given that the country possesses the fourth largest Muslim population in the world and that nearly 80 percent of citizens are less than 40 years of age. More pluralistic and responsive governance will contribute to the USAID/Bangladesh Country Development Cooperation Strategy (CDCS) goal for ―Bangladesh to become a knowledge-based, healthy, food-secure, and climate-resilient middle income democracy.‖1 Transparency and accountability, especially related to economic opportunity and the distribution of social services, will further stabilize Bangladesh‘s democracy by enhancing citizen confidence in government. USAID‘s CDCS Development Objective 1, ―Citizen Confidence in Governance Institutions Increased‖ provides a framework for PRODIP‘s aim to increase the effectiveness and responsiveness of democratic institutions and processes in Bangladesh. PROJECT BACKGROUND In April 2010, USAID awarded a five-year cooperative agreement with a total estimated cost of $23,210,783 to The Asia Foundation (TAF) to implement a project that seeks to improve legislative functions and processes in Bangladesh. TAF and its partner, the State University of New York Center for International Development (SUNY/CID), finalized the design for PRODIP to build upon democratic development opportunities in Bangladesh presented by the country‘s successful transition to elected government in December 2008. PRODIP aims to increase the effectiveness and responsiveness of democratic institutions and processes in Bangladesh through two simultaneous and mutually-reinforcing objectives: Component 1 addresses the ―supply side‖ to develop ―An Effective and Open Parliament,‖ and Component 2 focuses on the ―demand side‖ to achieve a ―More Constructive and Sustainable Role for Civil Society in Democratic Governance.‖ As intermediate benchmarks to achieving these objectives, PRODIP works toward the following results:2 1. Improved parliamentary lawmaking and oversight capacity. 2. A stronger parliamentary committee system. 1 USAID/Bangladesh Country Development Cooperation Strategy (FY2011–FY2016), August 11, 2011. 2 PRODIP Monitoring and Evaluation Plan, January 2011 Revision. 4 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 3. Increased public awareness of and participation in parliamentary functions and processes. 4. Increased institutional capacity of CSOs to advance public participation in democratic governance. 5. A more active role for CSOs and research institutions in informing and monitoring public policy. 6. A more active role for CSOs and the media in educating the public on democratic processes and practices. TAF maintains overall responsibility for the implementation of PRODIP and leads civil society strengthening interventions under Component 2. SUNY/CID leads the implementation of parliamentary activities under Component 1. Democratic development programs that seek to improve parliamentary institutions and strengthen the role of civil society often share a common development hypothesis. Effective legislative functioning and constructive civil society interventions follow from a number of conditions including: adoption and institutionalization of best practices for parliamentary procedures; meaningful oversight of executive branch activities by parliamentary factions; sustained efforts by civil society to monitor and promote political transparency; integration of citizen input and evidence-based knowledge into advocacy and legislative activities; and responsiveness of elected officials to citizen demands through information-sharing and public dialogue. With these cornerstones in place, legislative institutions and civil society can help to foster citizen confidence in government. PRODIP‘s development hypothesis is that increased citizen engagement in the political process and simultaneous movement toward a more democratic, inclusive and deliberative Parliament will increase citizen confidence in the Government of Bangladesh. The most critical assumption underlying this hypothesis is that the Jatya Sangsad will remain receptive and supportive of PRODIP and its objectives. Other assumptions are that the norms of behavior promoted by PRODIP activities will be adopted and sustained by MPs and CSOs, civil society actors will be able to communicate and collaborate freely and regularly with the Jatya Sangsad, and the operating environment for CSOs will continue to facilitate political activism and engagement. In the wake of the 2008 elections, donors rushed in to design political reform programs. Due to a lack of coordination among donors, multiple programs aimed at Parliament were established, most of them without prior extensive consultation with parliamentary leadership. USAID is not the only donor funding a parliamentary strengthening project in Bangladesh. DFID, through its Strengthening Political Participation program, co-funds with USAID both the civil society objective of PRODIP and the Democratic Participation and Reform project being implemented by Democracy International. DFID also funds a Westminster Foundation for Democracy project to begin in 2013 and, with other donors, co-funds the Strengthening Parliamentary Oversight (SPO) project run by the World Bank. The SPO project component that works with executive branch institutions is also known as SPEMP (Strengthening Public Expenditure Management). In addition, the Netherlands funds the Improving Democracy through Parliamentary Development (IPD) run by the United National Development Programme (UNDP). IPD‘s goals are similar to those of PRODIP: to improve the capacity of committees and parliamentary leadership. IPD‘s approach, however, differs in that UNDP has offices in Parliament, focuses on Secretariat and committee leadership only and is by design responsive to Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 5 the momentary needs and priorities of stakeholders in Parliament. Despite a lack of coordination in the beginning of both programs, within a short period of time the two programs divided up standing committees in Parliament in order to avoid duplication. Smaller programs include IT and infrastructure projects by Japan International Cooperation Agency and Korea International Cooperation Agency, a United Nations Population Fund (UNFPA) activity to support policy development in Parliament on population management and development, and a UN Women‘s project to work with women MPs. Objective 1: An Effective and Open Parliament Primary beneficiaries of PRODIP‘s activities under Objective 1 include: male and female MPs, professional parliamentary staff, the Parliamentary Secretariat, the parliamentary library, university students, parliamentary party caucuses, and chairs and MPs serving in parliamentary standing committees that oversee the 12 ministries listed in Table 1 below: TABLE 1: PARLIAMENTARY STANDING COMMITTEES TARGETED BY PRODIP ACTIVITIES 1. Health and Family Planning 2. Planning 3. Social Welfare 4. Finance 5. Local Government, Rural Development, Cooperatives 6. Housing and Public Works 7. Food and Disaster Management 8. Labor and Employment 9. Agriculture 10. Power, Energy, and Mineral Resources 11. Fisheries and Livestock 12. Commerce The Education Committee requested to join the PRODIP project in early 2012, thus becoming the 13th committee targeted for activities. Result 1 (Improved parliamentary lawmaking skills and oversight capacity) activities include: colloquia on legislative research and bill drafting; research and report training for parliamentary staff; support for policy research; development of research and documentation services in the Jatya Sangsad; parliamentary internship program; establishment of party caucuses on issues of national interest; engagement with external advisory groups; policy dialogue on matters of national interest; participation of women in lawmaking and policy formation; and orientation for new members of the Tenth Parliament anticipated for 2014.3 Result 2 (Stronger parliamentary committee system) activities include: assistance to committee clusters; development of committee work plans; improvement of committee work processes; engagement of committees in oversight functions; study tour to Canada for parliamentary leadership;4 and production of committee activity briefs. 3 A study tour for parliamentary leadership to the Parliament of Canada originally proposed under Result 1 was shifted to Result 2. 4 India was originally proposed as the destination for the study tour. 6 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Result 3 (Increased public awareness of and participation in parliamentary functions and processes) activities include: communications strategy development; production of a toolkit to assist MPs‘ work with the media; provision of expert testimony; support for public hearings; engagement of youth in Parliament; town hall meetings to link committees and community leaders; and information communication technology (ICT) support. Objective 2: A More Constructive and Sustainable Role for Civil Society in Democratic Governance Primary beneficiaries of PRODIP‘s activities under Objective 2 include CSOs, as well as male and female members of citizen groups in districts, upazilas and unions. Result 4 (Increased institutional capacity of CSOs to advance public participation in democratic governance) activities include: democratic governance grants, technical support and institutional development for CSOs; coordination meetings for CSO recipients of democratic governance grants; and regional study and exchange programs. Result 5 (More active role for CSOs and research institutions in informing and monitoring public policy) activities include: support for parliamentary policy research and public advocacy initiatives; development of parliamentary watch programs; direct democracy via public forums; and promotion of women‘s leadership in direct democracy activities. Result 6 (More active role for CSOs and the media in educating the public on democratic processes and practices) activities include: constituent perception surveys and assessments; development of an interactive PRODIP website; support for national information campaigns; support for national media outreach; and production of public service announcements (PSAs).5 PRODIP experienced serious delays in implementation for the first year or more of the project. Parliamentary programs, as with rule of law and party assistance programs, are highly demand￾driven and require significant relationship development. For programs that deal with state institutions, relationship development should begin with the donor, resulting in a scope of work that has the approval of the host institution before the project is awarded to an implementer. If this has not happened, as was the case with PRODIP, then programs will be slow to start up because one or more ―gatekeeper‖ at the host institution may withhold approval for long periods of time. That appears to be one reason for PRODIP‘s delay. Another reason was the slow approach taken by the original chief of party, who focused on assessing needs and establishing working relationships with MPs, the Parliamentary Secretariat and other international development partners working in the Parliament. For these reasons, PRODIP had very little activity during the first year and during much of the second. Approval was gained during the seventh quarter of PRODIP‘s cooperative agreement, on December 7, 2011, at the inaugural meeting of a Steering Committee established by the Speaker of Parliament to advise and assist PRODIP. PRODIP activities intensified toward the end of the period covered by this mid-term evaluation. 5 Value-added through the USAID Leaders of Influence (LOI) program was ultimately omitted from PRODIP‘s approved work plan. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 7 The original PRODIP work plan also evolved over time through revisions, deletions, and additions in accordance with political realities and the desires of Parliament. Again, this stems to some extent from the lack of a priori consultation with parliamentary leadership. In May 2012, USAID approved a contract modification to include three new activities under Objective 1: Activity 1.12 (Provide Financial Analysis Services to Parliament); Activity 3.8 (Implement a Constituency Service Offices Pilot Program); and the Asia Regional Women Parliamentarians‘ Conference under Activity 1.10 (Increase Women‘s Participation in Lawmaking and Policy Formation). This report notes adjustments to the work plan at the end of each Result discussion. 8 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE This mid-term performance evaluation, which covers PRODIP project activities conducted from April 2010 to January 2013, has four purposes that incorporate both performance and process aspects: 1. Evaluate PRODIP‘s overall performance by assessing results against stated targets and indicators; 2. Assess the efficacy and results of the PRODIP implementation approaches and management structure in meeting the objectives; 3. Make recommendations to USAID/Bangladesh concerning possible programming changes or adjustments to the second half of PRODIP‘s implementation; and 4. Analyze interventions under Objective 2 (Civil Society) to determine if they are supporting the cooperative agreement intent to strengthen legislative function and processes. The audience of this evaluation is USAID/Bangladesh, USAID/Washington, leaders of USAID Forward, TAF, SUNY/CID, DFID, United Nations Development Programme (UNDP) and other bilateral and multilateral donors to Bangladesh. USAID/Bangladesh plans to use the results of this evaluation to review the feasibility of the PRODIP work plan and consider possible programming changes or adjustments. EVALUATION QUESTIONS The evaluation questions are grouped into seven areas: A. Results: To what extent has PRODIP been successful in achieving the program objectives? How well did the activities of each of PRODIP‘s two components contribute, in a coordinated way, to building a stronger parliamentary institution? B. Management & Administration: To what extent has PRODIP‘s performance management system provided useful data to support management decisions? To what extent have PRODIP management and stakeholders incorporated knowledge gained through PRODIP‘s monitoring and evaluation (M&E) systems into project management? How effectively have TAF and SUNY/CID worked together in implementing PRODIP and achieving programmatic success? To what extent did PRODIP coordinate with other legislative strengthening programs? C. Client Satisfaction: What are the levels and areas of project stakeholder satisfaction or dissatisfaction with PRODIP‘s cooperation and performance? This group includes: leaders and staffs of Parliament, the Office of Speaker of the Parliament, and Parliamentary Secretariat; civil society and citizens‘ organizations; USAID and other implementers of legislative programs. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 9 D. Program Opportunities: To what degree is PRODIP‘s approach a cost-effective means to increase citizen confidence in governance institutions? E. Program Theory & Design: How well did USAID incorporate Section 5.3.2 of the 2009 Democracy and Governance (DG) Sectoral Assessment into the PRODIP design? F. Sustainability: How sustainable are PRODIP activities beyond USAID support? G. Cross-Cutting Issues: How well were gender issues addressed by PRODIP? For this evaluation, the analysis of project performance will combine areas A, C, F and G, while the analysis of project process will combine areas B, D and E. 10 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project EVALUATION METHODS AND LIMITATIONS To address the evaluation questions, this evaluation examines a mixture of performance and process aspects. Evaluation questions on results, client satisfaction, sustainability and cross￾cutting issues address performance aspects, while questions on management and administration, program opportunities, and program theory and design address the process or design aspects. The SI evaluation team was composed of Dr. Andrew Green, Senior Evaluation Specialist and Team Leader; Julia Rizvi, Evaluation Specialist; Bashir Ahmed, Parliamentary Sector Specialist; and Naim Mostofa, Local Evaluation Specialist. Dr. Green and Mr. Ahmed took the lead on the first PRODIP objective, and Ms. Rizvi and Mr. Mostofa took the lead on the second. Social Impact (SI) strongly believes in the use of mixed methods approaches to answer evaluation questions, a view that is reiterated in USAID‘s Evaluation Policy. Therefore, the evaluation team applied standard rapid appraisal techniques including semi-structured interviews (SSIs) of key informants, group interviews, a materials review and a mini-survey, gathering both quantitative and qualitative evidence. All data collection tools were developed and finalized in coordination with USAID/Bangladesh, and all qualitative and quantitative data were thoroughly coded and analyzed to ensure fluid identification of trends and outcomes. The SSI protocols were finalized by the evaluation team after in-depth discussions with PRODIP staff. Each SSI differed depending on the key informant‘s role and ―causal distance‖ from activities, as well as the extent of the key informant‘s involvement in PRODIP activities and the time available for interviewing. Illustrative questions are in Annex IV. The questions do not only address knowledge and general perceptions; more importantly, they probe for specific examples of attitude and behavior change. Group interviews were conducted with the same protocols with parliamentary interns and DPPF members. The evaluation team conducted one mini-survey of District Public Policy Forum (DPPF) leaders by phone, with protocols based closely on the related SSI questions. The evaluation team conducted site visits to CSOs and DPPFs in two divisions, but the sheer number and the geographic distribution of both precluded a more extensive effort. The value of mini-surveys is precisely to gather evidence from otherwise unreachable key informants. Parallel analysis was used to analyze the evidence from SSIs, the materials review and the mini￾survey. In this analytical approach, each type of data for an activity was analyzed in parallel and then across data type. For example, the evaluation team first analyzed SSIs with DPPF members to develop preliminary findings about the interaction of elected officials and DPPFs; second, the team analyzed the data from the DPPF leader mini-survey questions on this activity to develop preliminary findings; third, the team analyzed relevant materials to develop preliminary findings; and finally, the team analyzed preliminary findings across the types of data to develop activity￾level findings. The evaluation of PRODIP started with the review of important documents prior to beginning fieldwork. Expatriate team members Andrew Green and Julia Rizvi arrived in Dhaka, Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 11 Bangladesh, on January 5. On January 6, they conducted a Team Planning Meeting with local team members at the Bangladesh Democracy and Governance Program Evaluations (BDGPE) Dhaka office and participated in an in-brief with USAID/Bangladesh staff. Following these meetings, the evaluation team began data collection, which included the use of SSIs with key informants, group interviews, materials review, and the mini-survey. The relevant protocols and planned mix of evidence sources are detailed in the annexes. Data collection took approximately 14 business days and was followed by data analysis, report outline preparation and production of a draft presentation to USAID/Bangladesh. The team conducted several site visits: (1) Jahangirnagar University in Savar on January 8 to observe the dissemination of education policy research conducted by the Department of Anthropology, a PRODIP democratic governance research grantee; (2) a National Seminar on Maternal Health conducted by a PRODIP-supported DPPF in Dhaka on January 12; (3) PRODIP‘s CSO Coordination Meeting in Dhaka on January 15-16; (4) a DPPF Women‘s Leadership Training in Chittagong on January 18; and (5) a Sharing Meeting on Policy Briefs and Advocacy Issues conducted by a PRODIP-supported DPPF in Dhaka on January 21. The evaluation team presented its preliminary findings and conclusions to USAID, DFID and PRODIP at debriefing meetings on January 27 and 28. There are some noteworthy limitations on comparison, and hence attribution. First, recall bias is a common evaluation problem. MPs or parliamentary staff may have responded to evaluator questions by blending multiple experiences into a composite memory. For example, target committee staff may have received separate training on legislative research, oversight functions and incorporating expert testimony. In answering evaluation questions, they may have been unable to distinguish between these separate trainings. Additionally, because some trainings took place some time ago, respondents were not able to provide the level of detail needed for the evaluation. Second, response bias is a common problem for evaluations. For example, a former participant may have given an interviewer positive remarks about a study trip because she or he would like to go on another trip in the future. The evaluation team fully expected that parliamentary leaders, MPs and staff may understand that a negative evaluation could mean the end of a project that has provided needed training and other benefits. Third, selection bias, in the form of contacts provided by the implementers, can mean that the evaluation team may have only heard from people with positive experiences. Finally, PRODIP‘s practice of building the District Public Policy Forums (DPPFs) on the prior Citizen Alliance for Promotion of Transparency and Accountability (CAPTA) committees and the ready availability of constituency MPs may have obscured attribution for key project activities. An effective approach used by the evaluation team to combat most of these forms of bias was to use multiple sources of data to triangulate on evaluation issues, as is often accomplished through qualitative reliability matrices. The team combined information found in documents or interviews from multiple sources so that any one piece of biased data would not skew the analysis. The team also interviewed key informants who did not benefit directly from PRODIP and asked interviewees to provide specific examples of knowledge use. Social Impact‘s quality assurance process provided another review of the evaluation team‘s evidence and analysis. Nonetheless, the team was unable to overcome attribution problems for some aspects of DPPF and MP outreach activities. In addition, the team had limited evidence with which to evaluate a few of the project‘s activities. 12 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project FINDINGS AND CONCLUSIONS FINDINGS FOR OBJECTIVE 1: AN EFFECTIVE AND OPEN PARLIAMENT PRODIP has made identifiable contributions toward a more effective and open Parliament. Training promoted skills development, the project facilitated all-party groups, and cooperation with committee chairs led to public hearings. Stakeholders value these results, despite perceiving a lack of coherence and scope among PRODIP activities. However, the gains that have been made are vulnerable to turnover among MPs, Private Secretaries (PSs) and non￾permanent staff, and likely would not continue without continued PRODIP funding and engagement. Furthermore, the operational connection between Objective 1 and Objective 2 activities appears to be weak and incidental. Findings for Result 1: Improved Parliamentary Lawmaking Skills & Oversight Capacity The evaluation team found many positive aspects of the activities under this Result, but notes that the ongoing effect of the program is limited by the supply of training and threatened by the turnover of parliamentary personnel. The lack of human and financial resources in Parliament is a serious constraint to sustainability. Activity 1.1: Facilitate seminars on legislative research and bill drafting MPs that are chairs or members of PRODIP‘s targeted committees were offered research seminars on specific topics and training on bill drafting, with the former comprising the bulk of PRODIP‘s efforts. Seminars were held on agriculture, food security and domestic violence, while legislative drafting training was held in May 2012. The evaluation team found that while all committee chair and member MPs we interviewed thought the seminars and training valuable, it was not unusual for the MPs to have their own network of contacts in their policy issue areas. This was particularly true for the Agriculture Committee, which had a chair and PS with deep expertise in the issue. Moreover, we found that although two MPs we spoke to had submitted a Private Member‘s Bill, none of the MPs had time to do their own drafting and would rely on committee staff instead. Activity 1.2: Provide staff training on research, report-writing and drafting The reality that committee staff do most of the work on a committee raises the importance of this activity. Training on legislative research and policy analysis was offered to PSs and staff of the targeted committees in January 2012, with additional trainings on legislative drafting offered to committee staff in April and December 2012. We found that the committee staff valued the training and could point to Private Member‘s Bills as examples of how the drafting training had been used in practice, but staff felt that they could never have as much expertise or research knowledge as the ministries, particularly given the number of staff and assignments to multiple committees. In a Westminster system, the ministries typically draft legislation, and committees scrutinize and revise the bills. One key stakeholder, the Secretariat, noted improvements in the quality of Private Member‘s Bills and committee reports. The team found that none of the committee staff have a law background. The permanent staff from the committee support and legislative support wings complained that too many of the seconded staff were getting the training. However, the team found that that was not true (see Table 2 below) and that the Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 13 Secretariat had adopted the practice of spreading training opportunities across as many staff as possible. PRODIP also collaborated with The British Council to provide English training for permanent staff. In general, the demand for training on research, report-writing and drafting is greater than the supply, and past participants requested additional training in advanced topics. TABLE 2: DISTRIBUTION OF TRAINING (WOMEN) Status Writing Cmte Rpts Public Hearings Bill drafting English Comm’ctns Research Rules of Procedure Total 104 (20) 30 (5) 31 (6) 7 (-) 48 (10) 17 (4) 16 (3) 8 (2) Private Secretaries 7 (-) - (-) - (-) 7 (-) - (-) - (-) - (-) - (-) Seconded Personnel 20 (3) 6 (1) 17 (3) - (-) 7 (-) 3 (-) 1 (-) - (-) Permanent Staff 77 (17) 24 (4) 14 (3) - (-) 41 (10) 14 (4) 15 (3) 8 (2) Note: Rows will not add up to the Status figures, as some trainings involved multiple topics Activity 1.3: Support policy research PRODIP supported policy research for use by targeted committees after assessing committee needs. Activity 1.3 included partners under Objective 2 and was intended to complement public hearings (Activity 3.4, below). Chairs and member MPs cited PRODIP‘s assistance with experts and information for public hearings on climate change, agricultural mechanization and school management committees. Only one policy brief, on women‘s roles in Parliament, was produced by an Objective 2 partner, and the team found no knowledge among MPs, PSs and staff that CSOs and think tanks supported by PRODIP would be producing policy briefs. Activity 1.4: Establish research and documentation services in Parliament After a needs assessment, PRODIP provided training to library and documentation staff in February, April, July and October 2012 on a variety of topics: the role of legislative staff, the functioning of library and research services in parliaments, research techniques, rules of procedure, the legal context of Parliament in Bangladesh and the role of committees. Many of these staff were also in English training, and committee staff also participated in trainings. The team found that the library and documentation staff valued the research training, have implemented small research projects through PRODIP activities and were motivated to conduct additional research projects. Committee and Legislative Support wing staff reported better capacity in the Library and Documentation unit. Participating committee staff have also produced their own small research projects and reported that they would be able to do so again on their own. Activity 1.5: Parliamentary Internship Program As a means to supplement the capacity of committees and the Secretariat, PRODIP recruited and placed interns in Parliament. The team found that interns valued the experience, applied their academic knowledge to work and would all like to work in Parliament in the future. All 14 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project but one of the MP and PS interviewees valued interns and could give examples of their contributions to committee work; the sole negative response was a chair who wanted an intern that was a true policy expert for his committee‘s area. The team found that law interns were especially valued: Committee chairs and staff pointedly requested more interns with law backgrounds. The team noted that the set of interns included women in a professional context where few women work. Activity 1.6: Parliamentary caucuses us on policy issues In an effort to promote better policy on national issues, PRODIP facilitated the creation of party caucuses, ultimately helping to establish the BNP and Jatiya Party caucuses. The team found that caucus leaders acknowledged PRODIP‘s effective role in forming their caucuses. The Jatiya Party caucus conducted a public hearing with PRODIP‘s assistance on population management issues, but the BNP caucus merely discussed the party‘s political priorities in advance of the election. PRODIP also facilitated the creation of policy issue caucuses across party lines in order to promote better policy, building on the National Democratic Institute‘s (NDI) previous efforts with political parties. Established All Party Parliamentary Group (APPGs) include the Women‘s, Food Security, and Population Management and Development caucuses. All three of these caucuses conducted field investigations and public hearings, efforts that the MPs valued for the public feedback and attention they received. The team felt that the MPs were particularly stimulated by the APPG activities, though this is true of public hearings more generally. The team found that APPGs have no resources other than what PRODIP provides, including parliamentary staff, meeting space or office supplies. Activity 1.10: Increase women's participation in law and policy formation PRODIP also acted to strengthen the ability of women leaders to influence public policy meaningfully, going beyond the Women‘s Caucus (APPG). The key activity was a study tour of women MPs to Washington, DC, to meet with women in Congress and participate in a workshop by the Center for Development and Population Activities (CEDPA). Stimulated by the experience of how women in Congress cooperate across party lines on issues of mutual interest and using the leadership and organizing skills learned at CEDPA, the study tour participants designed action plans for how to work together collaboratively and began proposing policy solutions based on research, for example gender budgeting. The Women‘s Caucus was a direct result of this study tour, as were, for example, public hearings on women and climate change. The team found that the action plan had not been fully implemented, though, in large part because of a lack of caucus resources as noted above. Another major effort under this activity was the four-day Regional Women Parliamentarians Conference of 73 women MPs from Afghanistan, Bangladesh, India, Nepal, Pakistan and Sri Lanka, intended to promote networking, lesson-sharing and collaboration across borders. No interviewee mentioned this conference, although the team did not ask about it specifically. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 15 Other Activities not listed above As with any project, activities on original work plans are revised or removed as circumstances warrant. Under Result 1, two activities (1.7, 1.9) were integrated with Activity 1.6, one activity (1.8) was integrated with Activity 2.5, one activity (1.12) was too new to be evaluated,6 and one activity (1.11) will be conducted with the Parliament to be elected by the end of 2013. Conclusions for Result 1 For Result 1, the evaluation team concluded that there has been a broad but shallow improvement in the lawmaking and oversight capacity of parliamentary personnel produced by PRODIP training and stimulated by study tours. Stakeholders in targeted committees and related Secretariat units are generally satisfied with PRODIP‘s assistance, but viewed the supply of training as inadequate and not addressing different skill levels. In addition, the training was not always aimed at the most appropriate people. Moreover, this improvement is vulnerable to turnover by MPs, PSs and non-permanent staff. The APPGs, particularly the Women‘s Caucus, have been an appreciated, active, and successful activity facilitated by PRODIP. Lack of parliamentary budgets, equipment and other infrastructure limits committee activity, committee staff, and APPGs. Therefore, much of this progress would be reversed in the absence of continued PRODIP activities. Findings for Result 2: Stronger Parliamentary Committee System As with Result 1, the evaluation team found many positive aspects of the activities under this Result, and again notes that the ongoing effect of the program is limited by the supply of training and threatened by the turnover of parliamentary personnel. The lack of human and financial resources in Parliament is a serious sustainability issue, and current aspects of the Westminster model as applied in Bangladesh limit the ability of PRODIP to promote change. Activity 2.1: Technical support for committees In January 2012, PRODIP brought in a U.S. expert on parliamentary strengthening, George Crawford, to conduct a needs assessment of its targeted committees. Based on that assessment and approval from the Secretariat, PRODIP offered training to PSs and committee staff on the role of committees, report writing, research techniques and agenda preparation. The team found that participants valued the training and have applied skills at work, for example in report writing and meeting agendas. Secretariat leaders noted that some participating committees were submitting reports, but others were not reporting minutes or resolutions. As with Result 1, training opportunities do not meet current demand. Permanent staff complained that seconded staff were prioritized, but that is in part not true and in part due to the Secretariat‘s deliberate policy. The team found that the number of staff from the training section under the Secretariat was too small to conduct much training of staff, though it was capable of taking training modules and adapting them as needed. The team also found that Chairs, MPs and PSs, on one hand, and committee staff, on the other, need slightly different training. For example, Chairs, MPs and PSs need training on rules of procedure, committee procedures, and agenda 6 The Budget Analysis and Monitoring Unit (BAMU) had been part of the PROGATI project until PROGATI closed in mid-2012. The PROGATI evaluation covers BAMU activities. 16 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project preparation, while committee staff need training on rules of procedure, report writing and research. Building on a needs assessment conducted by former UK MP Tony Worthington in mid-2011, PRODIP consultant George Crawford also conducted training on oversight strategies, actions and public hearings (formerly listed as Activity 2.3). Although MPs were energized by the concept of public hearings from the Canada study tour (see Activity 2.5, below), the team found that language obstacles and at times inappropriate analogies to the U.S. context limited the utility of the training for participants. Additionally, the reporting section of the public hearing manual is not being used due to human resource issues. The team also found that the current rules of procedure limit committee oversight action, largely because the nature of a Westminster system involves a tight connection between committees and ministries, with the former largely dependent on the latter for information and goodwill. PRODIP designed a Committee Effectiveness Index that looks at seven facets of committee functioning: meetings, research, reports, transparency, law-making, oversight and public outreach. For each targeted committee, PRODIP self-scores from 1 (worst) to 5 (best) based on identifiable benchmarks of behavior, with time points of January 2011 and September 2012. The best performing committees are (a) Agriculture and (b) Fisheries and Livestock with average scores of 3.4 out of 5, and (c) Education, (d) Power, and (e) Social Welfare round out the next tier at 3.1 out of 5. The largest average gain of 1.3 was by Social Welfare, and the average gain across the 13 committees was .5, with a September 2012 average score of 2.8 out of 5. Activity 2.2: Committee annual work plans In July through August 2012, consultant George Crawford and PRODIP staff conducted committee work planning training for select chairs, MPs, PSs and staff from most of its targeted committees, covering scheduling, strategic planning, prioritization, work plans, and calendars. The team found that participating chairs and MPs developed and were following work plans and that they would not have done so if not for PRODIP‘s assistance. According to chairs interviewed by the team, their committees were capable of creating or updating work plans, but have not done so due to the approaching end of the Ninth Parliament. Activity 2.4: Committee oversight capacity building Following the training given to MPs and staff on public hearings (see Activity 2.1, above), PRODIP facilitated field investigations and public hearings (see Activity 3.4, below) with select targeted committees. The Agriculture, Fisheries and Livestock, and Education Committees all held investigations and then submitted reports for action to appropriate ministries via the plenary. The team found that interviewed MPs valued the field investigation experience for helping them to develop a better understanding of key problems in their policy issue areas and to better serve constituents‘ needs. Activity 2.5: Study tour to Canada PRODIP organized a study tour to the Canadian Parliament in Ottawa in November 2011, where select MPs, PSs, and Secretariat staff learned about the practice of public hearings, committee procedures and the value of parliamentary research services. Tour participants drafted action plans for their committees that included public hearings and research activities. The team found that tour participants were strongly stimulated by the lessons of the trip, citing Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 17 public hearings, the Parliamentary Library and Private Member‘s Bills as good examples to be emulated in Bangladesh. Every tour participant interviewed by the team noted specifically how her or his engagement with public hearings was fueled by the trip (see Activity 3.4). Interviewees faulted PRODIP, however, for a lack of advance communication about agendas and activities. Other Activities not listed above Activity 2.3 was integrated with Activity 2.1, and Activity 2.6 has not yet produced results. Conclusions for Result 2 For Result 2, the evaluation team concluded PRODIP assistance has produced a broad but shallow improvement in the capacity of committees to function. As with Result 1, there is general satisfaction with PRODIP‘s efforts among targeted committees and related Secretariat units, but the amount and content of training was not viewed as appropriate to demand. However, as with Result 1, this improvement is vulnerable to turnover by MPs, PSs and non￾permanent staff. The ability of committees to act as a vital part of the legislative branch of government is limited by the nature of the legislative system, gaps in the Rules of Procedure, and parliamentary budgets, equipment and infrastructure, so much of this progress would be reversed without PRODIP. Findings for Result 3: Increased Public Awareness/Participation in Parliamentary Functions/Processes This Result has had little to show over the life of the project, with the single noteworthy exception of public hearings. Activity 3.1: Develop a Comprehensive Communications Strategy PRODIP collaboratively developed a strategy with the Secretariat and UNDP‘s Improving Democracy through Parliamentary Development (IPD) program shortly before the evaluation began, covering aspects such as providing Sangsad TV with content from C-SPAN and other similar services from around the world. No interviewee in the Secretariat or at IPD mentioned this strategy, although the team did not ask about it specifically. Activity 3.2: MP-media outreach toolkit PRODIP held a discussion in September 2012 with journalists covering the Parliament on how to improve the interaction between MPs and media. The team found that the attempt to establish a Parliamentary Journalists‘ Association was unsuccessful, and no other actions have been taken since the discussion. Activity 3.3: Expert testimony capacity building The original intention of this PRODIP activity shifted from the facilitation of needed expert and specialist testimony for targeted committees to the coaching of citizens for public hearings. Beginning in January 2012, PRODIP coached citizens for public hearings in Jessore, Comilla, Cox‘s Bazar and Chittagong. The team found that committee chairs, member MPs and PSs already have informal networks of experts based on personal contacts, but committees do not have the resources to handle logistics or expenses for experts to attend committee meetings or events. As part of Activity 1.2 above, PRODIP developed a national database of policy 18 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project experts, but it is not yet available on its website. The team did not find anyone in Parliament who was aware of it. Activity 3.4: Public hearings Working in collaboration with targeted committees, PRODIP facilitated public hearings on issues of national interest:  Fisheries and Livestock Committee – three hearings in Jessore on a government fisheries project;  Agriculture Committee – two hearings in Comilla on agricultural mechanization, attended by a total of 130 citizens, local officials and journalists;  Fisheries and Livestock Committee – three hearings in Cox‘s Bazar on climate change, attended by over 500 people;  Social Welfare Committee – one hearing in Chittagong on social benefits for widows, the elderly and disabled, attended by approximately 200 people;  Labor and Employment Committee – one hearing in Dhaka on unorganized laborers;  Education Committee – one hearing in Dhaka on school management committees; and  An education policy research dissemination event in Savar. All five of the targeted committees that had Canada study tour participants have now held a public hearing. Chairs and PSs also said that PRODIP‘s facilitation was absolutely vital, from materials to guidance to funding. Indeed, committees have neither the human nor financial resources to conduct public hearings, due both to the current rules of procedure and to the budget of Parliament. Participating MPs were able to recall from memory specific aspects of testimony, and could point to reports sent by their committees to ministries. While the public hearings led to letters of inquiry sent to ministries, the committees have no means to compel ministries to respond to their letters and reports. This activity is supposed to interact with Objective 2‘s DPPF activity, but the team found that only one MP interviewed knew of the DPPFs, including those who had attended DPPF events. Activity 3.5: Youth in Parliament The original intention of this activity was to hold ―youth fairs‖ as a means of connecting youth with policy interests to relevant MPs, but other international donors had plans for youth engagement with Parliament. Instead, PRODIP began collaborating with the Secretariat on a booklet about Parliament and parliamentary processes to be piloted in three secondary schools in Dhaka. The Secretariat only gave its approval to the booklet in January 2013. Activity 3.6: Link MPs to Community Leaders through DPPFs This activity was explicitly designed to link the two objectives of PRODIP by facilitating the interaction between MPs and DPPFs through DPPF events and MP-run public hearings. For more information on DPPF events, see the discussion under Objective 2 below; for more information on MP-DPPF interaction at public hearings, see the end of the discussion of Activity 3.4 above. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 19 Other Activities not listed above Activity 3.7 is nearly indistinguishable from Activity 3.1, and two activities (3.8, 3.10) are too new to evaluate (there is no Activity 3.9). Conclusions for Result 3 For Result 3, the evaluation team concluded that the only real progress came from the public hearings activity, a successful but small set of events. Although parliamentary reports spurred ministry action in many cases, committees and APPGs had limited ability to follow-up or compel a response. MPs valued the public hearings and PRODIP‘s assistance in facilitating them and are interested in continuing this practice; however, they cannot do so without PRODIP. Only one MP knew of the existence of the DPPFs, even among the subset of MPs we interviewed who had attended a DPPF event. Overall Conclusions for Objective 1: An Effective and Open Parliament Targeted parts of Parliament have become slightly more open and effective in a number of ways, for example, in training on committee procedures, legislative drafting, reporting and public hearings. In addition, PRODIP can justly claim credit for promoting skills development, APPGs, the Women‘s Caucus and public hearings. The gender inclusion element of this objective is noteworthy, as can be seen in success of the Washington study tour, the ongoing Women‘s Caucus and the public hearings dealing with issues such as reproductive health and how climate change affects women. The progress PRODIP has made with stakeholders in Parliament is valued by those stakeholders, yet the activities are viewed as being not entirely coherent, not ambitious enough, and not always aimed at appropriate participants. All of the positive developments, moreover, are quite vulnerable to turnover among MPs, PSs and non-permanent staff, to the extent that much of the developed capacity may not continue in the next Parliament. These activities would also likely not continue without PRODIP‘s continued involvement and funding. The connection between the two objectives of PRODIP is almost non-existent, as if they were inhabiting separate worlds. FINDINGS FOR OBJECTIVE 2: A MORE CONSTRUCTIVE AND SUSTAINABLE ROLE FOR CIVIL SOCIETY IN DEMOCRATIC GOVERNANCE Findings for Result 4: Increased institutional capacity of CSOs to advance public participation in democratic governance This Result has shown promising progress in a short period of time, particularly on policy issues of local importance. Grantees value formal communication with MPs, but lack knowledge of the parliamentary oversight process. Activity 4.1: Democratic governance grants to CSOs As of January 2013, TAF issued democratic governance grants to 18 Bangladeshi CSOs. Proposals were solicited in two rounds, and grantees interviewed by the evaluation team reported that TAF conducted a transparent and competitive bidding process. Twelve of the 18 grantees received funding to facilitate direct democracy through DPPFs (Result 5), and the 20 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project other grantees are conducting public policy advocacy or media-related activities (Result 6) to promote citizen participation in the policymaking process. Four of the 18 grantees focus exclusively on women‘s advancement in various sectors (politics, economics, health, education and women‘s rights), and a number of grantees address gender as a cross-cutting issue. Prior to entering into grant agreements with TAF, many CSOs interviewed had conducted advocacy for policy changes, budget allocations, service delivery and implementation of laws. Areas of focus ranged from land distribution, to public-private partnerships, to rights and services for marginalized populations including women, coastal peoples and indigenous groups. Grantees emphasized their long-term commitment to specific advocacy issues, highlighted the continuity between their organizational missions and PRODIP activities, and referenced the presence or possibility of other funders to demonstrate the sustainability of their work. Many grantees noted that their interest in policy issues and their capacity to facilitate public forums existed on an informal basis before their participation in PRODIP, but they reported that PRODIP helped to formalize that potential and translate it into action. Specifically, grantees cited direct engagement with MPs regarding policy issues and the use of evidence-based research in policy discussions as new advocacy approaches emphasized by PRODIP. Many CSOs maintained existing, mutually beneficial relationships with MPs prior to receiving PRODIP grants: the Bangladesh Enterprise Institute (BEI) has been establishing partnerships between MPs and the private sector to modernize regulatory mechanisms; the Bangladesh Alliance for Women Leadership (BDAWL) worked to build the capacity of female MPs in the Ninth Parliament; and D.Net staff are sometimes called upon by MPs to provide speechwriting assistance. Overall, though, grantees‘ past advocacy activities typically targeted ministers and parliamentary standing committees at the national level or upazila officials and other service providers at the local level. A number of grantees reported to the evaluation team that PRODIP‘s recommendation to include MPs in dialogue about policy changes at the national level introduces a new dimension to their advocacy activities. For example, Bangladesh Women Chamber of Commerce & Industry (BWCCI) had been lobbying the Minister of Finance for four years to allocate a portion of the national budget to support female entrepreneurs.7 With funding from PRODIP, BWCCI organized a roundtable with the minister and, for the first time, invited MPs from both political parties. BWCCI believes that the presence of MPs was critical in pressuring the minister to fulfill his promise, and USD $1 billion was successfully allocated for the Equity and Entrepreneurship Fund in June 2012. Women entrepreneurs did not previously seek out MPs as target audiences for their advocacy activities. BWCCI noted that, now, they realize that MPs can be strategic stakeholders in policy debates and should be held accountable to their campaign promises after winning electoral seats. 7 Past BWCCI activities were supported with funding from the Center for International Private Enterprise (CIPE). Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 21 Activity 4.2: Technical support for democratic governance grantees PRODIP planned to involve training and technical assistance for grantees in conducting evidence-based research to support their parliamentary oversight and direct democracy activities.8 Some CSOs are indeed involved in conducting policy research to reinforce their own or their DPPFs‘ advocacy efforts, but none of the CSOs interviewed highlighted any training provided by TAF on the topic. Meanwhile, TAF conducted training on advocacy and women‘s leadership for CSO grantees. Those grantees that manage DPPFs used curricula and a toolkit designed by the BRAC Institute of Governance Studies (IGS) to replicate the trainings for DPPF members. Grantees with extensive advocacy expertise or women‘s leadership activities noted that they would have benefited from more advanced training content and materials. For accountability and capacity-building purposes, TAF conducted M&E training for PRODIP grantees. CSOs interviewed by the evaluation team described various levels of in-house M&E capacity and dedicated staff. A number of grantees received M&E plans from TAF, and grantees‘ quarterly reports typically contain quantitative outputs and case studies to demonstrate qualitative results. Grantees made no references to using monitoring data to inform decisions related to program implementation. Activity 4.3: Support institutional development of democratic governance grantees The majority of PRODIP grantees perceive TAF as a partner that provides strategic oversight and constructive recommendations. Grantees expressed gratitude for TAF‘s input during project design and one-year reviews, support for national public forum events, and convening role during coordination meetings. Several well-established PRODIP grantees that are funded by multiple donors expressed that TAF approval of grantee work plans and activities can delay planning and implementation. Overall, apart from completing financial management training to ensure compliance with TAF policies and procedures, PRODIP grantees did not report significant institutional changes or organizational development as a result of TAF interventions. One exception is the case of BDAWL, which benefited tremendously from TAF‘s financial support. PRODIP funding enabled BDAWL to move into its own offices, which enhanced BDAWL‘s ability to operate on a neutral basis and to implement its activities with greater independence. Activity 4.4: Coordination and information sharing meetings PRODIP grantees noted that they regularly participate in TAF-led coordination meetings, which have been held on a quarterly basis beginning in the third year of the project period. As of January 2013, CSOs were not involved in leading or facilitating these meetings. However, grantees value the opportunity to establish connections with each other, better understand the overall aim of PRODIP, and share limitations encountered during program implementation. 8 See Program Description, Page 26. 22 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Some CSOs were familiar with activities conducted by their peers before becoming involved in PRODIP – several through the Election Working Group (EWG),9 others due to program synergies. For example, IPDS and ASUS collaborate to raise awareness on the national level about indigenous groups‘ exclusion from particular services. In addition, Shammunay led the publication of a joint research paper on agricultural issues with the Wave Foundation. Activity 4.5: Regional study and exchange programs TAF conducted a Regional Inbound Study Visit Program through which five civil society experts from India, Nepal and Pakistan traveled to Dhaka, Bangladesh, from April 22-26, 2012, to learn about PRODIP as an example of advocating for public policy and laws that address citizen priorities. Representatives from PRODIP grantees IED, D.Net, Bangladesh Nari Progati Sangha (BNPS), Bangladesh National Women Lawyer‘s Association (BNWLA), BDAWL, Democracywatch, Wave Foundation, Management and Resources Development Initiative (MRDI) and BWCCI were invited to participate in the study visit. Some CSOs recalled meeting study visit participants and hearing about their activities. None of the CSO representatives interviewed mentioned any networking or continued exchange of best practices between study visit participants and PRODIP grantees. Additionally, none of them were able to attest to the effectiveness of this activity. However, the evaluation team did not interview study visit participants from India, Nepal or Pakistan, largely due to the limited scope of this activity. Conclusions for Result 4 PRODIP has achieved important foundational results in this area during the project period. In a relatively abbreviated time period (10-17 months),10 PRODIP grantees and PRODIP-supported DPPFs advanced public participation in the policymaking process and demonstrated initial progress toward potential policy successes. DPPF members have provided formal input to legislation including the draft Forest Act and the Small Anthropological Groups Act. Advocacy issues including irrigation and intercity train service have been raised in Parliament as an outcome of public forums conducted with PRODIP support. Policymakers have expressed commitments to support action at the local level, such as blocking the establishment of a coal￾fired power plant or devoting marketplace space to female entrepreneurs. In time, and with increased outreach to non-constituency MPs, PRODIP support for targeted advocacy efforts could lead to policy change at the national level. For PRODIP grantees with extensive advocacy experience in diverse issue areas, formal communication with MPs represents a new dimension to citizen engagement in policymaking. While some CSOs are conducting evidence-based research to support parliamentary oversight and direct democracy activities, PRODIP‘s technical assistance does not currently include training in this area. Grantees track outputs against M&E plans provided by TAF, but there is no evidence to suggest that knowledge gained through monitoring is used to inform grantees‘ project management. CSOs value the strategic oversight and constructive recommendations provided by TAF, and future technical support should continue to include internal governance 9 The Election Working Group (EWG) is a non-partisan, 32-member network of CSOs established in 2006 with funding from TAF. 10 Democratic governance grants were issued in two rounds: August 2011 and March 2012. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 23 and strategic planning, with a focus on sustainability. In addition, CSOs seek increased opportunities to discuss solutions to common challenges and plan collaborative activities. Findings for Result 5: More active role for CSOs and research institutions in informing and monitoring public policy This Result has seen significant progress, in the form of large numbers of DPPFs being established and becoming active. At the same time, though, limitations of the program are manifest, particularly with regard to the engagement of MPs and the identity of the DPPFs. Activity 5.1: Parliamentary policy research and public advocacy initiatives As of January 2013, TAF issued democratic governance research grants to seven Bangladeshi institutions, and scholars or policy experts based at those institutions led the development of draft research reports. Policy research areas include: public service delivery; social safety net policy; food grain procurement system; coastal agriculture and livestock development; citizen participation in primary education policy; healthcare governance issues; and the role of female MPs in Parliament. Researchers interviewed by the evaluation team identified the target audiences of their research as MPs and the general public, as well as CSOs and journalists. They emphasized their intention to develop policy briefs that would be used by PRODIP-supported DPPFs that engage in policymaking processes both inside and outside of Parliament. While the researchers interviewed had consulted with MPs, parliamentary standing committees and other relevant stakeholders during their data collection process, they did not solicit information about citizen priorities or public service quality from PRODIP-supported DPPFs. Activity 5.2: Development of parliamentary watch programs PRODIP supported the publication of a Parliamentary Watch Toolkit during the project period, but related activities have only been recently initiated and results have yet to be identified. TAF issued a democratic governance grant to D.Net in July 2012, and IID (D.Net‘s research wing) will implement the proposed activities. IID representatives interviewed by the evaluation team noted that they are mapping focus areas of other PRODIP grantees and highlighted a number of avenues to support the advocacy goals of the CSOs: policy research, e-governance, a parliamentary bill tracking system, budget visualizations and technical input to journalist training. Activity 5.3: Facilitate direct democracy in select districts Twelve of PRODIP‘s democratic governance grantees expanded upon working models of community-based groups to establish 140 public policy forum groups (DPPFs) across Bangladesh at the district, upazila and union levels. Following PRODIP guidelines regarding DPPF membership and structure, CSOs recruited civic activists from existing groups such as People‘s Reporting Centers, People‘s Alliances, People‘s Initiatives, Coordination Committees, Community Care Committees and Gona Unnayan.11 More than 75 percent of DPPF members 11 Some of these existing community-based groups were supported through the Citizen Alliance for Promotion of Transparency and Accountability (CAPTA), established with technical support from TAF. 24 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project surveyed reported past involvement in community groups that communicated directly with policymakers to advocate for policy change. At the same time, CSOs interviewed by the evaluation team noted that the diverse membership and politically neutral status of DPPFs distinguish them from other community-based groups. Lawyers, teachers, journalists, farmers and other relevant community stakeholders were invited to participate in addition to civic activists. While IDPS and ASUS had previously partnered with citizen groups wholly composed of indigenous minorities, the majority Bangla population now represents 20 percent of their DPPF membership rosters. Significantly, a number of CSOs noted that while MPs often seek to control forums like the DPPFs by positioning loyal party members as insiders, DPPF membership reflects supporters of both major political parties in Bangladesh. Several CSOs cited occasional challenges related to political bias or lack of committed participation on the part of some DPPF members. Such deficiencies are resolved by DPPF members themselves, with occasional input from CSOs. CSOs interviewed by the evaluation team reported that the DPPF initiative has contributed to formalizing policy dialogue between citizens and MPs, as well as focusing policy conversations on long-term – and sometimes national-level – advocacy goals. While community-based groups had relationships with MPs prior to PRODIP, nearly all DPPF members surveyed indicated that their relationships with policymakers have been expanded or strengthened as a result of involvement in the PRODIP-supported initiative. In most cases, DPPF members maintain direct contact with MPs from their constituencies. Two-thirds of DPPF survey respondents indicated that they follow up with policymakers after formal DPPF events. For instance, DPPF members track MPs‘ schedules and conduct individual meetings when MPs return to their constituencies on the weekends. TAF recorded 90 appearances by MPs at 53 DPPF events.12 However, repeated attendance by the same MPs was frequent, ultimately amounting to 26 appearances by distinct individuals. In addition, MPs are largely attending DPPF events in their own constituencies. The evaluation team found no evidence to suggest that non-constituency MPs attend DPPF events that may not be relevant to their local supporters but could be relevant to national-level policy debates. Similar to TAF‘s relationship with PRODIP grantees, CSOs provide oversight and guidance to DPPFs. (In at least three cases, direct involvement with DPPFs in the constituencies is handled by CSOs‘ implementing partners.) One CSO described its role as offering credibility for DPPFs, which are not always known to policymakers by name. Indeed, MPs interviewed by the evaluation team did not recognize the term ―DPPFs‖ and had difficulty distinguishing the initiative from other citizen forum projects. While DPPFs could be perceived as alternative, and therefore threatening, sources of leadership in communities, CSO involvement aims to underscore that DPPFs can be constructive and beneficial to MPs. At times, CSOs help DPPFs to facilitate public forums, conduct media outreach, and produce policy briefs. They also 12 DPPF survey respondents indicated that MPs or local elected officials ―sometimes‖ attend their events. The evaluation team notes that the survey question (―Do MPs or local elected officials ever attend your meetings?‖) was unclear and could have been interpreted as referring to internal DPPF monthly meetings. Other data substantiates the finding that MPs and local officials often attend DPPF public policy forums. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 25 provide external perspective and suggestions to DPPFs regarding the prioritization of advocacy topics or the management of sensitive issues. PRODIP grantees interviewed by the evaluation team believe that the DPPF model could be sustainable, especially in cases when members are committed to advocating for shared priority issues for their communities. Nearly two-thirds of the 35 DPPF members surveyed by the evaluation team reported that they would be self-motivated to continue their public policy advocacy activities without support from PRODIP. Other survey respondents expressed concern about the potential consequences of decreased financial support. DPPF members interviewed in Chittagong noted that bonds within their group could deteriorate without an external actor to help guide the group‘s strategy and activities. Activity 5.4: Promote women’s leadership in direct democracy activities In accordance with guidance from PRODIP, women comprise one third of DPPF membership across Bangladesh. In some cases, women actively serve in leadership roles such as president, convener, chair, vice chair or secretary. In several cases, CSOs noted that female members who participated in women‘s leadership training more effectively and confidently articulate their views to MPs. DPPF members interviewed in Chittagong reported that they do not feel dominated by men during internal DPPF meetings. Both female and male DPPF members interact, in coordination, with policymakers. One respondent noted that her voice seems louder and more respected in the DPPF setting, as opposed to feeling silenced and constrained in the upazila setting, where she also holds a leadership position. Several PRODIP grantees noted cases of DPPFs focusing on priority issues for women such as domestic violence, rights for widows, equal wages for women in the informal sector and the role of women in government. Both CSOs and DPPF members described that public policy advocacy issues that deal with agriculture or climate change, for example, are cross-cutting and affect women. For example, some DPPFs are seeking equal pay for women involved in home￾based processing of agricultural crops harvested by men. Coastal women and children are often most affected by natural disasters because they have difficulty reaching physical safety, and raising awareness about climate changes that provoke natural disasters could therefore be most beneficial to them. CSOs emphasized that female MPs who gain parliamentary seats via the quota system have limited power because they lack constituencies. In the limited cases when female MPs serve in the constituencies targeted by DPPF activities, CSOs reported that female policymakers attend DPPF events. One CSO also highlighted the need to strategically involve male policymakers as proponents for women‘s advancement in Bangladesh. Conclusions for Result 5 PRODIP has made significant progress in this area during the project period. PRODIP partners regularly provide inputs to policy discussions, both through politically-unbiased research and through formal dialogue between DPPF members and constituent MPs. Recipients of TAF research grants envision their research being used to inform dialogue and support citizen participation in the policymaking process. Researchers interact with MPs, parliamentary standing committees and other PRODIP CSO grantees, but not necessarily with DPPF members. Steps could be taken to increase the use of policy research by relevant PRODIP 26 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project grantees. If implemented as planned, parliamentary watch activities conducted by D.Net/IID could serve as a unifying force and mechanism for information-sharing among PRODIP grantees, as well as help to advance advocacy goals on the national stage. PRODIP democratic governance grantees mobilized the existing capacity of active citizen groups across Bangladesh to establish 140 public policy forums at the district, upazila and union levels. Gender inclusion was prioritized in the formation of DPPFs: one third of members are women, some women actively serve in leadership roles, and female DPPF members do not feel dominated by male colleagues in DPPF settings. With oversight and guidance from CSOs, DPPFs advocate for policy changes and monitor the implementation of laws at the local level. Yet, while MPs regularly attend DPPF events, familiarity with ―DPPFs‖ as formal public policy advocacy groups is almost non-existent among committee chairs, MPs and personal secretaries. With improved branding, dialogue with non-constituent MPs, continued citizen commitment, and internal capacity-building, DPPFs could prove to be a sustainable model for public policy advocacy at the national level. Findings for Result 6: More active role for CSOs and the media in educating the public on democratic processes and practices This Result has been underperforming, in part because many activities are so recent. Activity 6.1: Conduct constituent perception surveys, regulatory impact assessments and other diagnostic research PRODIP commissioned three perception surveys of MPs, citizens and CSOs, as well as a synthesis report of the survey findings. The synthesis report was finalized more than one year ago, but parliamentary approval for wider distribution of the report is still pending. PRODIP noted that it used some of the survey results to identify outcome indicators, progress against which it does not track every quarter. The baseline information included in the survey findings has not been used to set any targets; rather, targets for output indicators are set by PRODIP toward the end of each quarter. Activity 6.2: Develop interactive PRODIP website PRODIP formally launched its website (http://prodip.org) in the third year of the project period. Six months later, CSO and research grantees interviewed by the evaluation team were aware that a PRODIP website exists but were unfamiliar with its content and purpose, raising questions about whether they have used it at all. Most grantees assumed that they would be the primary target audience for the website, though some highlighted other possible users including media, MPs and DPPF members. Activity 6.3: Support national information campaigns PRODIP recently mobilized some of its grantees to develop media content on themes related to parliamentary governance. The evaluation team did not review the content, and this topic was not discussed during key informant interviews. Activity 6.4: Support national media outreach As of January 2013, two recipients of PRODIP democratic governance grants (MRDI and JATRI) were planning activities related to the media component of public policy advocacy. Given the Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 27 abbreviated implementation period (4-10 months),13 the evaluation team did not assess the results of this activity. Activity 6.5: Produce a series of public service announcements PRODIP commissioned four PSAs about parliamentary governance, policy advocacy and other PRODIP themes. TAF reported that the PSAs were broadcasted on radio stations with national coverage. While it is possible to track statistics about the radio audiences at the time of the PSAs‘ airing, the impact of the messages is undetermined and difficult to assess. Conclusions for Result 6 As of the mid-term of the project period, PRODIP is underperforming in this area. Thus far, PRODIP has yet to provide CSOs and the media with tools to educate the public on democratic processes and practices. All of PRODIP‘s proposed activities under this result have been initiated recently, most of them near or during the third year of the project. The PRODIP website remains in the preliminary stages of development, several PSAs of uncertain effect have been produced and aired, and two grants have been awarded to build the capacity of the media. Overall, it is too early to assess the impact of the limited activities conducted by PRODIP under this result. Overall Conclusions for Objective 2: A More Constructive and Sustainable Role for Civil Society in Democratic Governance Democratic governance grants and technical support provided by PRODIP have increased the capacity of civil society actors to provide constructive input to the policymaking process at local and national levels. PRODIP‘s guidance regarding gender-balanced DPPF membership enables women to participate as equal partners in issue-based policy dialogue. Formal engagement with MPs represents a strategic shift for community-based groups throughout Bangladesh, but commitments from policymakers have yet to be translated into action. In addition, MPs and parliamentary staff do not distinguish PRODIP-funded DPPFs from other citizen-based advocacy mechanisms in Bangladesh. DPPFs could become a sustainable model for public policy advocacy at the national level, especially with improved branding and engagement of non-constituency MPs around issues of shared national interest. Thus far, dialogue between citizens and policymakers is based largely upon pre-existing relationships with constituent MPs. By drawing upon its own relationships with MPs and identifying synergies between Component 1 parliamentary support activities and Component 2 public policy advocacy efforts, PRODIP could play a stronger role in reinforcing the DPPF brand and advancing its success. Several activities that intend to provide CSOs and the media with tools to educate the public on democratic processes and practices have only been recently initiated by PRODIP, and the effects of these activities are uncertain. PRODIP grantees identified a number of specific opportunities to use the PRODIP website as a vehicle for information-sharing and coordination. These recommendations are detailed below. 13 Media-related democratic governance grants were awarded in March and September 2012. 28 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project PERFORMANCE CONCLUSIONS With regard to evaluation question areas A (Results), C (Client Satisfaction), F (Sustainability) and G (Gender as a Cross-Cutting Issue), the team presents the following conclusions: PRODIP‘s activities are contributing to the development of Parliament, in a broad but shallow way. The capacity of MPs, PSs and staff have increased noticeably, but this progress is vulnerable to turnover through election, staff returning to ministries, and patronage. Other successes can be seen in the APPGs and the Women‘s Caucus, which are bringing together MPs across party lines to develop better public policy. Public hearings are a new addition to parliamentary activities, one that has captured the attention of MPs and resulted in ad hoc improvements in policy approaches. Further progress may be limited by the current rules of procedure and the general nature of a Westminster system. Stakeholders in Parliament are generally satisfied and value PRODIP activities. However, they made several salient points about the nature and quality of PRODIP‘s interventions that should influence the scope and structure of activities in the future. These activities would also likely not continue without PRODIP‘s involvement and funding, given the current state of Parliament‘s budget. While gender inclusion is more pronounced in Objective 2, there are clear signs of progress in Objective 1 in the Women‘s Caucus, the Washington study tour, and policy topics for public outreach events. The integration of the two PRODIP objectives at specific points is weak and incidental. Members of DPPFs may attend public hearings or field investigations, but their presence and significance are lost on the MPs. Experts from academia, think tanks and CSOs are not yet building relationships with committees and MPs in their policy areas. PRODIP interventions are contributing to the increased capacity of civil society to participate in policy dialogue at the local and national levels. Formal engagement with MPs about policy topics represents a strategic shift for community-based groups throughout Bangladesh, and PRODIP‘s project design and implementation emphasizes the importance of women‘s participation in the policymaking process. While PRODIP grantees and DPPFs have successfully provided input and conducted research to inform policy debates, commitments from policymakers have yet to be translated into action. In addition, dialogue between citizens and policymakers appears to be based largely upon pre￾existing relationships with constituent MPs. There is no evidence that PRODIP is drawing upon its own relationships with MPs to promote dialogue based on shared policy interests. Furthermore, MPs and parliamentary staff are unfamiliar with PRODIP-funded DPPFs as a formal mechanism to promote public policy advocacy in Bangladesh. With improved branding and increased dialogue with non-constituent MPs, DPPFs could prove to be a sustainable model for public policy advocacy at the national level. PRODIP has only recently initiated several activities that seek to provide CSOs and the media with tools to educate the public on democratic processes and practices; the effects of these activities are uncertain. PRODIP has made significant progress in this area during the project period. PRODIP partners regularly provide inputs to policy discussions, both through politically-unbiased research and Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 29 through formal dialogue between DPPF members and constituent MPs. Recipients of TAF research grants envision their research being used to inform dialogue and support citizen participation in the policymaking process. Researchers interact with MPs, parliamentary standing committees and other PRODIP CSO grantees, but not necessarily with DPPF members. Steps could be taken to increase the use of policy research by relevant PRODIP grantees. If implemented as planned, parliamentary watch activities conducted by D.Net/IID could serve as a unifying force and mechanism for information-sharing among PRODIP grantees, as well as help to advance advocacy goals on the national stage. PRODIP democratic governance grantees mobilized the existing capacity of active citizen groups across Bangladesh to establish 140 public policy forums at the district, upazila and union levels. Gender inclusion was prioritized in the formation of DPPFs: one third of members are women, some women actively serve in leadership roles, and female DPPF members do not feel dominated by male colleagues in DPPF settings. With oversight and guidance from CSOs, DPPFs advocate for policy changes and monitor the implementation of laws at the local level. Yet, while MPs regularly attend DPPF events, familiarity with ―DPPFs‖ as formal public policy advocacy groups is almost non-existent among committee chairs, MPs and personal secretaries. With improved branding, dialogue with non-constituent MPs, continued citizen commitment, and internal capacity-building, DPPFs could prove to be a sustainable model for public policy advocacy at the national level. PROCESS FINDINGS AND CONCLUSIONS The process evaluation side of this exercise covers questions of management, opportunities, and theory and design (evaluation question areas B, D and E). Management and Administration Findings PRODIP‘s performance management system is more extensive and thorough than is commonly seen in parliamentary strengthening programs, and it generally reflects good practices. However, the M&E plan does not capture data on important or priority activities, which raises questions about its utility for program management and for evaluation. Another issue with the plan is that both Objectives saw meaningful changes in expected activities and results almost every quarter, so again its value as a management or evaluation tool is discounted. For Objective 1, the volatile expected results and actual achievements are really a function of cooperation by the Parliament Secretariat and committee chairs; Objective 2 is in some ways more amenable to setting and achieving targets, because its relationships with beneficiaries or participants is quite different. It was not clear to the team that PRODIP‘s performance management system was being used to support management decisions, particularly in light of the demand-driven nature of parliamentary strengthening activities. The baseline data that had been gathered has not been shared with any partners or stakeholders and is not publicly available. Nor did we uncover any evidence that stakeholders know of PRODIP‘s M&E information. Moreover, PRODIP was not always gathering the kind of data that would be needed in dealing with stakeholders. For example, the evaluation team used paper documents supplied by PRODIP to identify which 30 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project trainees were MPs, seconded personnel or permanent staff, an important disaggregation for both PRODIP management and the Secretariat to use for training planning. Collaboration between TAF and SUNY is limited, as far as the team was able to find, which may affect the success of the program. The team was struck by the inconsistency of organizational labels throughout PRODIP. Parliamentary stakeholders and participants used the term ―PRODIP‖ to refer to the program, but the civil society stakeholders, partners and grantees all referred to ―Asia Foundation‖ in the materials and in interviews. Indeed, signs at trainings or public events list The Asia Foundation as prominently as the program funders, with the PRODIP label much smaller. SUNY seemed to be a label used only at PRODIP offices. PRODIP staff themselves identify by institution. Within the project itself, the two objectives interact in only tangential ways, as MPs will sometimes attend DPPF events and DPPF members will attend MP public hearings. The core CSOs do not interact with the SUNY staff, and committee chairs and staff do not interact with TAF staff. Coordination between large legislative strengthening programs in Bangladesh is on a basic information-sharing level, which mirrors the coordination between the donors funding parliamentary programs. Stakeholders are the least informed, although parliamentary leadership has been moving toward more deliberate engagement with donors and programs. There are a host of donor-funded programs besides PRODIP working in Parliament: UNDP IPD and SPEMP-C are the largest of these other programs, with smaller efforts by UNFPA, UN Women, and others. Over time and with pressure from parliamentary leadership, the two largest programs, IPD and PRODIP, settled on a committee division of labor for their activities. The programs do not coordinate activities as much as they simply inhabit parallel worlds. SPEMP-C targets the three ‗financial‘ committees of Public Accounts, Estimates, and Public Undertakings, which are not committees in PRODIP‘s portfolio. Leadership in all of the large programs talk to each other regularly, but there is no sharing of schedules or work plans. The lack of coordination among the donors has been a long-standing problem. In the aftermath of the 2008 elections, the large programs were designed without coordination with each other, or even much engagement with Parliament. While donor coordination at an early point took place in the Group on Political Governance (GPG), a narrower Parliamentary Task Force (PTF) emerged specifically to deal with the problem of coordinating parliamentary programs. A successful endeavor, the PTF merged back into the GPG in 2012. Neither the PTF nor the GPG include all of the programs providing assistance to Parliament. The relationship between parliamentary leadership and donor-funded parliamentary strengthening programs has been quite acrimonious at times, due in part to the lack of engagement with or communication to leadership by programs. One result of this lack of stakeholder consultation on program design has been limited ownership of programs or even understanding of the scope and structure of them. The Secretariat has assigned different focal points for each program, with the specific people changing periodically. An advisory committee has been in operation recently to help Parliament track and interact with the various programs. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 31 Management and Administration Conclusions The evaluation team concludes that significant management and coordination problems exist at all levels: within the program, with the program‘s stakeholders, between programs, with donors‘ stakeholders and between donors. Program Opportunity Findings The team believes that USAID would not have been better served by using its funding for the UNDP IPD program. While not based on a rigorous application of evaluation methodology, the team noted that IPD‘s approach of providing training and other assistance to party leadership, committee chairs and PSs leaves it even more vulnerable to the turnover problem that threatens PRODIP‘s progress. Moreover, PRODIP‘s posture of responding to needs articulated by its stakeholders ignores the very real need to both help stakeholders recognize their needs and introduce new ideas. The team also notes that PRODIP has a larger budget than many parliamentary strengthening programs in other assistance countries. Program Opportunity Conclusions The evaluation team concludes that a parliamentary program under the direction of USAID was and is the best option for U.S. foreign assistance. Program Theory and Design Findings The section of the 2009 DG Assessment calls for a ‗holistic‘ program designed to work with Parliament as an institution, and Objective 1 does that: It works with MPs, committees, parliamentary staff and issue-based non-partisan caucuses. The multi-year program aims to increase committee effectiveness and improve committee oversight. The DG Assessment, however, is much too optimistic about the potential for oversight, due to the nature of Westminster systems that fuse the executive and legislative branches. Another point of variation is that PRODIP does not really work with Parliament as a partner, because it does not share schedules and work plans with parliamentary leadership; this reinforces the impression of stakeholders that PRODIP activities are neither coherent nor strategic. Program Theory and Design Conclusions The PRODIP design incorporates Section 5.3.2 of the 2009 DG Sectoral Assessment well, and goes beyond it to add the civil society objective. 32 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project RECOMMENDATIONS AND LESSONS LEARNED The team would like to present recommendations for consideration by USAID for the future of the PRODIP project at the results level and at the overall program level. RESULTS-LEVEL RECOMMENDATIONS Objective 1: An Effective and Open Parliament Result 1: Improved Parliamentary Lawmaking Skills & Oversight Capacity Internship Program: Stakeholders viewed interns as a particularly valuable resource for Parliament, but the scope of the activity should be expanded considerably in specific ways. Law interns are especially prized for providing much-needed capacity on legislative drafting and legal research. Very few MPs are lawyers, and the committee staff ranks include no lawyers at all. PRODIP should increase the number of interns, focusing on interns with law backgrounds, with the aim of providing one such intern to each targeted committee. In addition, gender equality in the set of interns should be a guiding principle in the selection process. Chairs/PSs versus Permanent Staff: The two types of Parliament actors have distinctly different training needs and use their training in different ways. Chairs and PSs need training on managing the functioning of committees, i.e., parliamentary procedure, committee procedure, work plans, etc.; as much as they gained valued knowledge on legislative drafting and report writing, the reality is that committee staff actually apply those skills in practice. Among committee staff, it is the permanent staff that represents the best training investment for PRODIP, e.g., in legislative drafting, report writing, research, etc. PRODIP should re-configure its training structure and approach to better target each type of parliamentary actor with the content is most appropriate to the role each plays in a committee. Note that this emphasis would also eliminate many of the arbitrary differences in the structure of Results 1 and 2. Result 2: Stronger Parliamentary Committee System Multi-MP Caucuses PRODIP should devote more resources to expanding the activities of APPGs, a successful activity that participating MPs found energizing and valuable. In light of the possibility that the Tenth Parliament will see a wider distribution of seats among parties, an activity that brings together members of different parties in pursuit of common policy interests could provide significant benefits to the peaceful coexistence of rivals. Tying this activity to specific committees would enhance the planning and policy orientation of those committees and thus fit neatly into the goal of strengthening committee effectiveness. On the other hand, PRODIP should eliminate support for party caucuses, which is an activity that is normally part of a party assistance program. The very real possibility that a party caucus Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 33 could become an irritant in party relations represents a threat to the perceived non-partisan intent of a parliamentary strengthening program. Result 3: Increased Public Awareness/Participation in Parliamentary Functions/Processes Re-Focus Purpose Result 3 has very little progress to show after almost three years, though public hearings are a notable exception. PRODIP should reduce the number of activities under this result in order to conserve resources and clarify the purpose of the Result. More resources could then be devoted to facilitating public hearings, a successful and valued activity that has motivated chairs to plan and function as a committee. This Result also presents an opportunity for collaboration between the two Objectives, in the form of integrating media CSOs into activities such as public hearings and research. Objective 2: A More Constructive and Sustainable Role for Civil Society in Democratic Governance Result 4: Increased institutional capacity of CSOs to advance public participation in democratic governance Synergies Between Democratic Governance Grants PRODIP should facilitate stronger communication and information sharing between its democratic governance grantees and research grantees, as well as reconsider the format of its quarterly CSO coordination meetings to provide grantees with opportunities to discuss – and act upon – advocacy synergies. Grantees interviewed by the evaluation team highlighted potential avenues for collaboration based on thematic areas of focus or geographic location. Policy research published by PRODIP grantees, including by recipients of seven research grants, could better inform advocacy strategies of other CSOs working on relevant topics. For instance, BNWLA used the gender budget analysis produced by D.Net to advance its advocacy activities to enforce the implementation of laws that protect women and children. In addition, longtime partners D.Net and MRDI have initiated conversations about sharing technical expertise to benefit MRDI‘s training for journalists. Several CSOs suggested that they would benefit from experiential learning through domestic exchanges with DPPFs in other regions. Many grantees stated that PRODIP coordination meetings could be more effective for planning collaborative activities and discussing solutions to common challenges. CSOs provided specific recommendations for changes to the format and content of the meetings. For example, using the PRODIP website or other mechanisms for information sharing and programmatic updates would enable coordination meetings to be used more efficiently for in-depth discussions about program synergies or implementation challenges, as well as development of joint action plans to advance advocacy objectives from the local level to the national level. Grantees noted that such conversations are especially important in the lead-up to elections this year. 34 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Result 5: More active role for CSOs and research institutions in informing and monitoring public policy Further Development of DPPFs While the commitment and energy of DPPF members is genuine, the core CSOs provide valuable guidance and training that keeps DPPFs from a slow demise to oligarchic leadership and dissolution, so a strategy that recognizes this ongoing need is critical. PRODIP should expand its assistance to DPPFs to address sustainability issues. As a program, it should create a strategy for DPPF sustainability, as well as work with DPPFs to develop their own strategies. To promote sustainability, some CSOs suggested additional training for DPPF members on topics including leadership, meeting facilitation, parliamentary processes and media relations. In addition, CSOs recommended increased knowledge-sharing, domestic exchanges between DPPFs and collaboration among DPPFs that focus on common policy issues. A small but important step in the development of DPPFs is their branding or identity as DPPFs. PRODIP‘s strategy should address this issue. Better integration with relevant Objective 1 activities would also enhance their engagement in the policy process. Result 6: More active role for CSOs and the media in educating the public on democratic processes and practices Better Integration of PRODIP Website The PRODIP website must be better integrated with the extensive activities of the democratic governance grantees and support their interaction with relevant Objective 1 activities. PRODIP grantees identified a number of specific opportunities to use the website as a vehicle for information-sharing and coordination among PRODIP partners. Grantee recommendations for website content include: DPPF advocacy successes and CSO program achievements, research papers and policy briefs authored by PRODIP grantees, a parliamentary bill tracking system, TAF grant administration materials, and guidelines for DPPF program implementation. The website could serve as a critical link between democratic governance research grantees and DPPFs, which do not currently maintain direct contact with each other. With CSOs, media outlets and parliamentary staff as potential users, the parliamentary bill tracking system to be developed by D.Net/IID would be a strategic addition to the PRODIP website. Virtual discussion forums housed on the PRODIP website should be used to facilitate dialogue that advances democratic processes and practices in Bangladesh. (Two public forums exist on the PRODIP website, but they appear to be unused.) Select forums could remain public to enable input from and exchange with citizens and stakeholders on various topics. In this way, the PRODIP website could serve as a vehicle for educating the public on democratic processes and practices, potentially serving the same purpose as and replacing the need for additional PSAs. In addition, other private forums could be established to encourage frank discussion about DPPF implementation challenges and generate dialogue about solutions. Several grantees mentioned that they would benefit from such action-oriented exchanges on private forums. Finally, the PRODIP website should serve as a branding mechanism that emphasizes the synthesis between Component 1 and Component 2 of the PRODIP project. The website could also contribute to branding and outreach efforts for the DPPF initiative, which was rarely known as such by policymakers interviewed by the evaluation team. Information and press releases about upcoming events organized by DPPFs across Bangladesh should be highlighted on Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 35 the PRODIP website to encourage participation by media and citizens, as well as to share information about opportunities for PRODIP grantees and DPPFs to strengthen their advocacy efforts through collaboration. (A calendar exists on the PRODIP website, but it has not been populated with data.) Effective website development and maintenance would require sustained efforts by dedicated PRODIP staff. Use of coding tools could enable routine tracking of existing PRODIP grantee websites and automatic transfer and re-posting of relevant information on the PRODIP website. Otherwise, website content and virtual discussion forums would need to be managed by PRODIP staff. Those designated staff could also liaise with democratic governance grantees – especially those focusing on the media component of public policy advocacy – and public affairs and library staff in the Jatya Sangsad to ensure that PRODIP website content and resources are marketed sufficiently on other websites and in supplemental outreach materials. PROGRAM-LEVEL RECOMMENDATIONS Performance Management System PRODIP management should review and revise its performance management system. The project should be gathering information that would be (a) of better management use, e.g., trainee disaggregations by status, gender, training topic, and so on, or (b) of interest to stakeholders, e.g., trainee disaggregations, trainee coverage by committee, committee effectiveness index scores and DPPF members‘ perceptions of internal democracy. An annual review of indicators for stakeholders would encourage more stakeholder ownership. Collaboration between Objectives The two PRODIP objectives are largely stand-alone efforts. USAID should require PRODIP in the next few months to develop a strategy and detailed work plan that addresses collaboration specifically. The strategy should identify specific activities in the two objectives that provide opportunities to combine the strengths of both in the service of the overall purpose of the program. The detailed work plan should then address specifically what will be done, by which specific PRODIP personnel, in a specific scheduled period of time, with a budget specific to each objective. If USAID is not satisfied with the strategy and work plan, it should consider separating the objectives into two stand-alone programs. LESSONS LEARNED The evaluation team also draws USAID‘s attention to some lessons learned from this program‘s implementation that should inform future parliamentary programming efforts:  Parliamentary strengthening programs are by nature quite demand driven, which places a premium on prior consultation with parliamentary and party leaders before a program design is finalized in order to get a better sense of needs and increase ownership.  Increased ownership and consultation also reduce the volatility of program activities, enabling implementer management to better control work flow and budgets, and to focus more closely on achieving desired results. 36 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project  Program work plans should only be finalized when Parliament has a relevant current strategic plan, and then the implementation of program activities should be predicated on Parliament‘s achievement of relevant benchmarks, in order to reduce program design and activity volatility.  When there are multiple programs targeting different aspects of Parliament, donors and implementers must coordinate with each other to plan new MP orientation after elections, including a division of labor. MPs often attend redundant trainings, receive offers for conflicting timeframes and generally suffer from orientation program fatigue.  Parliamentary programs must differentiate the needs of MPs and senior advisors from the needs of parliamentary staff. The former group has different training needs and time horizons for results than the latter.  Parliamentary strengthening program implementers should agree with parliamentary leaders on the selection logic and scope of who gets what training and then track the application of this agreement with appropriate schedules and data, in order to use program resources more efficiently.  The executive-legislative relationship in Westminster systems places severe limits on the amount of oversight and policy entrepreneurship that can be practiced by MPs, for structural and incentive reasons that cannot be changed by assistance programs, which is important for setting reasonable expectations for reform progress.  Public policy advocacy programs are distinct from traditional advocacy programs and, accordingly, should incorporate training and capacity building in the areas of legislative processes, budget appropriations, public opinion research, policy brief drafting and other related topics to better meet the needs of partners.  Public policy advocacy initiatives that seek results at the national level must be primarily issue-based rather than constituency-based. National-level policy change may require targeted communication and relationships with parliamentary leaders outside citizens‘ constituencies in order to be most effective.  Informal citizen groups often benefit from strategic guidance and oversight provided through consultative relationships with external donors. Strategic planning and effective branding can contribute to longer-term sustainability of citizen groups.  Adherence to a gender-balanced membership policy can help position women to participate in citizen groups as equal partners in public policy dialogues. Such citizen groups can become alternative venues for women‘s political participation, as formal political structures are often dominated by men or male perspectives. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 37 ANNEX I: EVALUATION STATEMENT OF WORK Statement of Work for the Promoting Democratic Institutions and Practices (PRODIP) program Mid-Term Performance Evaluation USAID/Bangladesh Office of Democracy and Governance Program Identification Data Program Title: Promoting Democratic Institutions and Practices (PRODIP) Program Number: 388-A-00-10-00092-00 Program Dates: April 28, 2010 – April 27, 2015 Program Funding: $ 23,210,783 Implementing Organization The Asia Foundation (TAF) with State University of New York/Center for International Development Assistance Officer Representative (AOR): Billy Woodward I. BACKGROUND Bangladesh is among the world‘s most densely populated countries with more than 150 million people. As the seventh most populous country in the world and with the fourth largest Muslim population, Bangladesh‘s stability and prosperity are important to U.S. foreign policy interests. Ranked as the sixth most exposed country to natural disasters, Bangladesh‘s low lying landscape is subject to flooding and cyclones, which will become more severe with climate change. Bangladesh is poor, making it susceptible to social upheaval and political conflict. U.S. Government (USG) foreign assistance is helping Bangladesh build its democratic institutions, reduce poverty through broad-based economic growth and improve its human capital. Since independence, Bangladesh has held democratic elections sporadically, including three successive peaceful transfers of power from 1991 to 2006. Yet, the country‘s development as a democracy has been interlaced with military rule, debilitating political polarization, ineffective institutions of governance, and endemic corruption. Partisan deadlock over electoral issues resulted in the cancellation of the January 2007 parliamentary elections and the self-imposed rule by the military-backed caretaker government. Following the two-year state of emergency enforced by the caretaker government, a new chapter in the country‘s history opened for democratic rule with the success of the free and fair parliamentary elections held on December 38 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 29, 2008. The difficult but successful transition back to elected government brought about new opportunities for democratic development and a new sense of urgency in the country to reform political practices and institutions of governance. This new chapter also presented the United States with an historic opportunity to help Bangladesh improve the country‘s governance, foster economic development and deny space to terrorists. Political improvements in Bangladesh have not kept pace with economic expansion over the past decade. Despite continued economic gains, including economic growth averaging 5.66% over the past 10 years (compared to the world average of 3.49% for the same period), government institutions and political parties (particularly the Awami League and the Bangladesh National Party) continue to operate in a patron-client relationship with the people of Bangladesh. This relationship and a winner take all approach to governing after securing electoral victory has stalled Bangladesh‘s political and democratic advances. The Jatya Sangsad (Parliament) has especially suffered under this system, as ruling-coalition MPs take little interest in their legislative, oversight or public policy responsibilities and ―loyal opposition‖ MPs simply refuse to participate in government. As a result, Bangladeshis have traditionally had little space (or interest) in pressing their elected leaders for changes to the status quo, which only heightens the disconnect citizens have with their government. In April, 2010, USAID awarded a cooperative agreement to The Asia Foundation (TAF) to implement a five-year legislative function and processes improvement program known as Promoting Democratic Institutions and Processes (PRODIP) with a total estimated cost of $23,210,783. PRODIP‘s two objectives were (1) An Effective and Open Parliament and (2) A More Constructive and Sustainable Role for Civil Society in Democratic Governance. Under these objectives, PRODIP targeted the following results:  Improved parliamentary lawmaking skills and oversight capacity;  A stronger parliamentary committee system;  Increased public awareness of—and citizen participation in—parliamentary functions and processes;  Strengthened institutional capacity of Civil Society Organizations (CSOs) in areas of public and democratic governance;  More informed public policy-making and monitoring of policy implementation at the national and sub-national level; and,  Increased citizen ―voice‖ for transparent and accountable governance. PRODIP‘s strategic approach to achieving these results was to simultaneously assist both Parliament (supply-side – Component 1) and Civil Society Organizations (demand-side – Component 2) in creating an open and effective Parliament and to foster a more constructive and sustainable role for civil society in democratic governance. On the supply side of the governance equation, PRODIP is assisting Parliament to improve its lawmaking and oversight capability, strengthen its committees, increase public awareness of its functions and create opportunities for increased public input into the development of national policy. On the Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 39 demand side, PRODIP is helping civil society to assist the Parliament to become more effective, transparent, and participatory by building capacity in the areas of legislation, oversight, and representation.by providing technical assistance, training, and institutional support to CSOs to help shape public policy and provide a vehicle for public input. PRODIP‘s development hypothesis was to simultaneously increase citizen engagement in the political process and move the Parliament to becoming a more democratic, inclusive, and deliberative body. This combined approach was then expected to increase citizens‘ confidence in government. Perhaps the most critical assumption with this hypothesis was that the Jatya Sangsad would be receptive and support this approach. PRODIP is jointly funded with United Kingdom aid from the Department of International Development (DfID). They recently concluded an annual review of progress on PRODIP (September 2012) that outlines their successes and on-going challenges since program inception. A hard copy draft report was made available to USAID/Bangladesh and will also be provided to the evaluation team. II. OBJECTIVES OF THE EVALUATION This external, mid-term performance evaluation will review the progress made towards achieving the results/outcomes to date. The evaluation will also identify any lessons learned through the half-way point of PRODIP‘s five-year program. Specifically, the evaluation will:  Evaluate PRODIP overall performance by assessing results against stated targets and indicators;  Assess the efficacy and results of the PRODIP implementation approaches and management structure in meeting the objectives;  Make recommendations to USAID/Bangladesh concerning possible programming changes or adjustments to the second half of PRODIP‘s implementation; and  Analyze interventions under component 2 (Civil Society) to determine if they are supporting the cooperative agreement intent to strengthen legislative function and processes. The audience for this evaluation is USAID/Bangladesh, USAID/Washington, leaders of USAID Forward, TAF, and existing USAID partners such as DfID, UNDP and other bi-laterals and multi-lateral donors to Bangladesh. III. EVALUATION QUESTIONS This Scope of Work is for a mid-term evaluation of PRODIP‘s almost three years of implementation (April 2010 to January 2013). The evaluation should review, analyze, and evaluate the PRODIP program along the following criteria, and, where applicable, identify 40 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project opportunities and recommendations for improvement. In answering these questions, the Evaluation Team should assess the performance of both USAID and its implementing partner(s). A. Results 1. To what extent has PRODIP been successful in achieving the program objectives? 2. How well did the activities of each of PRODIP‘s two components contribute, in a coordinated way, to building a stronger parliamentary institution? B. Management and Administration 3. To what extent has PRODIP‘s performance management system provided useful data to support management decisions? To what extent have PRODIP management and stakeholders incorporated knowledge gained through PRODIP‘s M&E into project management? 4. How effectively have TAF and SUNY worked together in implementing PRODIP and achieving programmatic success? 5. To what extent did PRODIP coordinate with other legislative strengthening programs? C. Client Satisfaction (GOB, beneficiaries, other stakeholders) 6. What are the levels and areas of project stakeholder satisfaction or dissatisfaction with PRODIP cooperation and performance? This group includes: leaders and staffs of Parliament, the Office of Speaker of the Parliament, and Parliamentary Secretariat; civil society and citizens‘ organizations; USAID and other implementers of legislative programs. D. Program Opportunities 7. To what degree is PRODIP‘s approach a cost-effective means to increase citizen confidence in governance institutions? E. Program Theory and Design 8. How well did USAID incorporate section 5.3.2 of the 2009 DG Sectoral Assessment into the PRODIP design? F. Sustainability 9. How sustainable are PRODIP activities beyond USAID support? G. Cross Cutting Issues 10. How well were gender issues addressed by PRODIP program? Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 41 IV. PROPOSED EVALUATION METHODOLOGY The detailed methodology of this mid-term evaluation will be designed by the evaluation team in the work plan; this will include presentation of a data collection matrix that will explicitly link evaluation questions to particular data collection approaches and data sources. Some illustrative methods (to be finalized by the evaluation team in coordination with the Mission), include:  Review of background documents, including, for example: the technical proposal from the PRODIP award; all PRODIP Quarterly and Annual Performance Reports; PRODIP Work Plans and M&E Plan; DfID‘s September 2012 annual review that outlines PRODIP‘s successes and on-going challenges since the program inception; reports from PRODIP-sponsored surveys and analytical research; lists of PRODIP grantees and subcontractors, with activity descriptions, levels of PRODIP resources and points of contact.  Key Informant Interviews or group interviews with USAID/Bangladesh, USAID/Washington, , PRODIP staff, SUNY staff, leaders and staff of Parliament, the Parliament Speaker, the Parliament Secretariat, civil society, citizens‘ organizations and other donors, senior grantee staff.  Focus groups or joint discussions with civil society organizations, journalists, public intellectuals, etc., informed of PRODIP‘s program. Some of these interviews will take place outside of Dhaka. Interview tools or checklists of about 10-12 items will be prepared for each type of interview or discussion. The tools will be shared with USAID at the in-brief and as part of the evaluation report. The evaluation team should review the proposed methodology and alternative methods in light of the evaluation questions, timeframe, budget, data collection requirements, quality of existing data sources, and potential biases. The team will build on the proposed methodology and provide more specific details on the evaluation methodology in the evaluation work plan (see Deliverables below). The evaluation will be utilization-focused, and the evaluation methodology will be finalized through review and discussion between USAID/Bangladesh and the evaluation team. The methodology narrative should discuss the merits and limitations of the midterm evaluation methodology. The Evaluation Team will design appropriate tools for collecting data from various sources. The information collected should be evidence-based and will be analyzed by the Evaluation Team to establish credible answers to the questions, provide major trends and issues, draw conclusions and make action-oriented recommendations. USAID requires that evaluations explore issues of gender; thus, the evaluation should examine gender issues within the context of the evaluation of PRODIP activities. Specifically, what was the gender balance on PRODIP 42 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project sponsored workshops, trainings, and other forums? Was PRODIP able to mainstream gender into the national budget (p. 28 of Cooperative Agreement). The Team will make recommendations to USAID/Bangladesh concerning possible programming changes or adjustments to the second half of PRODIP‘s Cooperative Agreement. Methodological limitations and challenges for this evaluation are expected to include:  Ensuring that samples of interview sources are sufficient to qualitatively support evaluation findings;  Taking systematic actions to counter any biases in (a) reporting by data collection sources and (b) interpretations of collected data by the evaluation team and  Ensuring we are able to measure ―actual‖ results, which will only be possible if we can gather and analyze data beyond respondent perceptions – for example, changed activities of Parliamentary committees. The work plan should discuss the merits and limitations of the final evaluation methodology. The Evaluation Team will be required to perform tasks in Dhaka, Bangladesh. V. EXISTING SOURCES OF INFORMATION USAID/Bangladesh DG Office will provide documents for the desk review (see Annex B for a full list of documents) and contact information for relevant interviewees. In-country staff of Social Impact‘s Bangladesh Democracy and Governance Program Evaluations (BDGPE) project also has established a working relationship with PRODIP staff, to support sharing of information for the evaluation. The Evaluation Team will be responsible for identifying and reviewing additional materials relevant to the evaluation. VI. DELIVERABLES Evaluation deliverables include14: Evaluation Team Planning Meeting (TPM) – essential in organizing the evaluation team‘s efforts. During the meeting, the evaluation team should review and discuss the SOW in its entirety, clarify team members‘ roles and responsibilities in meeting the work plan, develop data collection methods, review and clarify any logistical and administrative procedures for the assignment/instruments and prepare for the in-brief with USAID/Bangladesh. Social Impact will provide the team with a Team Planning Guide and other support for the TPM. 14 Refer to the Evaluation Schedule for a detailed timeline on when the deliverables are required. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 43 Work Plan – Detailed draft work plan (including task timeline, methodology outlining approach to be used in answering each evaluation question, some illustrative questions, site selection, team responsibilities, and data analysis plan). The work plan is due to USAID/Bangladesh one working day after the Team Planning Meeting. In-Brief Meeting – In-brief with USAID/Bangladesh within one working day after submission of the Work Plan; USAID/Bangladesh will schedule the in-brief with the evaluation team through SI. Regular Updates – The Evaluation Team Leader (or his/her delegate) will brief the BDGPE COR and the PRODIP AOR on progress with the evaluation on at least a daily basis, in person or by electronic communication when the team is in Bangladesh. Any delays or complications that could comprise the approved timeline must be quickly communicated to USAID/Bangladesh as early as possible to allow quick resolution and to minimize any disruptions to the evaluation schedule. Emerging opportunities for the evaluation should also be discussed with USAID/Bangladesh. Debriefing with USAID/Bangladesh – Power Point presentation of initial findings, conclusions and preliminary recommendations to USAID/Bangladesh before the Evaluation Team departs from Bangladesh; Debriefing with Partners – The team will present the major findings from the evaluation to USAID partners (as appropriate and as defined by USAID) through a PowerPoint presentation prior to the team‘s departure from the country. The debriefing will include a discussion of achievements and activities, along with recommendations for possible modifications to project approaches, results or activities. The team will consider partner comments and incorporate them appropriately in drafting the evaluation report; Debriefing with USAID/W – Presentation of evaluation findings and recommendations to USAID/W (upon USAID/Bangladesh request); timeframe will be coordinated between USAID/Bangladesh and USAID/W; Draft Evaluation Report – A draft report on the findings and recommendations should be submitted to USAID/Bangladesh within 10 working days after departure of international team members from Bangladesh. The written report should clearly describe and distinguish findings, conclusions and recommendations. USAID will provide comments on the draft report within 10 working days of submission. Final Report – The Team will submit a final report that incorporates the Mission‘s comments and suggestions no later than five working days after USAID/Bangladesh provides written comments on the team‘s draft report. The final report should meet the following criteria to ensure the quality of the report:  The mid-term evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what has worked in the project, what did not and why.  Evaluation reports shall address all evaluation questions included in the scope of work. 44 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project  The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Evaluation findings will assess outcomes and impact on males and females, as applicable to the program and as relevant data are reasonably available.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people‘s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings and conclusions.  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. The format of the final evaluation report should strike a balance between depth and length. The report will include a table of contents, lists of tables and figures (as appropriate), acronyms, executive summary, introduction, project background, purpose of the evaluation, evaluation questions, evaluation methods and limitations, analysis, findings, conclusions and recommendations. The report may include, in the annex, any substantially dissenting views by any team member or by USAID/Bangladesh on any of the findings or recommendations. The body of the report should not exceed 30 pages, excluding annexes. The report will be submitted to USAID/Bangladesh in English, electronically. A second version of this report excluding any potentially procurement-sensitive information will be submitted (also electronically and in English) by Social Impact to USAID‘s Development Experience Clearinghouse (DEC) upon USAID/Bangladesh‘s approval of the final report. Once the report is submitted to the DEC, Social Impact will inform current USAID/Bangladesh Democracy and Governance implementing partners and stakeholders (DfID, UNDP, and other donors) of its availability at the DEC web site. All quantitative data, if gathered, should be (1) provided in an electronic file in easily readable format; (2) organized and fully documented for use by those not fully familiar with the project or the evaluation; (3) owned by USAID and made available to the public barring rare exceptions. A CD with all the data could be provided to the the BDGPE COR. To protect confidentiality and privacy of survey respondents, any personal identifiers present on quantitative data will be removed prior to submission to USAID. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 45 The final report will be edited/formatted by Social Impact and provided to USAID/Bangladesh 5 working days after the Mission has reviewed the content and approved the final revised version of the report. VII. TEAM COMPOSITION AND QUALIFICATIONS USAID/Bangladesh wants to engage the services of a team of one local and two international consultants to work on the evaluation team. The evaluation team will require: 1. An international Senior Evaluation Specialist (IQC Labor Category – Program Development, Senior) with experience in evaluating legislative strengthening programs in developing countries (Team Leader). The team leader will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to USAID/Bangladesh. At least eight (8) years of experience in evaluation management is required. Experience in conducting evaluations or assessments of legislative function and processes improvement programs in developing countries are required. Ability to produce a high quality evaluation report in English is essential. 2. An international Evaluation Specialist – Mid (IQC Labor Category – Social Scientist, Mid), also with experience in evaluation of legislative strengthening programs in developing countries (Senior International Specialist); At least six (6) years of experience in democracy and governance programs and some experience managing or implementing legislative function and processes improvement programs in developing countries is required. Experience preferred in assessing: parliamentary capacity in transitional economies and democracies; the role of civil society in strengthening citizen participation and involvement in holding Parliament accountable. Knowledge of legislative strengthening literature is required. Asian/regional experience is desired. Ability to conduct interviews and discussions and write well in English is essential. 3. A Bangladesh-based Parliamentary Sector Specialist. Team member experience should include graduate level economics, social science, law, and/or public finance training. In￾depth knowledge of issues relating to Bangladeshi legislative function and process improvement programs, improving governmental institutional capacity, and fostering civil society oversight of Parliament in developing and/or transitional economies and democracies is required. Familiarity with legislative strengthening ―best practices‖ methods and programming is essential. Some experience in conducting evaluations or assessments is expected. Ability to conduct interviews and discussions and write well in English is essential. Overall the team will need expertise in USAID practices and expectations in program evaluation; program design and analysis; quantitative and qualitative data collection and analysis; survey design and analysis; program issues, innovations and challenges in building parliamentary capacity; and USAID practices and requirements in program performance measurement. 46 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project VIII. CONFLICT OF INTEREST All evaluation team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the project being evaluated. USAID/Bangladesh will provide the conflict of interest forms. IX. SCHEDULING AND LOGISTICS Funding and Logistical Support The proposed evaluation will be funded and implemented through the BDGPE project. Social Impact will be responsible for all off-shore and in-country administrative and logistical support, including identification and fielding appropriate consultants. Social Impact support includes arranging and scheduling meetings, international and local travel, hotel bookings, working/office spaces, computers, printing and photocopying. The Social Impact will make all logistical arrangements, including the vehicles for travel around Dhaka and should not expect any logistical support from the Mission. The team should also make their own arrangement on space for team meetings and equipment support for producing the report. Evaluation Schedule The evaluation is planned to follow the schedule and Level of Effort breakdown given below. Task/ Deliverable Period of Performance Review background documents & preparation work (offshore) Dec 17 – Jan 3 Travel to Bangladesh by expat team members Jan 4-5 Team Planning Meeting Jan 6 In-brief with USAID/Bangladesh and submit draft work plan Jan 7 Information and data collection  Embassy is closed on 20 Jan Jan 8-22 Evaluation team submits annotated report outline and draft presentation for USAID/Bangladesh DG team review; Jan 23 USAID/Bangladesh provides comments (as needed) on report outline and draft presentation. Jan 24 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 47 Preparation for presentation Jan 26 Presentation and debriefing with USAID/Bangladesh Jan 27 Debriefing with key stakeholders, possibly including GOB Jan 28 Expat team members depart Bangladesh Jan 29 SI submits draft report to USAID/Bangladesh team Feb 12 USAID comments on draft  Embassy closed on 17 Feb Feb 27 Team revises draft report Feb 28-Mar 6 SI delivers final report Mar 8 X. REPORTING REQUIREMENTS The total pages of the final report, excluding the executive summary, references and annexes, should not be more than 30 pages. The following content (and suggested length) should be included in the report: Table of Contents Acronyms Executive Summary - concisely state the project purpose and background, key evaluation questions, methods, most salient findings and recommendations (2-3 pp.); 1. Introduction – country context, including a summary of any relevant history, demography, socio-economic status etc. (1 pp.); 2. The Development Problem and USAID’s Response - brief overview of the development problem and USAID‘s strategic response, including design and implementation of the PRODIP project and any previous USAID activities implemented in response to the problem, (2-3 pp.); 3. Purpose of the Evaluation - purpose, audience, and synopsis of task (1 pp.); 4. Evaluation Questions – clearly stated (1 pp); 5. Program Background – background on PRODIP (1 pp); 6. Evaluation Methods and Limitations - describe evaluation methods, including strengths, constraints and gaps (1 pp.); 48 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 7. Findings and Conclusions - describe and analyze findings for each objective area using graphs, figures and tables, as applicable, and also include data quality and reporting system that should present verification of spot checks, issues, and outcomes(12-18 pp.); 8. Lessons Learned: Provide a brief of key technical and/or administrative lessons on what has worked, not worked. 9. Recommendations – prioritized for each key question; should be separate from conclusions and be supported by clearly defined set of findings and conclusions. Include recommendations for future project implementation or relevant program designs and synergies with other USAID projects and other donor interventions as appropriate (3-4 pp.). Annexes – to include statement of work, documents reviewed, bibliographical documentation, evaluation methods, data generated from the evaluation, tools used, interview lists, meetings, focus group discussion protocols (if relevant), and tables. Annexes should be succinct, pertinent and readable. Annexes should also include if necessary, a statement of significant unresolved difference of opinion by funders, implementers or members of the evaluation team on any of the findings or recommendations. The report format should be restricted to Microsoft products and 12-point type font should be used throughout the body of the report, with page margins one inch top/bottom and left/right. Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 49 ANNEX II: SOURCES OF INFORMATION Documents Reviewed Donor Strategy, Assessments, Political Analysis  USAID Bangladesh Country Development Cooperation Strategy (FY2011 – FY2016)  USAID/Bangladesh Democracy and Governance Assessment (August 2009)  USAID Field Visit Reports  DFID Parliamentary Strengthening Assessment Mission of Strengthening Political Participation (SPP) (September 2012)  DFID Strengthening Political Participation (SPP) Annual Review (September 2012)  World Bank Mid-term Assessment of Parliamentary Strengthening Programs in Bangladesh (February 2012)  PRODIP Survey Report: A Synthesis of the Findings of Three Surveys of Members of Parliament, Citizens, and Civil Society Organizations (November 2011)  Strengthening Democracy in Bangladesh, Tim Meisburger, The Asia Foundation, Occasional Paper No. 13 (June 2012) PRODIP Project Strategy, Monitoring & Evaluation, Activity Documents, Performance Reports  PRODIP Program Description  PRODIP Contract Modification (May 2012)  PRODIP Work Plans (Year 1, Year 2, Year 3)  Strategies by PRODIP CSO Partners to Engage with the Parliament  PRODIP Results Framework  PRODIP Monitoring & Evaluation Plan (January 2011 Revision)  PRODIP Indicator Reference Sheets  PRODIP Performance Indicators Tracking  Notable Achievements by PRODIP CSO Team  PRODIP Performance Monitoring Reports o Year 1 (Quarters 1, 2, 3, 4) o Year 2 (Quarters 1, 2, 3, 4) o Year 3 (Quarters 1, 2)  Parliamentary Standing Committee Effectiveness Evaluation Index 50 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project  List of PRODIP Public Policy Forum Events Attended by MPs  PRODIP Training Agendas  PRODIP Training Participant Records  Guidelines for PRODIP Grant Applications  PRODIP Guidelines for Submitting Research Proposals  PRODIP Democratic Governance Grantee Profiles Sites Visited Chittagong, Bangladesh Women‘s Leadership Training conducted by PRODIP grantee Bangladesh Nari Progati Sangha (BNPS) for District Public Policy Forum (DPPF) members (January 18) Dhaka, Bangladesh  National Seminar on Maternal Health conducted by DPPF members with support from Sabalamby Unnayan Samity (SUS), a PRODIP democratic governance grantee (January 12)  CSO Coordination Meeting conducted by PRODIP for democratic governance grantees (January 15-16)  Sharing Meeting on Policy Briefs and Advocacy Issues conducted by DPPF members with support from the Institute for Environment and Development (IED), a PRODIP democratic governance grantee (January 21) Savar, Bangladesh Dissemination of education policy research conducted by the Department of Anthropology at Jahangirnagar University, a PRODIP democratic governance research grantee (January 8) Individuals Interviewed Parliamentary Contacts Col. Shawkat Ali (Retd.), Deputy Speaker, Jatya Sangsad Rowshan Jahan Sathi, Secretary, All Party Parliamentary Women's Caucus Chadni Roy, Intern, Chief Whip‘s Office Abhijit Kumar Saha, Intern, Finance Public Estimate, Committee 3 Mojibul Haq, Secretary, Jatiya Party Caucus Mahbubul Alam, Joint Secretary, Committee Support, Jatya Sangsad Md. Abdul Ahad Biswas, Intern, Law Officer Room, Government Assurance Shampad Barua, Additional Secretary, Parliament Secretariat Joinul Abedin, Private Secretary to Honorable Speaker, Parliament Secretariat Jebunnesa, Librarian, Parliament Secretariat Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 51 Zamila Kulsum, Assistant Librarian, Parliament Secretariat Mohammed Iqbal, Assistant Librarian, Parliament Secretariat Shaheenor Alam, Assistant Librarian, Parliament Secretariat Ali Akbar, Assistant, Research and Education, Parliament Secretariat Begum Farida Parvin, Deputy Secretary, Parliament Secretariat Shahadat Hossain, Senior Assistant Secretary, Parliament Secretariat Rezai Karim, Director, Reporting, Parliament Secretariat Obaidur Rahman, Deputy Director, Reporting, Parliament Secretariat Abu Al Helal, Deputy Secretary, Parliament Secretariat M. M. Fazlur Rahman, Legislative Draftsman, Parliament Secretariat A. H. M. Abdul Wadud, Senior Legislative Draftsman, Parliament Secretariat Md. Mahfuzur Rahman, Secretary, Parliament Secretariat Kamrunnahar Stella, Intern, Parliament Visit Section; Parliamentary Secretary's Office Shampad Barua, Additional Secretary, Parliamentary Affairs A.B.M. Abul Qasem, Chairman, Parliamentary Standing Committee on Ministry of Commerce Iqbal Bin Matin, Private Secretary to Chair, Parliamentary Standing Committee on Ministry of Commerce Hemayet Hussain, Private Secretary to Chair, Parliamentary Standing Committee on Ministry of Agriculture Shawkat Momen Shahjahan, Chairman, Parliamentary Standing Committee on Ministry of Agriculture Rashed Khan Menon, Chairman, Parliamentary Standing Committee on Ministry of Education Ferdous Rahaman, Intern, Parliamentary Standing Committee on Ministry of Education Md. Shah Alam, Member, Parliamentary Standing Committee on Ministry of Education Ziaur Rahman, Member, Parliamentary Standing Committee on Ministry of Education Shri Biren Sikder, Member, Parliamentary Standing Committee on Ministry of Education Naimul I. Khan, Personal Secretary, Parliamentary Standing Committee on Ministry of Education Shah Alam, Member, Parliamentary Standing Committee on Ministry of Education Sirajul Islam Badsha, Senior Committee Officer, Parliamentary Standing Committee on Ministry of Education A.B.M. Ashraf Uddin Nizan, Chairman, Parliamentary Standing Committee on Ministry of Fisheries and Livestock Samira Rahat Mohona, Intern, Parliamentary Standing Committee on Ministry of Labour and Employment 52 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Abu Hena Zaman, Private Secretary to Chair, Parliamentary Standing Committee on Ministry of Labour and Employment Saiful Islam Bhauiyan, Private Secretary to Chair, Parliamentary Standing Committee on Ministry of Planning Abdullah Shahed Miaji, Intern, Parliamentary Standing Committee on Ministry of Power, Energy, and Mineral Resources Alhaz Momotaz Begum, Chair, Parliamentary Standing Committee on Ministry of Primary and Mass Education Dr. Md. Mozammel Hossain, Chairman, Parliamentary Standing Committee on Ministry of Social Welfare K. N. Mahfuza Khatun, Member, Parliamentary Standing Committee on Ministry of Social Welfare Chemon Ara Begum, Member, Parliamentary Standing Committee on Ministry of Social Welfare Baby Moudud, Member, Parliamentary Standing Committee on Ministry of Social Welfare Professor Dr. M. A. Mannan, Member, Parliamentary Standing Committee on Ministry of Women and Children Affairs Fulguni Reza, Intern, Public Accounting Department, Committee Shaka 1 Nusrat Jaman Mini, Intern, Public Accounting Department, Committee Shaka 1 Civil Society Contacts Rajkumar Shaw, Executive Director, Adibashi Sangskritik Unnayan Songstha (ASUS) Mahbub Alam, Project Coordinator, Adibashi Sangskritik Unnayan Songstha (ASUS) Nasim Firdaus (Amb. Ret'd), Executive Director, Bangladesh Alliance for Women Leadership (BDAWL) Saquib Chowdhury, Advisor, Bangladesh Alliance for Women Leadership (BDAWL) Farooq Sobhan, President, Bangladesh Enterprise Institute (BEI) Shahab Enam Khan, Bangladesh Enterprise Institute (BEI) Abdullah Al Hasan, Director of Projects, Bangladesh National Women Lawyer‘s Association (BNWLA) Selima Ahmed, President, Bangladesh Women Chamber of Commerce and Industry (BWCCI) Md. Mojibur Rahaman, Acting CEO, Head of Program, Bangladesh Women Chamber of Commerce and Industry (BWCCI) Syed Sultan Chand, Project Manager, Bangladesh Women Chamber of Commerce and Industry (BWCCI) Majeda Begum Siru, DPPF Member, BNPS-supported DPPF Shila Das, DPPF Member, BNPS-supported DPPF Hosna Ara Begum, DPPF Member, BNPS-supported DPPF Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 53 Nazma Begum, DPPF Member, BNPS-supported DPPF Kamal Sengupta, Deputy Executive Director, Community Development Centre (CODEC) Syeed Ahamed, Chief Operating Officer, D.Net/Institute of Informatics and Development (IID) Ashiq Iqbal, Research Fellow, D.Net/Institute of Informatics and Development (IID) Taleya Rehman, Executive Director, Democracywatch Minhaj Alam, Program Director, Democracywatch Feroze Nurun-Nabi Jugal, Program Coordinator, Democracywatch Dr. Siddiqur Rahman, Associate Professor, Department of Anthropology, Jahangirnagar University Dr. Akther Hossian, Professor, Department of Public Administration, University of Dhaka Sanjeeb Drong, General Secretary, Indigenous Peoples Development Services (IPDS) Numan Ahmed Khan, Executive Director, Institute for Environment and Development (IED) Jyoti Chattapadhyaya, Project Coordinator, Institute for Environment and Development (IED) Khaled Golam Mortuza, Monitoring Officer, Institute for Environment and Development (IED) Hasibur Rahman Mukar, Executive Director, Management and Resources Development Initiative (MRDI) Sanker Ranjan Sarker, Chief Coordinator, Nabolok Mostafizur Rahman Setu, Project Coordinator, Nabolok Mahfuz Kabir, Shamunnay Abdullah Nadvi, Junior Consultant, Shamunnay Rashidul Azad, Senior Research Associate, Shamunnay S. M Sirajul Islam, Project Manager, Speed Trust Rehana Siddique, DPPF Convener, SUS-supported DPPF Mostafizur Rahman, DPPF Secretary, SUS-supported DPPF Shaymalendo Pal, DPPF Member, Journalist, SUS-supported DPPF Mohsin Ali, Executive Director, WAVE Foundation Tapan Saha, Assistant Executive Director, WAVE Foundation Md. Shaheen Hossen Gazi, Project Coordinator, WAVE Foundation Donor and Implementer Contacts Russell Pepe, Chief of Party, PRODIP Niamh O‘Connor, Deputy Chief of Party, PRODIP Catherine Cecil, Parliamentary Program Director, PRODIP Sheela Tasneem Haq, Civil Society Program Director, PRODIP 54 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Hasan Mazumdar, Country Representative, The Asia Foundation (TAF) Md. Mafizur Rahman, Senior Program Officer, Monitoring & Evaluation PRODIP Kazi Maruful Islam, Director, Research, PRODIP Md. Anwar Hossain, Grant Manager, PRODIP Helen Barnes, Governance Adviser, Department for International Development (DFID) Henrick van Asch van Wijck, First Secretary Political Affairs, Ministry of Foreign Affairs (Netherlands) Niall Johnston, Lead on Parliamentary Oversight, Strengthening Parliamentary Oversight Project (SPO) Marcia Monge, Chief Technical Officer, United Nations Development Programme (UNDP) Md. Mozaharul Islam Khan, Senior Program Officer, United Nations Population Fund (UNFPA) Tapati Saha, Senior Consultant, UN Women Alexious Butler, Director, Democracy and Governance Office, USAID/Bangladesh Billy Woodward, Democracy and Governance Advisor, Democracy and Governance Office, USAID/Bangladesh Rumana Amin, Monitoring and Evaluation Specialist, Democracy and Governance Office, USAID/Bangladesh Farheen Khurrum, Senior Program Development Specialist, Monitoring & Evaluation and Gender Program Office, USAID/Bangladesh Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 55 ANNEX III: EVIDENCE MATRIX Gender Implications? Implementer Materials Secondary Materials Implementer Personnel Direct Participants or Beneficiaries Indirect Beneficiaries External Experts, Non-Involved Mini-Survey Objective 1: An Effective & Open Parliament Result 1: Improved Parliamentary Lawmaking Skills & Oversight Capacity A1.1 - MP training on legislative research & bill drafting Guide X MPs Cmte staff 'Private Members Bills', Sec staff A1.2 - Staff training on research, reports, & drafting DB, evals X Staff (esp trainers) Cmte chairs, core CSOs A1.3 - Support policy research G X Briefings, memos X Researchers, testimony experts Target cmte MPs/staff Non-target cmte MPs/staff A1.4 - Establish research/documentation services in parliament X Budget X Interns, mentors, target cmte MPs/staff Non-target cmte MPs/staff A1.5 - Parliamentary internship program G Selection criteria, evals X Interns, mentors Supervising staff 56 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project A1.6 - Party caucasus on policy issues G X Agendas, policy ex X Relevant target cmte MPs/staff A1.7 - External advisory groups G X Agendas X CSOs, party reps Other CSOs A1.8 - Study tour (CAN) Agendas X JS leaders, target cmte chairs Target cmte MPs/staff A1.9 - APPG policy dialogues X Agendas X MPs ?Oversight cmtes? A1.10 - Women's participation in law/policy formation G X Agendas, plans X Women's Parl'ty Cauc Relevant target cmte MPs/staff Relevant non-target cmte MPs/staff A1.11 - New MP orientation JS Sec Result 2: Stronger Parliamentary Committee System A2.1 - Technical support for committees Agendas, evals X Target cmte MPs/staff JS Sec staff Non-target cmte MPs/staff A2.2 - Committee annual work plans X Plans, ∆RoP X Sr Speclsts, target cmte chairs/staff Non-target cmte MPs/staff A2.3 - Committee member capacity building Agendas, evals X Target cmte MPs/staff Non-target cmte MPs/staff A2.4 - Committee oversight capacity building X Site visit report X Target cmte MPs/staff Non-target cmte MPs/staff Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 57 A2.5 - Study tour (IND) G Agenda X JS leaders, target cmte chairs A2.6 - Committee activity briefs X Briefing books X Target cmte MPs/staff Non-target cmte MPs/staff Result 3: Increased Public Awareness/Participation in Parliamentary Functions/Processes A3.1 - Parliament communications strategy Agendas Strategy X JS Sec A3.2 - MP media outreach toolkit G Toolkit X Target cmte MPs/staff Media, non￾target cmte MPs/staff A3.3 - Expert testimony capacity building G X Issue briefs X Target cmte MPs/staff, CSOs Non-target cmte MPs/staff A3.4 - Public hearings G Guides, list X Target cmte MPs/staff CSOs A3.5 - Youth in Parliament G Agendas X MP, youth A3.6 - Committee outreach to local/regional levels G X X Target cmte MPs/staff, DPPFs A3.7 - ICT support X Updates X JS Sec 58 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Objective 2: More Constructive & Sustainable Role for CS in DG Result 4: Increased Institutional Capacity of CSOs to Advance Public Participation A4.1 - DG grants to CSOs G List Project results X CSOs R5, CSOs R6 A4.2 - Technical support to CSOs on policy research Manuals, toolkits X CSOs R5, CSOs R6 A4.3 - Organizational development of CSOs Agendas X CSOs R5, CSOs R6 Other donors A4.4 - Coordination/sharing meetings Agendas X CSOs R5, CSOs R6 A4.5 - Regional study/exchanges Agendas X CSOs R5, CSOs R6 Result 5: More Active Role for CSOs/Research in Public Policy A5.1 - Policy research/advocacy initiatives G Agendas, guides, toolkits Research materials X CSOs, research instns Target cmte MPs/staff Media A5.2 - Parliament 'watch' activities [new activity] A5.3 - Facilitate direct democracy in select districts G X X DPPFs, CSOs, Target cmte MPs/staff Other donors DPPFs A5.4 - Promote DPPF gender inclusion G X X DPPFs, CSOs, Target cmte MPs/staff Other donors DPPFs Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 59 Result 6: More Active CSO/Media Roles in Educating Public A6.1 - Surveys, assessments, other research X A6.2 - Interactive PRODIP website Traffic stats X JS Sec, target cmte MPs/staff, core CSOs A6.3 - National information campaigns Examples X JS Sec A6.4 - National media outreach Examples X A6.5 - PSAs on parliament, policy, etc G Examples, viewership stats X 60 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Implementer Materials Secondary Materials Implementer Personnel Direct Participants or Beneficiaries Indirect Beneficiaries External Experts, Non-Involved Mini-Survey Evaluation Question B - Management & Administration Useful data from PRODIP's performance management system? X X PRODIP's M&E used/valued by PRODIP/stakeholders? X Cites, examples X JS Sec, target cmte MPs/staff, core CSOs, CSO grantees Non-target cmte MPs/staff, other donors TAF-SUNY collaboration? X X DfID, JS Sec, target cmte MPs/staff, core CSOs CSO grantees Other donors PRODIP-otherLS coordination? Memos X Other donors JS Sec Evaluation Question D - Program Opportunities Cost-effective way to increase citizen confidence? X Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 61 Evaluation Question E - Program Theory & Design Incorporation of §5.3.2 of 2009 DG SAF? X X USAID Other donors Evaluation Question G - Cross-Cutting Issues How well are gender issues addressed by PRODIP? X X 62 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project ANNEX IV: SEMI-STRUCTURED INTERVIEW PROTOCOLS Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved Objective 1: An Effective & Open Parliament Result 1: Improved Parliamentary Lawmaking Skills & Oversight Capacity A1.1 - MP training on legislative research & bill drafting What topics did the training cover? Could you give a specific example of how you have used the skills in legislative drafting? Could you give a specific example of how you used research to shape legislation? Did the training meet your needs? What else might you need? Do you feel that you could draft legislation and reference research? How would you describe the legislative drafting skills of MPs on [names] committees? Could you give a specific example of a well-drafted bill by one of the committees? Could you give a specific example of research used in legislative drafts? Do you think that these committees will be able to produce well-drafted bills based on research in the next parliament? How would you describe the legislative drafting skills of MPs on your committee? Could you give a specific example of a well￾drafted bill by one of the committees? Could you give a specific example of research used in legislative drafts? A1.2 - Staff training on research, reports, & drafting What topics did the training cover? Could you give a specific How would you describe the legislative drafting skills of staff on [names] How would you describe the legislative drafting skills of staff on your committee? Could you give a specific example of a well- Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 63 Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved example of how you have used the skills in legislative drafting? Could you give a specific example of how you used research to shape legislation? Did the training meet your needs? What else might you need? Do you feel that you could draft legislation and reference research? committees? Could you give a specific example of a well-drafted bill by one of the committees? Could you give a specific example of research used in legislative drafts? Do you think that these committees will be able to produce well-drafted bills based on research in the next parliament? drafted bill by one of the committees? Could you give a specific example of research used in legislative drafts? A1.3 - Support policy research Were the legal memos written by lawyers provided by PRODIP useful to you in drafting legislation? A1.4 - Establish research, documentation services in parliament What topics did the training cover? Could you give a specific example of how you have used the skills from the training? Did the training meet your needs? What else might you need? Could you give an example of Is the research/documentation service of more or less help for you than the past? 64 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved how digitization has helped your work? A1.5 - Parliamentary internship program How were you selected? Did you have a clear understanding of what work you would be doing? Did you a clear understanding of parliamentary processes? Could you give an example of what you have learned about parliament? How will this help you professionally? Did you have a clear understanding of what work the intern would be doing? Could you give a specific example of how the intern helped the work of your committee? A1.6 - Party caucuses on policy issues How was your party caucus established? Would it persist without PRODIP’s facilitation? Would this have happened on its own anyway? What value does the caucus have to you and other members? How has the caucus contributed to committee work or policy discussions? What do the BNP and JT caucuses contribute to policy discussions? Could you give a specific example of how a party caucus has contributed to your committee’s work? Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 65 Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved A1.7 - External advisory groups Has your committee engaged with any external groups for advice or research on particular policy issues? A1.8 - Study tour (CAN) Could you give a specific example of a committee practice or procedure that you would like to implement here? A1.9 - APPG policy dialogues How was the APPG established? Would it persist without PRODIP’s facilitation? Would this have happened anyway? What value does the caucus have to you and other members? How has the caucus contributed to policy discussions and the work of committees? What do the APPGs contribute to policy discussions? Could you give a specific example of how a party caucus has contributed to your committee’s work? A1.10 - Women's participation in law/policy formation Do you feel that the APPG has raised issues of importance to women? Would this have happened anyway? Could you give an example of how [specific woman MP on committee] has contributed to the committee’s work because of her activity with 66 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved How has this experience benefitted you professionally? Was the action plan you developed in Washington implemented? Could you develop another action plan again? the APPG? A1.11 - New MP orientation [New activity for next program year] Result 2: Stronger Parliamentary Committee System A2.1 - Technical support for committees What topics did the training cover? Could you give a specific example of how you have used the skills? Did the training meet your needs? What else might you need? A2.2 - Committee annual work plans Does your committee have a work plan? Is the plan being followed, or have there been changes? Could your committee update a work plan for another period of time? How have [names] committees’ work been affected by following a work plan? Are those committees capable of updating a work plan without PRODIP’s Does your committee have a work plan? Would a work plan be valuable to your committee’s work? Do you know of committees that have a work plan, and has this helped them? Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 67 Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved Would your committee have developed a work plan without PRODIP’s assistance? assistance? A2.3 - Committee member capacity building What topics did the training cover? Could you give a specific example of how you have used the skills in report writing? Did the training meet your needs? What else might you need? Do you feel that you could write committee reports? Which committees produce reports? What is the quality of the reports? A2.4 - Committee oversight capacity building Was the training by Tony Worthington on committee report writing useful to you? Could you give a specific example of how you have used these skills? Have [names] committees been submitting reports? Are they useful to you for understanding the work of that committee? Does your committee submit written reports when it conducts oversight activities? A2.5 - Study tour (CAN) [see Activity 1.8] A2.6 - Committee activity briefs [new activity for next program year] 68 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved Result 3: Increased Public Awareness/Participation in Parliamentary Functions/Processes A3.1 - Parliament communications strategy Was the communication strategy assessment an inclusive and useful process? Has PRODIP contributed ideas of how to change Sangsad TV, and how have those aligned with parliament’s strategy? Have any of the changes improved the quality of Sangsad TV? Coud you give a specific eample of how the communication strategy assessment has been translated into action? A3.2 - MP media outreach toolkit What topics did the workshop cover? Could you give a specific example of how you used information from the workshop? Did the workshop meet your needs? What else might you need? Are MPs better prepared, more open than before? Could you give a specific example of an MP/s that have improved their media interaction? A3.3 - Expert testimony capacity building How have experts helped the development of legislation in your committee? How would you find experts? Could you give an example of how expert testimony has helped the work of a committee? Does your committee use policy experts to help the development of legislation? How would you find experts? Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 69 Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved Would your committee be able to use experts without PRODIP’s assistance? A3.4 - Public hearings Did the materials provided by PRODIP on holding public hearings help your committee? How useful was the public hearing? Did it influence your committee’s view on a policy issue? Would your committee be able to conduct a public hearing without PRODIP’s assistance? Could you give a specific example of how a public hearing has influenced legislation or public policy? Are the committees that have held public hearings capable of doing so in the future without PRODIP’s assistance? Has your committee ever conducted a public hearing? Would public hearings be useful to your committee? How would your committee go about conducting a public hearing? A3.5 - Youth in Parliament Are you satisfied with the content of the booklet on parliament, and its strategy for use in schools? A3.6 - Committee outreach to local/regional levels Have you participated in any events with DPPFs? What policy issues were discussed? How did that discussion influence your thinking on that policy issue? 70 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved A3.7 - ICT support Are you satisfied with PRODIP’s strategy and plans? A3.8 – Constituency Offices [new activity for next program year] A3.10 – Visitors’ Center [new activity for next program year] Objective 2: More Constructive & Sustainable Role for CS in DG Result 4: Increased Institutional Capacity of CSOs to Advance Public Participation A4.1 - DG grants to CSOs How would you describe the grant selection process? Does your CSO’s/DPPF’s mission relate to policy topics discussed in parliamentary or local government? How? Could you give a specific example of your CSO’s/DPPF members’ prior interactions with policymakers? Would your public policy advocacy activities have occurred without support from PRODIP? Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 71 Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved A4.2 - Technical support to CSOs on policy research What materials or training has PRODIP provided to you on bringing policy research to parliamentary committees, local MPs, or other elected officials? How have you been following the M&E plan developed with PRODIP assistance? What share of the design and implementation comes from your CSO/DPPF and from PRODIP/CSO? What additional training or materials would be useful to your CSO/DPPF? What CSOs conduct research relevant to your committee’s policy issues? Have CSOs brought policy research to your attention in the past? A4.3 - Organizational development of CSOs Has your CSO made any organizational changes as a result of PRODIP training, field visits, or a one-year review? How would you continue funding of these activities without PRODIP grants? A4.4 - Coordination/sharing meetings Has your CSO planned or led a coordination meeting with other PRODIP grantees? 72 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved Did you have any prior collaborative activities with other PRODIP grantees? Have you attended any of the information sharing meetings? Have you used any lessons learned from other PRODIP grantees? What have you learned about the policy research conducted by other organizations? Are there other ways or opportunities for collaboration among PRODIP grantees? A4.5 - Regional study/exchanges Did you meet with any civil society experts from India, Nepal, or Pakistan during their recent visit? Have you used any lessons learned from the India, Nepal, and Pakistan? Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 73 Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved Result 5: More Active Role for CSOs/Research in Public Policy A5.1 - Policy research/advocacy initiatives Have you participated in any DPPF meetings? What does your activity intend to achieve? Who is the target audience for your activity? How do you envision your policy briefs being used? What interaction do you have with policymakers? What CSOs conduct research relevant to your committee’s policy issues? Have you attended any policy advocacy events by [any of the six grantees]? A5.2 - Parliament 'watch' activities [new activity] A5.3 - Facilitate direct democracy in select districts How was the DPPF established? How were DPPF members selected? Do MPs or local elected officials ever attend meetings? Could you give specific examples of policy issues discussed at DPPF meetings? How did you apply skills from the advocacy training? Are there ways or opportunities for How did you come into contact with the DPPF? Could you give specific examples of policy issues discussed at DFFP meetings? Does the DPPF contribute to your understanding of a policy issue? Has it affected policy in your constituency or committee? 74 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved collaboration among DPPFs? What would happen to the DPPF if the CSO did not receive a grant? Has your relationship with policymakers expanded or strengthened? A5.4 - Promote DPPF gender inclusion What percentage of women are on your DPPF? What role do they play on the DPPF? How have you applied skills from leadership training to your DPPF activities? Do women MPs or local officials participate in DPPF activities? How do your DPPF’s objectives address gender issues? Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 75 Direct Participants or Beneficiaries Indirect Beneficiaries External or Non-Involved Result 6: More Active CSO/Media Roles in Educating Public A6.1 - Surveys, assessments, other research [nothing done beyond initial baselines that were not publicized] A6.2 - Interactive PRODIP website How do you use the website? How could the website be changed to be more useful? A6.3 - National information campaigns How has your media content improved policy advocacy? Have you received feedback from policymakers based on your media content? A6.4 - National media outreach [concurrent with Activity 6.3] 76 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project ANNEX V: MINI-SURVEY PROTOCOL Social Impact is conducting a mini-survey of at least 30 District Public Policy Forum (DPPF) leaders from among 140 DPPFs at the district, upazila, and union levels. Respondents will be chosen to emphasize geographic diversity. Your responses are confidential, and your name will not be used in the evaluation report. Name: ____ Male / ____ Female Contact Information: Location: Occupation: CSO Support From: Member of: _____ District Public Policy Forum (DPPF) _____ Upazila public policy forum _____ Union public policy forum 1. Prior to PRODIP, were you involved in a community group that communicated directly with policymakers to advocate for policy changes? ____ Yes (1) ____ No (0) 2. Does your DPPF‘s mission relate to policy topics discussed in parliament/local government? If yes, please provide an example. ____ Yes (1) ____ No (0) 3. How do the objectives of your public policy forum address gender issues? 4. Do MPs or local elected officials ever attend your meetings? ____ Never (0) ____ Sometimes (1) ____ Frequently (2) ____ Always (3) 5. Have your relationships with policymakers been expanded or strengthened as a result of your involvement in the PRODIP-supported public policy forum? Please describe. ____ Much less than before (-1) ____ Same as before (0) ____ Much more than before (+1) 6. Do you follow up with policymakers between formal DPPF meetings? If yes, how? ____ Yes (1) ____ No (0) 7. Could your public policy advocacy activities continue without support from PRODIP? Why/not? ____ Yes (1) ____ No (0) Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 77 জে঱া পাবল঱ক পল঱ল঴ জ ারামঃ লমলি ঴ার্ভে প্রর্ াক঱ জ঴াল঴য়া঱ আ ১৪০ টি লিলপলপএ এর মর্যে জে঱া, উপর্ে঱া, উআলিয়ি পযোয় জের্ক িূেিতম ৩০ েি লিলপলপএ জিতার্ের উপর ঴ংলিপ্ত েলরপ পলরচা঱িা করর্ে। জভৌগল঱ক বোলপ্ত লবর্বচিা কর্র উত্তরোতা লিবোচি করা ঵র্ব। ( অপিার মতামর্তর ঴ম্পূর্ে জগাপিীয়তা রিা করা ঵র্ব, প্রলতর্বের্ির জকাোয়ও অপিার িাম বেব঵ালরত ঵র্ব িা।) িামঃ ____ পুরু঳/ ___ মল঵঱া জযাগার্যার্গর ঠিকািাঃ লিে জে঱া/ উপর্ে঱াঃ জপলাঃ ঴ংলিস্ট ল঴এ঴ওঃ অপলি জকাি জ ারার্মর ঴ে঴েঃ _________ জে঱া পযোয় __________ উপর্ে঱া পযোয় __________ আউলিওি পযোয় ১। প্রেীপ-লিলপলপএ এর পূর্বেঅপলি ঄িে জকাি ঴ামালেক গ্রুর্পর ঴ার্ে েলরত লের্঱ি লক , যা লকিা পল঱ল঴ পলরবতে র্ির উর্ের্লে ঴রা঴লর িীলত লিযোরকর্ের ঴ার্ে জযাগার্যাগ রিা কর্রলে঱? _____ ঵ো, ______ িা । ২। অপিার্ের লিলপলপএ টির ঱িে এমি লকেুলক, যা পল঱ল঴ অর্঱াচে লব঳য় ল঵র্঴র্ব ঴ং঴ে/ স্঵ািীয় ঴রকার বেবস্঵াপিায় অর্঱ালচত ঵য়? _____ ঵ো, ______ িা । উত্তর ঵ো ঵র্঱, উো঵রি লেি। ৩। অপিার্ের লিলপলপএ টির উর্েলে লকভার্ব জেন্ডার/ িারীর ঄ংলগ্র঵ি লব঳য়টির্ক প্রাযািে লের্ে? ৪। অপিার্ের লিলপলপএ লমটিং এ কখি লক এমলপ ঄েবা স্঵ািীয় ভার্ব লিবোলচত জকাি প্রলতিীলয ঄ংলগ্র঵ি কর্রর্ে? _____কখিও িা _____ কখিও কখিও _____ঘিঘি _____঴ব ঴ব঴ময় ৫। প্রেীপ ঄েোয়র্ির জ ারার্ম ঴ংলিস্ট ঵র্য় অপিার্ের লক িীলতলিযোরকর্ের ঴ার্ে জযাগার্যাগ জোরা঱ বা ললিলা঱ী ঵র্য়র্ে ? _____ পূর্বের জচর্য় কম _____ পূর্বের মর্তাআ _____ পূর্বের জচর্য় জবলল ৬। অিুস্ঠালিক লিলপলপএ লমটিং এর পর িীলতলিযোরকর্ের জ ার্঱া অপ/ পযের্বিি কর্রর্েি লক ? _____ ঵ো, ______ িা । উত্তর ঵ো ঵র্঱, লক ভার্ব ? ৭। প্রেীপ এর ঴঵র্যালগতা বোতীত অপিার্ের জ ারার্মর এোির্ভার্কল঴ কাে চ঱র্ত পারর্ব লক ? _____ ঵ো, ______ িা । উত্তর ঵ো ঄েবা িা ঵র্঱, লক ভার্ব বোখো করুি ? 78 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project ANNEX VI: DISCLOSURE OF CONFLICT OF INTEREST FOR USAID EVALUATION TEAM MEMBERS Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 79 80 Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project Midterm Performance Evaluation of the Promoting Democratic Institutions and Practices (PRODIP) Project 81 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523