1 Performance Evaluation of USAID’s Promoting Governance, Accountability, Transparency, and Integrity (PROGATI) Project December 2012 This publication was produced for review by the United States Agency for International Development. It was prepared independently by Corbin Lyday and Owen Lippert of Social Impact, Inc.; Saiful Islam of Research, Training, and Management, Inc.; and Mohammed Sirajul Islam of BRAC University’s Institute of Governance Studies under USAID’s Bangladesh Democracy and Governance Program Evaluations (BDGPE) Project. PERFORMANCE EVALUATION OF USAID’S PROMOTING GOVERNANCE, ACCOUNTABILITY, TRANSPARENCY AND INTEGRITY (PROGATI) PROJECT EVALUATING THE SUCCESS OF THE PROGATI ANTI￾CORRUPTION PROJECT IN BANGLADESH December 9, 2012 Contract Number - AID-OAA-I-10-00003 Task Order Number - AID-388-TO-12-00001 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS Executive Summary ……………………………………….…………………………………………….1 The Development Problem and USAID’s Response………………………………………………1 Evaluation Purpose…………………………………………………………………………………1 Evaluation Questions………………………………………………………………………………1 Evaluation Methodology……………………………………………………………………………1 Findings…………………………………………………………………………………………..…2 Conclusions and Lesson Learned………………………………………………………………..…3 Recommendations………………………………………………………………..………………...3 Introduction …………………………………………………………………………………………….5 The Development Problem and USAID's Response ……………………………………………………….6 Purpose of the Evaluation…………………………………………………………………………………………..7 Evaluation Questions………………………………………………………………………………7 Evaluation Methodolody…………………………………………………………………………………………….9 Key Informant Interviews………………………………………………………………………….9 Focus Group Discussions………………………………………………………………………….9 Cross-Tabulation between KIIs and FGDs………………………………………………………..9 Methodological Limitations……………………………………………………………………...10 Findings………………………………………………………………………………………………………………11 Results…………………………………………………………………………………………….11 Program Theory and Design and Program Opportunities……………………………………….20 Implementation and Management Issues…………………………………………………………22 Cost Effectiveness………………………………………………………………………………...24 Sustainability………………………………………………………………………………………24 Crosscutting Issues……………………………………………………………………………….25 Client Satisfaction—Inconsistencies Amid Successes……………………………………………26 Conclusions and Lessons Learned……………………………………………………………………………….27 Results…………………………………………………………………………………………….27 Program Design………………………………………………………………………………...…28 Media……………………………………………………………………………………………...28 Recommendations …………………………………………………………..…………………………………….29 Annexes …………………………………………………………………………………………………………...31 - Annex A: Evaluation Statement of Work …………………………………………………………………...32 - Annex B: List of Documents Reviewed/Bibliography ……………………………………………..……...41 - Annex C: Data Collection Instruments and Approach …………………………………………....……...48 - Annex D: Focus Group Discussion Results …………………………………….…………………....……...50 - Annex E: List of Persons Interviewed for KIIs …………………………………….…………………........54 - Annex F: List of Focus Group Discussions and Participants …………………………………………….56 - Annex G: Disclosure of Any Conflicts of Interest ……………………………………………………….….57 - Annex H: Letter from Alexious Butler to Mahfuzur Rahman, Parliament Secretariat ……..........62 - Annex I: Focus Group Discussions’ Attendance Lists …………………………………………………….65 - Annex J: PROGATI Targets and Achievements ……………………………………………………………74 TABLES 1.PROGATI Project PMP Targets and Achievements ………………………………………………….11 II. PROGATI Training: Indicator Actuals Through June 2012. …………………………………………….15 III.PROGATI Targets and Achievements Project Years One through Five………………………………75 ACRONYMS ACC Anti-Corruption Commission ADB Asian Development Bank CAG Comptroller and Auditor General AL Awami League BAMU Budget and Monitoring Unit BDGPE Bangladesh Democracy and Governance Program Evaluations BEI Bangladesh Enterprise Institute BNP Bangladesh Nationalist Party BNNRC Bangladesh NGOs Network for Radio and Communication BRAC IGS Bangladesh Rural Advancement Committee Institute of Governance Studies BRDG Building Recovery and Reform through Democratic Governance BWCC Bangladesh Women’s Chamber of Commerce CIDA Canadian International Development Agency COP Chief of Party CSO Civil Society Organization CTG Caretaker Government DAI Development Alternatives, Inc. DANIDA Danish International Development Agency DFID Department for International Development (UK) DG Democracy and Governance DI Democracy International FAPAD Foreign Aided Project Audit Directorate FGD Focus Group Discussion FY Fiscal Year GOB Government of Bangladesh IC Information Commission INTOSAI International Organization of Supreme Audit Institutions IQC Indefinite Quantity Contract JATRI Journalism Training and Research Initiative KII Key Informant Interview MOU Memorandum of Understanding MP Member of Parliament MRDI Management and Resources Development Initiative NGO Non-Governmental organization NGOAB Non-Governmental Organization Affairs Bureau OCAG Office of the Comptroller and Auditor General OIG Office of Inspector General (US) PMP Performance Monitoring Plan PROGATI Promoting Governance, Accountability, Transparency and Integrity RTI Right to Information Act SCOPE Strengthening Comptrollership and Oversight of Public Expenditure SOW Scope of Work STTA Short-Term Technical Advisor TA Technical Assistance TAC Technical Advisory Committee TEC Total Estimated Cost TI Transparency International TOT Training of Trainers UP Union Parishad UNDP United Nations Development Programme USAID United States Agency for International Development USG United States Government VGD Vulnerable Group Development Program 1 EXECUTIVE SUMMARY INTRODUCTION Due to a lack of accountability, transparency, and access by the public, the Government of Bangladesh (GOB) has struggled since independence to successfully establish democratic practices, political stability, and major economic development. Additionally, corruption on the federal level has prevented growth and has exposed the country to vulnerabilities such as social upheaval and instability. However, Bangladesh is a rapidly growing and developing country with the seventh largest population and fourth largest Muslim population in the world, which makes the goal of reducing corruption a priority for the United States. Promoting greater transparency and accountability in public resource management is essential to obtaining this goal and enhancing the development potential of Bangladesh. THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE In response to this problem, the United States Agency for International Development (USAID) began contracting with Development Alternatives, Inc. (DAI) in 2007 to implement a four-year, $18 million anti-corruption program. The program was intended to strengthen parliamentary oversight committees, support the legislative strengthening of the Parliamentary Budget Unit, increase citizen group action and civil society development, build the capacity of the mass media, and create a dissemination plan to promote access to information. In 2011, the task order was extended through September, 2012, increasing the total cost to more than $19 million. EVALUATION PURPOSE The purpose of this end-line evaluation of the Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Project was to assess the results and outcomes of the USAID-funded anti￾corruption initiative, and identify lessons that could be taken from the five-year program for use in designing future interventions of this nature. The specific goals of the evaluation were to evaluate PROGATI’s overall performance, assess the efficacy of implementation, and make recommendations for future interventions to USAID/Bangladesh. EVALUATION QUESTIONS The evaluation Scope of Work (SOW) outlines evaluation questions focused on the following areas: project results; project design and implementation; the cost effectiveness of the project’s interventions; how well the project supported women, the disabled, and minorities; client satisfaction; sustainability of project results; and future program opportunities. EVALUATION METHODOLOGY The evaluation team used a mixed-methods approach for the evaluation, which included a desk review, key informant interviews (KIIs), and focus group discussions (FGDs). The documents included in the desk review are located in Annex B, and the instruments used in the KIIs and FGDs are located in Annex C. 2 The team conducted KIIs with 25 individuals who were either involved with designing the project, are Democracy and Governance (DG) sector stakeholders, or are public opinion leaders in the country (such as academics, journalist, barristers, researchers, and policy analysts). KII questions focused on the performance, effectiveness, and impact of PROGATI. The evaluation team used FGDs to learn more about the external perspectives of PROGATI, USAID, and key DG stakeholders. The FGDs were intended to complement the KIIs, which focused almost exclusively on internal perspectives. The team conducted three FGDs—two in Dhaka and one in Chittagong. The conversations in Dhaka included experts familiar with PROGATI, and the conversation in Chittagong included regional leaders and experts. The results of the FGDs were cross-tabulated with the KIIs to develop major findings and themes for the report. Methodological Limitations The characteristics of PROGATI somewhat limited the evaluation methodology and affected the type, quantity, and quality of data available. These limitations included:  Small sample of regional community leaders and experts. The sample for FGDs was relatively small and very geographically focused in Dhaka and Chittagong, which may prevent the results of the FGDs from being fully representative of the general population;  Lack of counterfactual. Because it is difficult to know what would have happened in the absence of PROGATI, it is difficult to evaluate PROGATI’s success and attribute specific project interventions. To compensate for the lack of the counterfactual, the team triangulated all findings to support validity of findings; and  Sensitivity of responses dealing with corruption. The nature of anti-corruption evaluations and assessments prevents respondents from feeling fully comfortable expressing honest opinions. To overcome this challenge, KIIs and FGDs centered questions and conversations on PROGATI’s effectiveness, corruption in general, and anti-corruption methods. FINDINGS Given the evaluation team’s number of findings across many topical areas, the reader is advised to consult the Findings section of the report for a complete description of findings. Key findings include: 1. On an individual beneficiary level, the media component has been quite effective in supporting professional skills among journalists, but concerns are present among stakeholders regarding the sustainability of JATRI work and the extent to which it actually has built institutional capacity and incentives in media organizations for investigative journalism 2. Effective design and useful training characterize the civil society and citizen participation component. Collaboration with the Khan Foundation and its Civic Forums is an example; expanded application of citizen report cards is another. Support to the BWCC displays a strong record of sustainability 3. Assistance to public institutions has delivered some key results, highlighted by (a) stakeholder meetings to bring together the OCAG, IC and ACC at both national and subnational levels; and (b) key consultative meetings between OCAG and civil society organizations. Concerns are present, however, regarding the sustainability of OCAG outreach and citizen accountability actions. 4. PROGATI successfully helped to set up the Budget Monitoring Unit (BAMU) under the Ministry of Parliamentary Affairs, but the level and extent of engagement of members of Parliament appear mixed. 3 5. PROGATI’s Windows of Opportunity component was a major aspect of the project’s efforts during its last year, focusing on support to media action in response to the RTI Act (ratified in 2009). While this work was effective in engaging many stakeholders its post-project sustainability is uncertain. CONCLUSIONS AND LESSONS LEARNED In recent months, government and donor stakeholders in Bangladesh have reported a general sense that corruption is on the rise again in the county. This is not a positive development, but the donor community is not powerless to work with Bangladeshi society to help reverse this.1 The country’s robust civil society must be re-engaged to reverse the backsliding of recent years. However incompletely, PROGATI did establish an important foundation to continue this effort. One clear lesson learned is that a future anti-corruption program should devote more effort to identifying a theory of change that could be empirically tested by external evaluators over time. From PROGATI’s media work, future programs would benefit it definitions of sustainability were agreed upon by all parties at the project’s outset. In the civil society arena, one of PROGATI’s strongest accomplishments (and the one that received the greatest amount of resources from USAID) was its ability to convey “life after project” technical resources, skillsets and deliverables that strengthened, systematized or transferred organizational capacity to a sector that was galvanized around the issue of corruption, but lacked specific knowledge of how to translate that mobilization into concrete action. Yet this work formally ended by Year 4, and the evaluation team heard convincing arguments for re-energizing the civil society sector specifically around the RTI law. In work with Parliament and with the OCAG, a more effective strategy would make greater efforts toward detailing specific, more stable, predictable outcomes in future years to justify the investment made in these institutions. RECOMMENDATIONS Based on the findings and conclusions presented above, the evaluation team has identified the following recommendations: Recommendation 1: USAID should continue the engagement process around the new RTI legislation, working closely with a Bangladeshi Technical Advisory Committee (TAC) in the process. Should additional resources be available for FY2013, more concentrated engagements with government—in conjunction with DFID, the World Bank the Asian Development Bank (ADB), Danish 1 “[T]he rule of law has deteriorated quite rapidly in the last nine or ten months, and some of the efforts at governance reform do seem to have been stalled or, in some cases, reversed.” House of Commons, International Development Committee, 3rd Report of Session 2009-10, “DFID’s Programme in Bangladesh,” p. 8 http://www.publications.parliament.uk/pa/cm200910/cmselect/cmintdev/95/95i.pdf. 4 Strengthening the Office of the Public Prosecutor USAID’s Knowledge Management database contains instructive examples of best practices from other countries in this sector. Working with the Department of Justice’s OPDAT program, USAID might support an Economic Crimes Unit responsible for monitoring money-laundering proceeds—a complex technical area involving skilled expertise and significant inter￾governmental cooperation. USAID might provide training necessary to develop an information management system to track currency flows, provide case management and guide prosecutors toward successful interventions. International Development Agency (DANIDA), and others working to promote increased civic awareness of transparency-related issues—could be employed in more collaborative ways than have been used previously. Recommendation 2: USAID should provide future assistance through a Bangladeshi-led Cooperative Agreement, augmented by a local Technical Advisory Committee. While a contract-based system of TA offers advantages to USAID in terms of focused delivery and greater financial accountability, its core disadvantage is that it lodges ultimate responsibility for program strategy, mid-course corrections and tactical approaches with the US Government, rather than with the host country. The nature of the assistance itself translates to more passive involvement by counterparts and stakeholders. After 5 years of engagement, both USAID and Bangladesh have reached a literal and symbolic fork in the road. USAID Forward reforms rightly place future onus for program direction with local actors (a combination of state, society and the private sector). A post-PROGATI cooperative agreement with a local consortium would be a good arena for this, especially given the sensitivities surrounding corruption. Its work could be augmented with a small 5-7 member TAC, made up of former government officials, emerging private sector groups, and influential members of the implementer/civil society and microcredit communities. Recommendation 3: Re-Engage CSOs around RTI Issues Specifically A more strategic design for a post￾PROGATI cooperative agreement could focus on the implementation of the RTI Law. CSOs have largely been left behind by this process (despite their support for the original legislation). Recommendation 4: ‘Focus and Concentrate’ Resources and Technical Assistance on an Integrity Model: Timing issues surrounding contract issuance in 2008, coupled with new elections, severely hampered PROGATI’s implementation as has been pointed out. To avoid these issues, a post-PROGATI project should wait for the outcome of the next election. Once in power, USAID (rather than its implementer or partner) should then enter into a dialogue with the Government, using the Cabinet or Principal Secretary as primary interlocutor to enlist its support for an intensive (yet also hands-off) program working with a single agency, department, or even ministry to develop its integrity plan. Choosing one thing to accomplish (and doing it well) makes more sense than working on a variety of components in a relatively superficial manner. 5 INTRODUCTION As poverty and vulnerability to social unrest and political conflict are common to Bangladesh, the U.S. Government (USG) has made development of Bangladesh’s democratic institutions, economic growth, and improvement of human capital foreign aid priorities. Bangladesh is a rapidly growing and developing country with the seventh largest population and fourth largest Muslim population in the world, making its political and economic prosperity and stability critically important to U.S. foreign policy interests.2 While Bangladesh has only held democratic elections sporadically since independence, the country did record three successive peaceful transfers of power from 1991 to 2006. Nonetheless, the country’s development as a democracy has been interlaced with military rule, debilitating political polarization, ineffective governance institutions, and endemic corruption. In January 2007, artisan deadlock over electoral issues resulted in the cancellation of the planned parliamentary elections and the self-imposed rule by the military-backed caretaker government (CTG), which, though unconstitutional, successfully curbed corruption and violence that existed under the surface of elections.3 Two years later, democratic parliamentary elections held on December 29, 2008 proved successful. This success established a precedent for democratic elections and also opened up an opportunity to build human capacity for good governance. The U.S. capitalized on the opportunity, in an attempt to meet its foreign assistance goals, to reduce corruption in the Bangladeshi government and help bolster the economy. The GOB is marked by a lack of accountability, narrow transparency, and weak responsiveness to public input.4 Specifically, while the checks and balances of democracy exist in Bangladesh, they exist without consensus among national leadership in regard to policy-making processes for these democratic institutions. For example, without a robust media, judiciary, active political parties, competitive and free elections, and overall freedom of speech, Bangladesh’s system is merely a shell of a democracy. National politics is grounds for competition for power and resources for a few political elites and their followers, and only rarely about domestic issues important to citizens, which results in a failure to distinguish between the partisan interests of their party and those of the nation.5 Once considered an externality to development—the “cost of doing business overseas” —corruption has finally come to be understood by multilateral and bilateral donors as perhaps the greatest challenge to development.6 It diverts scarce resources away from poverty-fighting efforts and skews broader￾based economic growth toward narrow, short-term projects that benefit only a few.7 2 USAID/Bangladesh, “History,” Dhaka, 2012 http://www.usaid.gov/where-we-work/asia/bangladesh/history. 3 Scope of Work, PROGATI Performance Evaluation, 2012. 4 Scope of Work, PROGATI Performance Evaluation, 2012. 5 Ibid. 6 See “Global Integrity: Developing Anti-Corruption Strategies in a Changing World,” 9th International Anti￾Corruption Conference (Durban, South Africa, 1999), the Monterrey Consensus of the International Conference on Financing for Development (Mexico, March 2002) and the yearly Global Competitiveness Report, www.weforum.org/issues/global-competitiveness, among others. 7 World Bank, “Helping Countries Combat Corruption,” Washington, 2006 http://www1.worldbank.org/publicsector/anticorrupt/corruptn/cor02.htm. 6 This is especially true in Bangladesh, where economic and political developments have been constrained by pervasive corruption, making the country vulnerable to social upheaval and political instability. As such, a key objective of both USG foreign assistance in Bangladesh and the GOB is to reduce corruption by promoting greater transparency and accountability in the management of public resources.8 USAID’s approach emphasizes transparency in public management and enhancing the watchdog capabilities of civil society organizations (CSOs) and the media.9 THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE USAID awarded DAI a task order in September 2007 to implement a four-year, $18 million anti￾corruption program known as PROGATI. In 2011, the task order was modified to extend the program up to September 2012 and increase the total estimated cost to more than $19 million. At a minimum, the contract specified that DAI, Inc. achieve the following principal objectives: (1) Strengthen a Parliamentary Budget Unit; (2) Provide technical assistance (TA) to support a civil society anti-corruption coalition development, public-private partnerships, and significant citizen advocacy and watchdog initiatives; (3) Increase citizen group participation in understanding and developing program-based budgets for more external oversight of GOB budgets; (4) Support the mass media to build its capacity as a public watchdog through the establishment of a Center for Investigative Journalism; (5) Support parliamentary oversight committees and parliamentary accountability reforms; and (6) Continue to support the Parliamentary Budget Analysis and Monitoring Unit during its transition to USAID’s legislative strengthening program.10 USAID also included additional objectives thought to further promote the main principals of PROGATI in the task order SOW, as provided under the DCHA/DG Building Recovery and Reform through Democratic Governance (BRDG) Indefinite Quantity Contract (IQC). PROGATI developed a two-pronged approach to achieve the goals of the program: 1) strengthen public institutions, including civil servants and elected officials to better carry out their oversight functions and 2) strengthen CSOs, including media, to generate demand for increased oversight.11 To achieve these goals, the program focused on five main components: media, civil society, public institutions, parliament, and windows of opportunity. While it was not explicitly stated as a component of PROGATI, USAID emphasized the need for the project to also assess and incorporate activities related to policy reform under each of the components. The expectation was to build on the successful foundation of previous 8 BRAC IGS, The State of Governance in Bangladesh 2008: Confrontation, Competition, Accountability p. 120 9 Scope of Work, PROGATI Performance Evaluation, 2012. 10 Scope of Work, PROGATI Performance Evaluation, 2012 11 Scope of Work, PROGATI Performance Evaluation, 2012 7 anti-corruption programs funded by other donors as well as establish synergies with partners involved in on-going USAID/Bangladesh DG work and work under other USAID objectives (e.g., population, health, nutrition, and education; economic growth; and food, disaster, and humanitarian assistance). PROGATI was expected to achieve the following results:  Increase the capacity of the Parliament and its members to analyze, monitor and influence national policy and budgetary priorities and also strengthen its oversight capacity;  Enhance the effectiveness of internal procedures of selected public institutions of accountability dealing with public finance monitoring and auditing, transparency and accountability, and right to information;  Build new and strengthen existing civil society coalitions, networks, and public-private partnerships to increase awareness on corruption issues;  Encourage opportunities for citizen participation in and oversight of the national budget and government decision-making processes;  Foster media capacity to report on transparency and accountability issues, and advocate for legal reforms; and  Advance the dissemination of information by selected public institutions at both national and local levels to promote citizen access to improved government information.12 PURPOSE OF THE EVALUATION The purpose of this end-line evaluation of the PROGATI Project is to assess the results and outcomes of the USAID-funded anti-corruption initiative and identify lessons that could be taken from the five-year program for use in designing future interventions of this nature. This report seeks to provide USAID/Bangladesh with information that helps it to make more informed decisions as the Mission embarks on integrating USAID Forward into its programming. The specific goals of the evaluation follow:  Evaluate PROGATI’s overall performance by assessing results against contractual goals and indicators;  Assess the efficacy, cost-effectiveness, and results of the PROGATI implementation approaches and management structure in meeting objectives;  Make recommendations to USAID/Bangladesh for future interventions and direct programming of local organizations, including those that DAI worked with under PROGATI.13 The audience for the evaluation report includes USAID/Bangladesh; USAID/Washington, specifically leaders of USAID Forward; DAI; and other existing USAID implementing partners. EVALUATION QUESTIONS The evaluation SOW outlines the following key evaluation questions: 12 Scope of Work, PROGATI Performance Evaluation, 2012 13 USAID. Scope of Work for the Promoting Governance, Accountability, Transparency, and Integrity Program Performance Evaluation, August 2012. 8 Results 1. To what extent has PROGATI been successful in achieving program objectives? (Describe what have been the most significant outcomes of activities implemented under PROGATI, explore variation in performance - may be performance has been relatively successful in some areas but less successful in others, explain the way and effectiveness of program components of PROGATI in contributing to program objectives) 2. Were there any significant unintended results (positive or negative)of PROGATI interventions? Program Theory and Design 3. To what extent has the original hypothesis, “strengthening public institutions, parliament and CSOs, including the media, and promoting principles of accountability, transparency, and responsiveness will result in decreased levels of corruption in Bangladesh,” on which the PROGATI Project approach was designed, proved valid? (Explain validity of PROGATI Project assumptions regarding requirements for promoting anti-corruption, good governance, and integrity, and describe the extent to which political events during the implementation period affect the success or limitations of PROGATI interventions.) Implementation and Management 4. To what extent has the PROGATI Project implementation approach been successful in achieving the results of the program? (Describe effectiveness of implementation methods; indicate coordination level with various USAID/Bangladesh Technical Teams - i.e., Economic Growth; Population, Health, Nutrition, and Education; Food Disaster and Humanitarian Assistance - as well as within other DG activities; and describe coordination with the various other donors’ anti-corruption activities.) 5. How effective has PROGATI’s performance management system been to achieving program objectives? (Explain relevance of indicators to manage results.) 6. How effective was USAID management role to implement the program successfully? 7. To what extent has the PROGATI management structure and its mix of acquisition and assistance tools (e.g., short-term technical assistance [STTA], sub-contracts, sub-grants, training, etc.) been appropriate for supporting PROGATI objectives in an efficient and flexible way? Cost Effectiveness 8. To what extent was targeting of program beneficiaries appropriate to support cost-effective achievement of program results? (Describe if results achieved under PROGATI are being produced at a reasonable cost as compared to other programs the Evaluation Team may identify in Bangladesh with similar objectives, and explain the extent PROGATI took advantage of cost￾share and/or leveraging opportunities.) Sustainability 9. Are processes, systems, and activities in place to ensure that the results of PROGATI activities will be sustainable? (Describe evidence indicating that the GOB and other Bangladeshi partners will take ownership of PROGATI activities, and describe obstacles that exist for achieving or maintaining sustainability.) 9 Cross Cutting Issues 10. How well has PROGATI integrated support to gender equality, the disabled, and minorities into its program? Client Satisfaction (GOB, beneficiaries, other stakeholders) 11. What are the levels and areas of satisfaction or dissatisfaction with PROGATI cooperation and performance on the parts of project stakeholders? This group includes: leaders and staff of Parliament and Parliament Secretariat; leaders and staff in the Office of the Comptroller and Auditor General (OCAG), the Information Commission (IC), the Anti-Corruption Commission (ACC) and partner ministries; media representatives; non-governmental organizations (NGOs), CSOs and citizens’ organizations; and development partners and local organizations that work on anti-corruption issues but who are not PROGATI partners. Program Opportunities 12. Is there evidence from PROGATI and similar programs to suggest that alternative program approaches (such as one in which anti-corruption activities are interspersed across the Mission portfolio) may have been more successful? EVALUATION METHODOLOGY The evaluation team applied a mixed-methods approach to the evaluation, which included: 1) A desk review, 2) KIIs, and 3) three FGDs. The desk review included the review of PROGATI semi-annual and quarterly reports, the initial and amended Performance Monitoring Plan (PMPs), the corruption baseline and endline surveys, perception-based surveys, other relevant program documents, and relevant external reports (See Annex B for a list of documents reviewed). More details on the KIIs and FGDs follow. KEY INFORMANT INTERVIEWS The team sought informed, in-depth, open-ended interviews with 25 individuals from three broad categories: 1) individuals directly involved with the design and implementation of PROGATI programming, including USAID officials, DAI and PROGATI staff, CSO partners, and program recipients (such as the Parliamentary Secretariat); 2) individuals who are knowledgeable about USAID programming in the DG area, but not necessarily linked with PROGATI (such as TetraTech ARD, Democracy International [DI], the Solidarity Center, local CSOs implementing other DG programs, other donors, such as the Department for International Development [DFID], and active/retired GOB officials with knowledge of the program environment and its constraints); and 3) public opinion leaders in the country, including academics, journalists, barristers, researchers, and policy analysts.14 For purposes of consistency across interviews, the evaluation team structured interviews around three principal questions: 14 The protocol used by the evaluation team for KIIs may be found in Annex C. 10  How would you assess PROGATI’s performance against its own goals and indicator targets?  How would you rate the impact or effectiveness of those indicators or results (using whatever yardstick they chose, not necessarily those chosen by DAI or USAID)?  What advice or suggestions would you give USAID on future governance and integrity-related programming? The first two questions were more relevant for PROGATI staff, implementers, and beneficiaries, but other key informants also provided good information in these areas. KIIs were semi-structured, i.e., a respondent was asked to give her or his views rather than choose from among a set scale of responses. A key to the integrity of the process was to use multiple accounts of the KIIs, as recorded by the four members of the team, to reinforce reliability of data from the interviews. As much as possible, the team members discussed KIIs amongst themselves to ensure the most accurate recollection. FOCUS GROUP DISCUSSIONS The evaluation team used FGDs to provide a perspective external to that of program, USAID, and key DG stakeholders. The FGDs were meant to balance KIIs that often rely excessively on internal perspectives, debates, and issues. FGDs are a rapid-assessment, semi-structured data collection method where a narrowly selected set of informed participants gather to discuss issues and concerns with one another, guided by a skilled moderator. The evaluation team convened three FGDs. Two of those were held in Dhaka. One took place on September 12th and included a group of experts familiar with PROGATI, and the other took place on September 13th with a group of senior community leaders and experts not familiar with PROGATI. The evaluation team convened the third FGD in Chittagong with regional leaders and experts. Chittagong was chosen due to the city’s economic influence as a port and center of industry with a critical mass of English speakers and the relative ease of travel and facilities present there.15 The FGDs centered around three key issues similar to, but not duplicative, of the KII questions.16 CROSS-TABULATION BETWEEN KIIS AND FGDS While the KII data were written up separately from the FGDs, the two sets of data were cross￾referenced to develop major findings and themes for this report, with the goal of triangulating findings and incorporating as many sources as possible to capture the depth and richness of the issues. METHODOLOGICAL LIMITATIONS The evaluation team was somewhat limited methodologically in the evaluation by the characteristics of the program, the lack of counterfactuals, and the time available for the evaluation. These limitations affected the type, quantity, and quality of data available for the evaluation and make it difficult to extrapolate conclusions and recommendations to programs operating in other environmental contexts. The most serious limitations to the evaluation include the following: 15 A list of the FGDs and more details on numbers of participants and locations can be found in Annex F. 16 Guidelines for FGD questions and rules of engagement can be found in Annex C. 11  Small sample of regional community leaders and experts. The sample of regional community leaders and experts that the evaluation team was able to visit for FGDs was relatively small and very geographically focused in Dhaka and Chittagong. Thus, FGD responses may not be representative of the feelings of the general populous or even of the populations in those specific regions. However, they do help to balance perspectives of those typically interviewed for KIIs.  Lack of a counterfactual. The general nature of DG programs makes them difficult to evaluate because, in most cases, there is no valid counterfactual (e.g., what would have happened within an institution, such as the GOB, if it weren’t for a specific intervention?). This was true of PROGATI, which was further disadvantaged by the fact that there was also no other program in the country in which to compare it against. The combination of both these factors makes outcomes difficult to attribute to specific program interventions. The evaluation team found this lack of a counterfactual or other comparative program to be especially challenging in trying to address the larger results and program theory questions.  Sensitivity of responses regarding corruption. An additional major limitation of anti￾corruption evaluations and assessments relates to the sensitivity of the topic. Respondents often don’t feel comfortable telling the truth about their own experiences with corruption, they often don't want to admit to bribe-giving, for example. Further, corruption is difficult to measure directly, and respondents often do not have enough information themselves to provide specific details on its pervasiveness, scope, or cost. To best avoid these response validity traps, the evaluation team emphasized asking KIIs and FGD participants about PROGATI's effectiveness, corruption in general (not personal experiences with it), and various approaches to combating it. However, there is still the potential that some response bias may have skewed the team’s results slightly. FINDINGS RESULTS To what extent has PROGATI been successful in achieving program objectives? Were there any significant unintended results (positive or negative) of PROGATI interventions? PROGATI’s original PMP, dated September 2008, identified five main project objectives: 1. Strengthen the media to serve as an effective public “watchdog”; 2. Strengthen civil society to support and promote anti-corruption reforms; 3. Strengthen public institutions’ oversight capacity; 4. Strengthen parliamentary oversight capacity; and 5. Respond to urgent, unexpected, or politically sensitive situations as they arise. These were identified as the five “components” of the project and shortened to media, civil society and citizen participation, public institutions, parliamentary, and windows of opportunity. A summary follows of PROGATI achievements with regards to the indicators in its PMP. (A more complete table by year is also available in Annex J): 12 Table 1: PROGATI Project PMP Targets and Achievements COMPONENT AND GOALS TARGET ACHIEVEMENT % ACHIEVED 1. Strengthening media to serve as an effective public watchdog Number of media CSOs and/or support institutions assisted 20 21 105.00% Number of government media relations staff trained 545 658 120.73% Person-days government media relations staff trained 1,084 2,195 202.49% Journalists trained 1,408 1,652 117.33% Person-days journalists trained 3,370 3,478 103.20% Non-state news outlets assisted 155 349 225.16% 2. Strengthening civil society to support and promote anti-corruption reforms People affiliated with NGOs receiving anti￾corruption training 1,305 1,463 112.11% Person-days people affiliated with NGOs receiving anti-corruption training 2,660 2,801 105.30% Anti-corruption measures implemented 39 38 97.44% CSO Advocacy campaign supported 813 794 97.66% Persons participating in CSO advocacy campaigns 178,950 201,340 112.51% CSO that engage in advocacy and watchdog functions supported 436 727 166.74% Positive modifications to enabling legislation/regulation for civil society accomplished 0 0 0% CSOs improved internal organizational capacity 20 10 50.00% 3. Strengthening public institutions oversight capacity Mechanism for external oversight of public resource use supported 10 5 50.00% Government Officials receiving anti-corruption training 504 676 134.13% Person-days government officials receiving anti￾corruption training 980 1,689 172.35% 4. Strengthening Parliament’s oversight capacity Members of Parliament and Secretariat staff trained on budget transparency and accountability issues 230 173 75.22% Members of Parliament using budget analysis services (Budget and Monitoring Unit [BAMU] and community in practice) 171 187 109.36% The evaluation team further assessed PROGATI’s results for each of the five components, and findings related to those areas follow:17 17 PROGATI. PMP, September 22, 2008. 13 Component 1: Media The objective of Component 1 was to strengthen the media to serve as an effective public watchdog. To achieve this objective, PROGATI supported the BRAC University Institute of Governance Studies (IGS) in establishing the Journalism Training and Research Initiative (JATRI) and also worked with the Management and Resources Development Initiative (MRDI) to advocate for the Right to Information Act (RTI). JATRI is an investigative media center designed to increase the media’s capacity to report on transparency and corruption within the government as well as advocate for improved media access laws. Since its opening in May 2009,18 JATRI has trained more than 1,650 journalists and nearly 660 government relations staff on topics such as investigative reporting techniques, interviewing and source development, and ethics and standards.19 A 2011 U.S. Office of the Inspector General (OIG) Audit Report found that these trainees were pleased with the training they had received, saying it “strengthened their interviewing skills, enhanced the accuracy of their reporting, and increased the credibility of their news stories.” At the end of FY 2011 (the last financial report for which expenditures are broken down by component), PROGATI had spent close to $3.98 million of its $18 million budget for activities under this component.20 Despite the largely positive results of this component identified by the OIG, when asked at the beginning of this evaluation to identify PROGATI’s most successful results, project staff highlighted achievements in every project component except this one. While the evaluation team identified similar results for JATRI as the OIG’s Office, stakeholders asked about the center provided mixed reviews as to its effectiveness. They expressed concerns over:  The level of sustainability for the center. PROGATI staff also noted that nobody—not project staff, USAID, or the various implementers who worked on Component 1—effectively defined sustainability for the center. More details about the sustainability of JATRI are provided under the Sustainability subsection of this report; and  Whether JATRI’s work is truly “investigative” in nature. Stakeholders interviewed by the evaluation team said that despite all of the training completed by the center, Bangladeshi media outlets have yet to invest seriously in the kind of highly labor-intensive, detailed follow￾through required of investigative reporting. They said they think this is because there is simply too much money to be made in easier ways. JATRI staff confirmed this, saying that they believe a lot of the media outlets make their money through public service campaigns. Component 2: Civil Society and Citizen Participation The objective of Component 2 was to build demand for transparency and accountability within government by strengthening civil society to support and promote anti-corruption reforms. To achieve this objective, PROGATI worked with a grassroots network of CSOs to create demand at the local level. During the first four years of the project, PROGATI supported 727 CSOs engaged in advocacy 18 The center was actually scheduled to open in October 2008, according to a February, 2011 audit report by the U.S. Office of the Inspector General, but it opened seven months late. 19 Appendix IV, PROGATI Semi-Annual Report: 1 April to September 30, 2011, USAID/Bangladesh, October 2011. 20 Appendix IV, PROGATI Semi-Annual Report: 1 April to September 30, 2011, USAID/Bangladesh, October 2011. 14 The Civic Forum Process The Khan Foundation, with PROGATI support, carried out Civic Forums in 99 different Union Parishads (UPs). The Civic Forum Process contains 6-7 discrete steps, each of which requires action and follow-through: (1) An issue is chosen by influential members of a given community, which then forms a smaller working group; (2) This working group collects basic information connected with public service delivery; (3) A report is prepared covering findings, and a forum for dialogue is set up with stakeholders to present it to the larger community; (4) Formal recommendations are adopted; (5) A smaller committee is designated to monitor the outcome of interventions; (6) This committee presents follow-up findings; and (7) The process—as much as the final outcomes—generates robust levels of engagement, discussion, and participatory government. Engagement in this process resulted in an increase of the Vulnerable Group Development (VGD) Program distribution of rice for marginalized populations from 22 kilograms (kgs) to 28 kgs/month. While the new allotments are not in line with planned distribution (the planned allotment for these UPs was actually 30kg per month), this shift does represent a significant improvement. It is clear that the Forum Process significantly altered an atmosphere of indifference toward incidental corruption at local levels. and watchdog functions, and trained 1,463 NGO staff members on ways to combat corruption (see Table 1 for more information). At the end of FY 2011, PROGATI had spent a total of $5.3 million on its work with CSOs—among components the largest share of its $18 million budget before the final-year extension.21 PROGATI staff said that prior to the program’s intervention, most CSOs operated in isolation from one another and from society due to a history of societal mistrust and high politicization (low social capital). This environment hampered real progress toward both transparency and improved state-societal relationships. An Example of PROGATI’s Work with CSOs The Khan Foundation, one of PROGATI’s partner organizations that was already working with CSOs before the PROGATI Project began, detailed the methodology PROGATI introduced to its on-going civil society work. This included the development of at least three different strategies for engaging CSOs and government officials, especially at the upazila22 and union parishad23 levels: (1) A Mass Awareness Strategy, including anti￾corruption campaigns and special events used to bring both CSOs and officials together in a train-the-trainer process; (2) A Budget Awareness Strategy, which sought to excite interest and energy in a review of national budget priorities;24 and (3) Citizen Scorecards, which allow citizens to play the government watchdog role. Citizen Report Cards or Scorecards, which were first developed in the late 1990s in East Africa, act to spot-check government-managed supply of key services in health, education, and other areas (such as road-building). The use of Scorecards really took off in several Indian states in the 1990s, and one CSO in particular—Mazdoor Kisan Shakti Sangathan (MKSS)— became known as a leader in citizen-based anti-corruption work.25 Following the MKSS model, 21 Appendix IV, PROGATI Semi-Annual Report: 1 April to September 30, 2011, USAID/Bangladesh, October 2011. 22 Upazilla’s are divisions similar to counties. 23 Union Parishads (UPs) are the smallest rural administrative and local government units in Bangladesh. 24 While USAID wanted the budget awareness strategy under ER 2.3 to focus on national, rather than local budgets, it was not a great match, considering that local organizations were far more interested (as they are in most countries) in local budget issues and concerns, and this point was raised by more than one CSO during this Evaluation. 25 http://www.mkssindia.org/about-us/ 15 PROGATI used many of the same principles in its efforts to organize Bangladeshi CSOs to adopt similar techniques, and many of the steps described by Khan and others bear strong resemblance to tactics employed and field-tested in India some years earlier.26 According to Khan Foundation staff, PROGATI worked with the Foundation to develop specific methodologies for different groups based on levels of education and engagement. They developed one methodology for ‘change agents’ (involving small dialogues and group activities bringing together officials in a consultative process) and another methodology for service providers and service receivers to jointly review specific findings, such as discrepancies between the allocated amount of rice marginalized populations received under the government’s Vulnerable Group Development (VGD) Program. Khan’s methodology helped to detect corruption in the VGD Program, which was essentially robbing impoverished families of their fair share of rice allocation through this government program. Stakeholders reported that, now, this program is more transparent and families are receiving higher allocations (See the Text Box on Page 13 for more information). The Khan Foundation was not the only organization to champion these kinds of systematic approaches. Other PROGATI partners, including Rupantar, Grameen Alo, and others, did the same. CSOs reported that training provided by PROGATI partners as well as experiences in implementing the approaches described above helped them to understand some of the genuine constraints to service delivery faced by local governments. The process educated both service providers and service recipients, helping them to better understand one another, and the opaque processes of budgeting and accounting which some stakeholders reported probably increased the stigma surrounding malfeasance and fraud at local levels. Program Administration and Monitoring CSOs leaders interviewed told the evaluation team that there were short interruptions of funds flowing from PROGATI for their projects, particularly during the first three to four years, which made longer￾term institutional partnerships more challenging. Some implementers said this impacted the quality and quantity of civic forums that emerged out of this component. The end of the CSO Component PROGATI and nearly all its CSO partners reported that the use of citizen scorecards formed the basis of a societal change in attitudes and perspectives toward corruption. PROGATI, however, ended its work with the CSOs in late calendar year 2011 (as shown in Table 2). PROGATI and its CSO partners all said that the reason for the shift away from this component was a loss in USAID interest in the project’s work with CSOs. USAID said that it wasn’t a loss of interest, but instead, that it wanted the CSO work to be completed by Year 4 so the project could focus its efforts in other areas. 26 Indian civil society engagement around right-to-Information issues has generally been more powerful than Bangladesh’s, and opposition parties more skillful at using the issue as an organizational tool, tracking incidences of bribery and malfeasance quite effectively. See www.ipaidabribe.com/?utm_source=google&utm_medium=adwords&utm_campaign=c-others. 16 Table 2: PROGATI Training: Indicator Actuals Through June 2012 * Figures cannot be verified beyond 9/30/2010 (PROGATI Mid-Term Evaluation 2010), p. 15. ** Figures are for 1 Oct 2010 - 30 Sep 2011 (PROGATI Semi-Annual Report, April-September 2011). ***PROGATI Quarterly Report (October-December 2011). ****PROGATI Quarterly Report (January-March 2012). *****PROGATI Quarterly Report (April-June 2012). The Women’s Chamber of Commerce Another activity completed by PROGATI under Component 2 was training for the Bangladesh Women’s Chamber of Commerce (BWCC), a professional NGO made up of 4,000 entrepreneurs. This training centered on issues such as business registration, licensing, and tax policies. The BWCC then provided a training of trainers (TOT) to 600 women entrepreneurs so that the lessons they learned could be expanded to other groups. Experience gleaned not only from PROGATI technical support but also from a study tour to the International Anti-Corruption Conference in Athens in 2008 sparked an idea within the BWCC for a citizens’ hotline, which the group later established in Khulna to provide instant assistance to members facing harassment or corruption in their dealings with the government. In addition, with PROGATI’s support, the BWCC, set up division-level working groups within offices at the National Board of Revenue to help members on tax issues. These working groups have effectively improved the environment for business licensing and registration. BWCC staff told the evaluation team that the group has institutionalized these processes and is still operating the hotline and finance/administration system, proof of the sustainability of some of PROGATI’s work. PROGATI Indicator 2008 PMP 2010 PMP * 2011 PMP** 2012 PMP*** 2012 PMP**** 2012 PMP***** Dates of Coverage -- -- 1 Oct 2010 - 30 Sep 2011 1 Oct to 31 Dec 2011 1 Jan to 31 Mar 2012 1 Apr to 30 Jun 2012 Government official anti-corruption training 254 386 445 0 68 96 NGO staff trained 1,104 2,194 758 38 174 0 CSO advocacy campaigns 288 478 316 0 0 0 Persons participating in CSO advocacy campaigns 0 130,206 71,134 0 0 0 CSOs engaging in advocacy and watchdog activities 182 517 103 18 87 0 Number of Government-owned media staff trained 30 394 258 0 0 21 Journalists trained 665 1,439 469 0 118 6 Number of media CSOs assisted 8 13 8 0 0 0 Non-State news outlets assisted 0 225 85 0 37 2 17 Component 3: Public Institutions The objective of Component 3 was to strengthen public institutions and parliament’s oversight capacity of these institutions by expanding the reporting authority and effectiveness of the government’s audit office and establishing a functioning parliamentary budget analysis and monitoring unit. To achieve this objective, PROGATI produced a series of TOT, user manuals, inter-ministerial and governmental audit surveys; provided TA to support an OCAG-created outreach division; and convened key stakeholder meetings between the OCAG’s office and line ministries. During the last year of the program, PROGATI organized stakeholder meetings to bring the three oversight institutions (the OCAG, the IC, and the ACC) together at both the national and sub-national levels. During the life cycle of the PROGATI project it supported training of 676 government officials on anti-corruption (See Table 1 for more information). At the end of fiscal year (FY) 2011 (the last FY for which expenditures are broken down by component), PROGATI had spent a total of $2.8 million on this Component.27 The OCAG constituted PROGATI’s “primary government counterpart” during the course of project, and it experienced significant improvements in its organizational capacity. Additionally, the quality and quantity of inter-governmental communications and civic outreach in general improved during the PROGATI Project. Details about each of these activities follow. OCAG Changes In 2009, no internal policies governed the public dissemination of information from the OCAG. But, in 2010, the OCAG began to publish audit reports on its Web site for the first time.28 Then, in May 2011, with PROGATI’s support, the OCAG began to gradually establish an outreach division (Communications and Media Cell) in compliance with requirements under the 2009 RTI Law, in partnership with the Canadian International Development Agency’s (CIDA’s) Strengthening Comptrollership and Oversight of Public Expenditure (SCOPE) Project using PROGATI TA. According to PROGATI semi-annual reports, its Web site, 29 and USAID’s Mid-Term Evaluation of the project, PROGATI also helped set up key meetings between the OCAG and CSOs with the goal of increased collaboration between the two groups on how best to monitor and create demand for government accountability.30 This was the first time these two key Bangladeshi institutions had ever collaborated. In addition to these meetings the Comptroller/Auditor General (AG) Office also inaugurated another aspect of outreach—key stakeholder meetings between auditors and auditees in various line ministries (Health, Food & Disaster Management, and others). Between these two activities, PROGATI’s goal of 27 Appendix 4, PROGATI Semi-Annual Report: 1 April to September 30, 2011, USAID/Bangladesh, October 2011. 28 U.S. OIG, Audit of the PROGATI Project. February 2011. 29 PROGATI’s website: http://www.progatibd.org/index.php?option=com content&view=article&id=165:office-of￾the-comptroller-and-auditor-general-engages-public&catid=28:success-stories&Itemid=58 (accessed 9/29/2012). 30 PROGATI was never the sole source of donor institutional support to OCAG. In addition to the current CIDA￾funded SCOPE Project, earlier programs included a DFID-funded Reforms in Budgeting and Expenditure Control program implemented by the GOB, ending in 2002, whose aims included the improvement of accounting systems by revising rules and procedures within the OCAG; and the DFID-funded, HELM Corporation-implemented Reforms in Government Audit program also ending in 2002, focusing on civil and local audits. Another related United Nations Development Programme (UNDP)-funded program implemented by OCAG itself, designed to introduce performance and Value for Money auditing, ended in 2002. 18 “increasing citizen group participation…for [ ] external oversight of GOB budgets” (Section C.1 – PROGATI SOW) was met. PROGATI Engagement with the OCAG In the early days of the PROGATI Project, PROGATI staff said they saw the Auditor General as one of the chief obstacles to implementing any program connected with the OCAG. The OCAG was not cooperating with PROGATI on its efforts to increase transparency within the agency, To overcome this obstacle, PROGATI staff said they hired former OCAG officials familiar with the inner-workings and limitations of the OCAG to liaise with the Auditor General. PROGATI staff report that these officials helped the project earn the support of the Auditor General. From January through March 2011, PROGATI collected significant information about audit capacity within OCAG and used that to design trainings for officials within OCAG district and upazilla offices.31 These trainings focused on fraud investigation techniques and the development of a code of ethics, according to PROGATI Semi-Annual Reports, which also provided evidence that the project arranged a divisional workshop on the role of auditors to raise awareness among government officials, civil society, and the media on the purpose of government audits and how they should function. OCAG staff said the trainings greatly assisted the agency, which was functioning with at least a 30 percent vacancy rate and a large backlog, to deliver audit reports.32 Sustainability of OCAG Results The current Comptroller/Auditor General leaves office in February, 2013.33 A recurrent theme of conversations with PROGATI revolved around concerns that TA directed toward the OCAG was too heavily oriented around the personality of the current AG. PROGATI staff reported (and evaluation team observations and discussions with OCAG staff confirmed) that mid-level OCAG officials remain very opposed to collaboration with PROGATI. As of the end of the project, in 2012, the mid-level officials were still cooperating with PROGATI, but PROGATI believes this is largely only due to the presence of the current AG. PROGATI staff report a concern that after the AG’s departure, these officials could “proceed with[out] all possible dispatch,” particularly during a heavily politicized pre￾election period. A high-level PROGATI staff member said, “A base has been established, but it’s not solid.” This staff member further reported that even the project’s last phase of work with the OCAG— the work done through the complex, collaborative discussions among the OCAG, the IC, and the ACC at both national and sub-national levels—appeared precarious, unstable, and perhaps reversible. 31 See the PROGATI Survey on CAG Audit Skill, USAID/Bangladesh, June 2011. 32 As of May 2012, the PAC has submitted 586 Audit Reports for review, with only 293 (about 50%) actually being discussed in Parliament during its current session. Presumably, many more Reports await even submission by PAC. The Daily Star, May 6, 2012, (www.thedailystar.net/newDesign/news-details.php?nid=232941), accessed 9/29/2012. 33 In the past, Auditor Generals have been appointed as ministers or advisers to a caretaker government, and a former Auditor General currently holds the post of Chairman of the Energy Regulatory Commission. But, the Bangladesh High Court recently declared his appointment to be unconstitutional, and the verdict was stayed by the Supreme Court’s Appellate Division in March 2011 (http://www.thedailystar.net/newDesign/news￾details.php?nid=175574). 19 The Draft Audit Act In 2002, the OCAG began work on a new Audit Act governing relations between the OCAG, government, and parliament. The Act was designed to give OCAG greater financial and personnel autonomy than it enjoyed at the time. PROGATI staff said that their partner institutions, using a “soft advocacy” approach, worked behind the scenes to secure support for this new act from both the Standing Committee on the Ministry of Law, Justice, and Parliamentary Affairs and the Ministry of Finance. Early in 2012, the Minister of Finance publicly stated his intention to submit the Audit Act formally to Parliament and get its approval to make the OCAG completely independent from government’s fold. However, at the time of this evaluation, this had still not occurred, limiting the OCAG’s independence (as described below). OCAG Reporting PROGATI staff explained that in addition to delivering audit reports to the Speaker of Parliament (and through the Speaker to the Chair of the Parliamentary Standing Committee for Public Accounts), as is usual practice in Westminster-style AG arrangements, the Bangladeshi OCAG also delivers reports to the Minister of Finance—a member of the Government’s Council of Ministers (the cabinet)—a practice which undermines the institution’s standing as an independent organization (since the OCAG remains tied to the Ministry of Finance and bound by its decisions).34 PROGATI staff expressed their concern about this process and the fact that it does not conform to International Organization of Supreme Audit Institutions (INTOSAI) guidelines. In fact, the evaluation team found that the practice is in direct violation of the United Nations Convention Against Corruption, to which Bangladesh is a formal signatory.35 PROGATI staff also questioned the fusion of responsibilities of the CAG within one office in Bangladesh. Transparency International (TI) staff called this a system of ‘dual management.’ In many Parliamentary systems, the OCAG has either been abolished altogether, or constitutionally separated from that of Auditor General.36 34 The actual process is somewhat more convoluted: the Auditor General submits Reports to the President, who then presents them to Parliament through the Minister of Finance as well as to the Prime Minister as head of Government every six months, assembling reports from several different directorates. The process is quite lengthy, and delays of six months after finalization before submission to Parliament are common (See www.igs-bracu.ac.bd/UserFiles/File/archive_file/OCAG_Background_Paper.pdf [39-40] for more information). 35 “States Parties [institutions] should legislate to ensure the separate entity of the State Audit, its operational independence, the appointment of an appropriately qualified Head by the legislature, adequate capacity to undertake its work, right of access to the expenditure of any public funds, and the right to report to the legislature.” Technical Guide to the United Nations Convention Against Corruption, UN Office of Drugs and Crime (Vienna) and United Nations (New York), 2009, page 39 (emphasis added). As of December 2010, Bangladesh had both signed and ratified UNCAC. See also Capacity Development of Supreme Audit Institutions: Status, Needs and Good Practices, INTOSAI-Donor Stocktaking Report, 2010, http://www.idi.no/Filnedlasting.aspx?MId1=24&FilId=285. 36 While India and the UK both retain the name of Comptroller in their supreme audit institutions, they do not assign any specific responsibilities to the position. In Bangladesh, overall recommendations to support the political independence, institutional integrity and greater effectiveness of the OCAG can be found in “Office of the Comptroller and Auditor General: Policy Note,” Institute of Governance Studies, BRAC University, January 2010, Dhaka. 20 PROGATI, the Bangladesh Enterprise Institute (BEI), and other stakeholders said that the OCAG’s current system of reporting and irregular consolidation of responsibilities, prevents OCAG’s full independence and, therefore, its effectiveness and ability to report fairly on government waste and abuse. Component 4: Parliamentary The objective of Component 4 was to strengthen Parliament’s oversight capacity. As part of this component, PROGATI helped to set up the BAMU under the Ministry of Parliamentary Affairs. BAMU runs the Budget Help Desk, which provides support to line ministries upon the Minister of Finance’s introduction of the budget to Parliament every year. BAMU also published a Budget Compendium in both English and Bangla explaining virtually every term, law, and constitutional procedure in connection with the entire budget process. As of October 2011—before the Budget Compendium was established or the Help Desk was set up, PROGATI had spent $2.3 million on this component. Despite PROGATI’s successes under this component, project staff said they could have accomplished more had there been a Memorandum of Understanding (MOU) with Parliament from the outset of the project. They said an MOU would have helped to identify key champions, interlocutors, and supporters for the work. Work under this component was very limited at first and more substantive only toward the end of the project. It also proceeded without any apparent rules of the game. USAID disagreed with PROGATI’s assessment of the situation, saying that the Agency already had an MOU with the Government of Bangladesh and that establishing one with Parliament would have been a duplication of efforts.37 At the end of five years BAMU had still not been integrated into Parliament’s library or research section. Further, there is little evidence as to whether the services offered by BAMU will actually engage members of Parliament (MPs) who—in the words of many stakeholders interviewed—do not have much interest in the oversight aspects of their jobs. No one interviewed could state with certainty what the outcome of BAMU would be, and PROGATI staff said they were relieved that ultimate responsibility for it had been handed off another USAID program (PRODIP). Component 5: Windows of Opportunity Ten percent of PROGATI’s overall budget was set aside to allow the project flexibility in responding to urgent, unexpected, or politically sensitive situations. PROGATI was to react quickly to the unexpected situations and provide TA, training, or commodities in response. Later this component was redesigned to advance the dissemination of information by selected public institutions at both national and local levels to promote citizen access to improved government information. As of October, 2011, only $0.9 million had been spent on Component 5. Activities under Component 5 centered around inciting media action in response to the RTI Act, which Parliament ratified in March 2009. The RTI Act required proactive disclosure of government activities and decisions. Project stakeholders reported that this component was the only working component of the project from about December, 2011 through September, 2012. They said the work PROGATI embarked on—bringing reluctant parties to the table at national and sub-national levels to discuss 37 See Annex VI, Letter from Alexious Butler at USAID/Bangladesh, dated November 11, 2011 to Mahfuzur Rahman, Parliament Secretariat, responding to Parliament’s queries on PROGATI’s Work plan. 21 implementation—was critical. Staff from the IC said “the only way a reluctant government can respond to the challenges of corruption is by improving its systems of governance, and the only way civil society can get the state to do that is by compelling its officers to release information about activities, services, and processes as called for (indeed encouraged) by law.”38 This was exactly the reason PROGATI shifted its focus to efforts to engage media organizations in demanding information under RTI Act in its final year. PROGRAM THEORY AND DESIGN AND PROGRAM OPPORTUNITIES To what extent has the original hypothesis: “Strengthening public institutions, parliament and CSO, including the media, and promoting principles of accountability, transparency, and responsiveness will result in decreased levels of corruption in Bangladesh” on which the PROGATI Project approach was designed prove valid? Is there evidence from PROGATI and similar programs to suggest that alternative program approaches (such as one in which anti-corruption activities are interspersed across the Mission portfolio) may have been more successful? Following Robert Klitgaard’s line of reasoning in 1997, which argued for a thorough assessment of the corruption problem before identifying appropriate solutions, USAID developed and published its own Anti-Corruption Strategy in 2004. One of its recommendations required that USAID Missions carefully delineate programs that were designed to address administrative (petty or incidental) corruption from those designed to address systemic or grand corruption (involving large transactions in the form of access to rents and the exchange of resources or key advantages and favors bestowed to high-level officials, ‘favored’ enterprises, or elite operatives).39 Michael Johnston used this principle to help USAID characterize and sort emerging democracies and post-conflict societies into specific syndromes.40 In short, the Agency’s new strategy asked Missions and implementers to understand the nature of the patron-client game first, and then devise programs in USAID’s manageable interests that differentiated between easy-to-tackle problems and those that required more thought, time, and overall resources. Despite the increasing sophistication of USAID’s assessment and research agenda around corruption, the PROGATI SOW made no critical distinctions between grand and petty corruption or between strategy and tactics. It did not specify a theory of change (TOC), but instead laid out a series of tasks separated by component, with little explanation as to how individual components related to each other by sequence or significance, an approach that seemed to ignore the comprehensive approach advocated for by the Agency’s Anti-Corruption Strategy. The team also found that the SOW USAID created for the project did not address corruption in some sectors, such as land registration, business licensing, and public procurement. Further, it did not require the implementing partner to link project activities with those of other DG programs.41 USAID staff reported that the Agency designed the project to work with 38 Remarks at PROGATI Lessons Learned Workshop, BRAC Center Inn, September 12, 2012. 39 USAID Anti-Corruption Strategy, October 2004, http://pdf.usaid.gov/pdf_docs/PDACA557.pdf, 9. 40 Johnston characterized Bangladesh as a country run by ‘Official Moguls’—elite, family, or clan-based networks who use their influence and positions with the state and its ruling parties to pursue economic wealth. From Syndromes of Corruption: Wealth, Power and Democracy, 2005 (Annex 8 to USAID’s Anti-Corruption Assessment Handbook), http://transition.usaid.gov/our_work/democracy_and_governance/technical_areas/anticorruption_handbook/annex es/Annex%208.pdf, accessed 9/29/2012. 41 USAID SOW for PROGATI, Contract No. DFD-I-04-05-00220-00, 2008. 22 Corruption in the Banking Sector “More than 50% of the loans issued by state-owned commercial banks are non-performing and not backed by adequate capital…[Instead of relying on robust accounting systems], they are being urged to simply make more and more loans….The potential of a genuine financial crisis exists, complete with a run on state banks, and the central bank having to produce large amounts of cash in order to stabilize the situation… It is remarkable that a four-year, anti-corruption project missed one of the greatest sources of corruption in the country. Perhaps USAID thought that the IMF was going to take care of this. As we see, this was a wrong assumption. Surely your evaluation should recognize that the project…missed [one of the] real points.” -A long-time Western observer of Bangladesh’s banking sector the OCAG and Parliament and not other institutions because the Mission did not want to duplicate efforts of other donors, such as those undertaken by the World Bank in procurement reform. But the design limitations also prevented the PROGATI Project from addressing all levels of grand and petty corruption in programming. For instance, while outside the formal scope of this evaluation, the evaluation team saw evidence of significant corruption within government-owned and operated banks during the course of the evaluation. While the evaluation team was in Bangladesh (in September, 2012), the country’s media reported daily on scandals surrounding the financial integrity of state-run banks, such as the suspected Hallmark-Sonali Bank embezzlement case, which involved alleged collusion between the Hallmark Group and Sonali Bank.42 There is no evidence that donors—USAID or others—are addressing the vast corruption in this sector. The evaluation team did not have time to look into any other sectors outside of PROGATI’s scope for further evidence of missed opportunities. PROGATI’s Design PROGATI’s assumptions with regard to its work with government organizations at the national level were never overtly stated, but PROGATI staff reported they were predicated on a supply-driven approach that assumed demand would follow, which remains at the end of the program almost as untested as it was in the beginning. In fairness, USAID said state institutions have tremendous power to shape individual and group behavior, corrupting otherwise honest employees or conversely, re￾channeling would-be corrupt behavior into activities and approaches that contribute to the public good. PROGATI staff said they tried to design the project around this understanding, but the evaluation team found little evidence of a theory of change that for evaluators to test for counterfactuals. BRAC IGS staff and other key stakeholders felt that PROGATI approached anti-corruption with too much of a “checkbox” approach, relying on artificially designed ‘expected results’ rather than more meaningful, organic indicators of change in overall state-society relationships. IMPLEMENTATION AND MANAGEMENT ISSUES To what extent has the PROGATI Project implementation approach been successful in achieving the results of the program? How effective has PROGATI’s performance management system been to achieve program objectives? How effective was USAID management role to implement the program successfully? To what extent has the 42 The team also noted the near-daily mention of the ACC and its work in this area, mostly arraigning and deposing prominent officials connected with the Hallmark Group and its related companies. See “ACC Suspects Links of Many Groups, Individuals,” New Age, September 13, 2012. 23 PROGATI management structure and its mix of acquisition and assistance tools been appropriate for supporting PROGATI objectives in an efficient and flexible way? Delays in Project Start-Up The PROGATI Project got a very late start on its activities. Despite signing its task order with USAID in September, 2007, the PROGATI Project did not achieve significant results until 2010. PROGATI staff reported that the absence of a strategic design in the SOW resulted in delays with the development of an initial work plan and PMP, which took up much of the first year of activities. The OIG Audit Report, however, cites the inability of the project to hire staff on time and procure needed equipment in a timely way as well as uncertainty in the political environment as major reasons for the delay. Whatever the reason for the delay, PROGATI staff said this problem was accentuated by the fact that CSOs “did not really want to talk about corruption in an organized way”. The 2008 elections, held during PROGATI’s second year, further compounded issues by replacing the CTG with the Awami League (AL) Government, with which the project had few working relationships. According to PROGATI, the absence of specific interlocutors in Parliament led to delays in the implementation of project. Management Outside of the slow start-up of the project, the evaluation team found mixed results regarding DAI’s management of the project. Over time, many of the issues with civil society were resolved. In part, PROGATI found resolution through its collaborative working approach, which it insisted on through its work with implementers. This approach basically brought together a group, including PROGATI members working in the five different component areas, members of the government, media representatives, and CSO staff to discuss corruption issues and ensure collaboration and coordination between the different stakeholders and among the different components. The project held 17 meetings using the cross-component approach during the second year alone. This approach and the fact that it was led by Bangladeshi management helped to build local ownership for project components and kept costs low. Nonetheless, multiple-component projects (with cross-component management) are difficult to manage and also make it difficult to attribute various outcomes and impacts, especially in the absence of a clear strategy. As such, PROGATI’s design made it difficult for the evaluation team to make clean judgments of what worked, what did not work, and why. Despite the difficulty of attributing key project results, the evaluation team did identify some challenges with the project—namely in the frequent turnover of key personnel (four COPs in five years and the departure of the original designer of the program at USAID), which several PROGATI partner organizations cited as a major issue affecting continuity and consistency of activities. Additionally, while re-tooling the project at the end of the fourth year toward public institutions allowed the project to respond to new opportunities within those institutions, it also left somewhat unanswered key questions surrounding the expectations and sustainability of the CSO work in the process. Further, sub-contracts and grants tended to be fairly process-oriented. Grants had to be approved by Bangladesh’s NGO review board, Non-Governmental Organization Affairs Bureau, (NGOAB), which delayed distribution of these grants, sometimes by four to six months. But this delay was augmented (not solved) by USAID’s own contractual delays, as grant agreements sometimes took months when they should arguably have taken only days. And finally, the CSO and media work abruptly shifted during the 3rd and 4th year of 24 PROGATI, as has been mentioned, with little involvement or participation and arguably at a cost to a long-term sustainability plan for either component. Performance Monitoring Plan Project indicators had only a “marginal relationship” to the actual activities and results of the project. In the words of one senior PROGATI staff member, “[W]e make assertions about things we cannot really prove because we don’t have the capacity or funding for robust monitoring and evaluation.” Nonetheless, USAID said that PROGATI had a well-developed PMP, a strong M&E manager (who left only in Year 5 of the program), and a budget sufficient for engaging in M&E. While DAI’s TAMIS data management system allowed key PROGATI staff to monitor progress after every event, meeting, workshop, or technical delivery, indicators were mostly output-oriented (focused on activites completed rather than results), with very few outcome-level indicators (i.e., how did this project change the environment for corruption). In fact, the evaluation team found that only 2 of the 19 indicators in the PMP focused on outcomes. The remaining 17 focused on outputs (See Table 3 in Annex Dfor more details). This PMP and data management arrangement made it difficult for PROGATI to facilitate mid-course corrections and improvements. Mix of Acquisition and Assistance Tools The evaluation team was unable to fully assess the effectiveness of PROGATI’s mix of acquisition and assistance tools because it could not obtain consensus among stakeholders as to what the “right” mix of tools should be for a project of this kind. We have further remarks about this issue in the Recommendations section. COST EFFECTIVENESS To what extent was targeting of program beneficiaries appropriate to support cost-effective achievement of program results? The evaluation team was unable to conduct a thorough cost effectiveness study for two main reasons. First, the team could not identify other projects of the same scale or focus with which to compare project costs and results. For instance, the World Bank's procurement programs were of a much larger scale and focused on only one aspect of governance. DFID also has plans for a project in the future, but it has not yet incurred any costs or accrued any results for this project. Second, as mentioned in the limitations section of this report, it is difficult to determine attribution of any results. While the evaluation team can say that PROGATI contributed to some outcomes; it is difficult to say to what extent. Some subcontract work suffered from the use of process indicators alone, making it somewhat more difficult to ascertain effectiveness over time. Nonetheless, the evaluation team found that work under PROGATI’s civil society component consumed the largest share of its budget ($5.3 million by the end of FY2011). Yet that component was also the one least valued by USAID toward the end of the program. The collaborative program design allowed PROGATI to work with CSOs and other organizations to determine the best way to focus its funds. Work with the OCAG cost about $2.8 million by the end of FY2011; while setting up a Parliamentary Budget Unit cost about $2.3 million by the end of FY2011, money which was spent on work with CSOs, the Ministry of Agriculture, and related sub-components. Expenditures in connection with the Compendium and the Help Desk had not accrued by the writing of this evaluation report. Throughout 2012, the burn rate for this component (Component 4) approached $300,000 per month as work accelerated. The evaluation team found that most of the additional $1 million obligated to DAI for its 5th year included work under this Component. 25 SUSTAINABILITY Are processes, systems, and activities in place to ensure that the results of PROGATI activities will be sustainable? Questions of sustainability are closely tied to implementation and management, and the evaluation team found that for many project components, sustainability was not satisfactorily addressed by PROGATI, its subcontractors, or any other project stakeholders. PROGATI staff said that none of the project stakeholders, not USAID,43 PROGATI, or other stakeholders formally thought through or defined what was meant by sustainability of project activities. The evaluation team found that this and problems of measurement and attribution of outcomes made it difficult to measure sustainability of results. Part of this problem of measurability was due to project work plans that did not differentiate between outputs, which do not measure quality of results, and outcomes. Despite the difficulty in defining and measuring sustainability, the evaluation team did identify some characteristics that speak to sustainability of project results:  PROGATI’s work plans and activities focused on building supply without building the demand for that supply.  PROGATI did collaborate extensively with local organizations, such as the BWCC, and also designed processes still in effect today, including the civic forum and citizen report card processes. The team found that the BWCC has institutionalized many of PROGATI’s interventions.  This kind of collaboration was more possible in the civil society arena than it was with Parliament where all donor-funded projects, including PROGATI faced severe relationship constraints.  There is no evidence that PROGATI established a sustainability plan to ensure the ongoing impacts of any of its interventions.  According to the CSOs interviewed, some PROGATI subcontractors found their relationships with PROGATI suddenly terminated for reasons either unclear to them or reasons that reflected differences of opinion (sometimes with PROGATI, sometimes with USAID itself).  Almost all of the CSO partners interviewed added that if PROGATI had a sustainability plan in mind for the CSO component, it was never implemented and engagement simply stopped suddenly. When asked why this was, CSO staff said they believed USAID had basically given instructions to PROGATI to terminate the component and move onto other things (such as greater engagement with OCAG and Parliament).44  The evaluation team found that use of PROGATI-developed resources—the Help Desk, and the Compendium—has increased, though these sources are useful to Parliament only during a few 43 USAID’s Mission Statement does not really define sustainable development, and only mentions that in the DG area, it would include work “that permanently increases the cohesion and productive capacity of the society; and that builds local institutions that involve and empower the citizenry” (http://govinfo.library.unt.edu/npr/library/status/mission/musaid.htm). But in the more detailed 200 ADs series, new policy in place since January 2012 (200.3.1.5—Build in Sustainability from the Start) mentions as first priority the importance of “mak[ing] assistance investments where there is demonstrable local demand and ownership”[emphasis by the evaluation team] http://transition.usaid.gov/policy/ads/200/200.pdf. 44 In February-March 2011, USAID presented its new 5-year strategy to all of its partners. PROGATI subcontractors said the partners ‘got to ask questions’ but were not part of the re-design. 26 weeks in June when the Minister of Finance presents a formal budget. While MPs said they are “satisfied” with these changes, it is also clear that after five years, BAMU is not integrated into formal parliamentary structures, and without continued donor sponsorship, it may not survive. CROSSCUTTING ISSUES How well has PROGATI integrated support to gender equality, the disabled, and minorities into its program? The evaluation team reviewed PROGATI’s gender-focused work in the CSO sector. As mentioned, trainings for the Chamber of Commerce matched the BWCC’s enthusiasm by helping it to identify key issues faced by women entrepreneurs in regional markets. PROGATI’s resources also helped the Bangladesh NGOs Network for Radio and Communication (BNNRC) to train some of the first women journalists, who now provide new perspectives on service delivery issues not previously captured when the field was dominated by men. When asked about PROGATI’s efforts in relation to minorities and the disabled, all stakeholders interviewed said that while the project did not discriminate against any groups in its programming, it also did not conduct any assessments or design any project components to specifically address differential impacts that might accrue to minority groups. CLIENT SATISFACTION—INCONSISTENCIES AMID SUCCESSES What are the levels and areas of satisfaction or dissatisfaction with PROGATI cooperation and performance on the parts of project stakeholders? This group includes: leaders and staffs of Parliament and Parliament Secretariat; leaders and staffs in the OCAG, the IC, the Anti-Corruption Commission and partner ministries; media representatives; NGOs, civil society and citizens’ organizations; and development partners and local organizations that work on anti-corruption issues but who are not PROGATI partners. Media Stakeholders In regards to PROGATI’s work under Component 1 with the media and the Center for Investigative Journalists, JATRI staff interviewed said they appreciated PROGATI’s work with the center but also believed some of the project-provided US STTA was less than valuable and may have been unwanted or unneeded. Additionally, interviewees wondered whether a market for investigative journalism really existed in Bangladesh. CSO Stakeholders The evaluation team found that as a rule most of the CSOs interviewed were largely pleased with the training offered by PROGATI under component 2 as well as the short-term grants and occasional sub￾contracts they received through the project. At the same time, several of them spoke of the often short-term, start-and-stop nature of PROGATI’s engagement, where limited duration work was contracted out, but then relationships with the organization went into a kind of hiatus or dormancy until the next task order, making sustainability of results difficult. PROGATI staff said this may be partly because their CSO engagements reflected a strategy of forming contractual or grant relationships with a select group of subcontractors, rather than a broader strategy to engage and develop civil society more generally. 27 Component 5 Stakeholders Through this component, PROGATI was able to bring together a variety of stakeholders—the IC and ACC, OCAG, MRDI, BEI, and others—particularly during the last year. All such stakeholders said the RTI Act was one of the best such acts in South Asia, and it had created a sudden, unexpected opening for engagement at many different levels. They also said that PROGATI had done a good job taking advantage of that opening. Staff from the IC said the law created a new, informal ‘supply and demand’ for integrity-related services, as thousands of government bodies struggled to respond to new requests for information. The IC staff said these requests diminished in 2011, but they said this may have been due to some government institutions actually starting to respond more proactively before the formal request process (a positive externalist of the law). Further, staff from the IC, ACC, OCAG, MRDI, and BEI expressed a strong sense that after the passing of the RTI Act, ‘oversight’ work was perhaps more important than ever. Many of them said that they wondered what would happen without a specific donor-supported engagement in this area. USAID USAID/Bangladesh/DG expressed a sense of frustration to the evaluation team with regard to the complexity of PROGATI’s framework, mentioning USAID’s desire for a follow-on activity of some kind “with fewer moving parts”. CONCLUSIONS AND LESSONS LEARNED RESULTS In recent months, government and donor stakeholders in Bangladesh have reported a general sense that corruption is on the rise again in the county. This is not a positive development, but the donor community is not powerless to work with Bangladeshi society to help reverse this.45 The country’s robust civil society must be re-engaged to reverse the backsliding of recent years. However incompletely, PROGATI did establish an important foundation to continue this effort. PROGRAM DESIGN Over the past decade in particular, USAID’s Center of Excellence on Democracy, Human Rights and Governance has accumulated considerable expertise in the field of anti-corruption programming from diverse geographical and developmental backgrounds. Its 2004 Anti-Corruption Strategy stressed the importance of strategic distinctions in mission programming designed to address both grand (systemic) and petty (incidental) corruption and lodge the approach in a specific theory of change that could be 45 “[T]he rule of law has deteriorated quite rapidly in the last nine or ten months, and some of the efforts at governance reform do seem to have been stalled or, in some cases, reversed.” House of Commons, International Development Committee, 3rd Report of Session 2009-10, “DFID’s Programme in Bangladesh,” p. 8 http://www.publications.parliament.uk/pa/cm200910/cmselect/cmintdev/95/95i.pdf. 28 tested by external evaluators over time. PROGATI’s design did not incorporate such a strategy, but instead served as a pilot program for USAID/Bangladesh to fill in assistance gaps left behind by other donors. The advantages to such a program included response flexibility (resources could be, and were, mobilized to address emerging issues in rapid fashion). But the disadvantages were that a coherent strategy and accompanying theory of change were never clearly specified. Thus, one clear lesson learned is that a future anti-corruption program should devote more effort into identifying a theory of change that could be empirically tested by external evaluators over time. MEDIA Despite limitations of program design, PROGATI managed to supply specific deliverables that most observers would agree form part of the overall environment considered necessary (though not sufficient) for addressing corruption. While there were delays and disagreements over the best ways to render TA and definitions of sustainability, PROGATI established a Center for Investigative Journalism in Bangladesh, which has become a fully integrated component within a larger, thriving commercial media market. Questions remain, however, as to how much that commercialization reflects the profitability of other kinds of media services, as opposed to investigative journalism. One lesson that emerged from the media sector was that definitions of sustainability needed to be agreed upon by all parties at the project’s outset. Civil Society In the civil society arena, one of PROGATI’s strongest accomplishments (and the one that received the greatest amount of resources from USAID) was its ability to convey “life after project” technical resources, skillsets and deliverables that strengthened, systematized or transferred organizational capacity to a sector that was galvanized around the issue of corruption, but lacked specific knowledge of how to translate that mobilization into concrete action. PROGATI’s work helped strengthen public￾private partnerships with different levels of government and civil society by assigning specific roles, follow-up activities and watchdog functions to both parties. PROGATI’s own civil society surveys showed that this work led to an increase in social trust and corresponding decreases in people’s subjective perceptions of victimization as a result of corruption over time. Yet this work formally ended by Year 4, and the evaluation team heard convincing arguments for re-energizing the civil society sector specifically around the RTI law. A genuine theory of change could even be constructed around future work with that sector. For example, can the increasingly sophisticated demand for information from civil society, coupled with increasingly detailed compliance by government at all levels, really improve overall governance in Bangladesh? Could a future program where civil society was re-energized specifically around the specifics stemming from RTI strengthen the chances for this? Parliament and OCAG In work with Parliament and with the OCAG, different and more complicated lessons can be drawn. PROGATI itself remained uncertain as to whether the BAMU will be institutionalized within Parliament’s library and research functions as the program is transferred to PRODIP. Even if it is, it remains unclear how this knowledge will help members of Parliament, who are subject to frequent media criticism for not taking greater interest in their own oversight functions, to fulfill that aspect of their jobs more efficiently. With regard to the Comptroller’s office, PROGATI seems to have played a supporting though not instrumental role. It certainly helped to professionalize its staff, but the office itself still does not routinely engage in program or performance-based audits. Observers point out that the ability of the office to conduct investigations still depends on the willingness of the Ministry of Finance to introduce a draft Audit Act that funds this office at levels that guarantee its independence. Similar levels of concern can be heard over the future status of the office after the departure of the present Comptroller in early 2013. A lesson learned for any future program work with an agency or department of government, or 29 even an independent office such as the OCAG, is that a strategy should make greater efforts toward detailing specific, more stable, predictable outcomes in future years to justify the investment made in that office. Possible Opportunities Ahead The issue of corruption has been gaining greater public attention throughout South Asia as a whole over the past few years, and it is likely that Bangladesh will also be caught in this larger—and welcome— wave. USAID/Bangladesh should be prepared to capitalize on sudden cross-border opportunities that may arise. In India, an integrity movement that arguably began in poorer, less urban areas seems now to be escalating, and is increasingly being accepted by the urban and middle classes in the country’s cities frustrated with the pace of political change in their own country. It is not unreasonable to imagine similar processes unfolding in Bangladesh over time. Opportunities for civil society watchdogs in one country to study and learn from each other could be encouraged in a future program that would seek to address this need in a more systematic way than had been possible before. RECOMMENDATIONS Based on the findings and conclusions presented above, the evaluation team has identified the following recommendations: Recommendation 1: USAID should continue the engagement process around the new RTI legislation, working closely with a Bangladeshi Technical Advisory Committee (TAC) in the process. Should additional resources be available for FY2013, more concentrated engagements with government—in conjunction with DFID, the World Bank the Asian Development Bank (ADB), Danish International Development Agency (DANIDA), and others working to promote increased civic awareness of transparency-related issues—could be employed in more collaborative ways than have been used previously. Recommendation 2: Provide future assistance through a Bangladeshi-led Cooperative Agreement, augmented by a local Technical Advisory Committee . While a contract-based system of TA offers advantages to USAID in terms of focused delivery and greater financial accountability, its core disadvantage is that it lodges ultimate responsibility for program strategy, mid-course corrections and tactical approaches with the US Government, rather than with the host country. The nature of the assistance itself translates to more passive involvement by counterparts and stakeholders. After 5 years of engagement, both USAID and Bangladesh have reached a literal and symbolic fork in the road. USAID Forward reforms rightly place future onus for program direction with local actors (a combination of state, society and the private sector). A post-PROGATI cooperative agreement with a local consortium would be a good arena for this, especially given the sensitivities surrounding corruption. Its work could be augmented with a small 5-7 member TAC, made up of former government officials, emerging private sector groups, and influential members of the implementer/civil society and microcredit communities. TAC members would supply USAID and the larger anti-corruption donor network) a series of deliverables—recommendations on future direction for programs, ways for the cooperator(s) to engage government and parliament more directly, specific platforms for the acceleration of RTI implementation. The TAC could help do what PROGATI often could not—offer political direction, provide institutional guidance, further communication between 30 Strengthening the Office of the Public Prosecutor USAID’s Knowledge Management database contains instructive examples of best practices from other countries in this sector. Working with the Department of Justice’s OPDAT program, USAID might support an Economic Crimes Unit responsible for monitoring money-laundering proceeds—a complex technical area involving skilled expertise and significant inter￾governmental cooperation. USAID might provide training necessary to develop an information management system to track currency flows, provide case management and guide prosecutors toward successful interventions. current and past government officials, build political capital, guide implementation through pitfalls, publicize integrity champions more effectively, troubleshoot through obstacles and logjams. Ultimately, the work would be owned not by the USG, but by Bangladesh, in accordance with an emerging global consensus on aid ownership.46 USAID Forward itself envisions that under such arrangements, a cooperator would ‘flip the relationship’ between contractors and sub-implementers, purchasing short￾term contractual STTA (from any country) as needed to provide specific services.47 Recommendation 3: Re-Engage CSOs Around RTI Issues Specifically A more strategic design for a post￾PROGATI cooperative agreement could focus on the implementation of the RTI Law. CSOs have largely been left behind by this process (despite their support for the original legislation). The process will test both the government’s commitment as well as that of CSOs to a specific accountability process and will help satisfy an entirely new supply and demand sub-market for information, analysis and response that watchdog groups are well-positioned to seize. USAID is encouraged to consider roles for rural cooperatives and microcredit groups, who have important stakes in successful implementation of the law. Recommendation 4: ‘Focus and Concentrate’ Resources and Technical Assistance on an Integrity Model Timing issues surrounding contract issuance in 2008, coupled with new elections, severely hampered PROGATI’s implementation as has been pointed out. To avoid these issues, a post-PROGATI project should wait for the outcome of the next election. Once in power, USAID (rather than its implementer or partner) should then enter into a dialogue with the Government’s, using the Cabinet or Principal Secretary as primary interlocutor to enlist its support for an intensive (yet also hands-off) program working with a single agency, department, or even ministry to develop its integrity plan. In consulting with interviewees, the Team sought opinions whether they thought this made sense, and then if so, what agency should be the focus. The answers exhibited a consensus that choosing one thing (and doing it well) made more sense than working on a variety of components in a superficial manner. Many responses to the Team singled out the Ministry of Local Government, Rural Development and Cooperatives (MoLGRD), both because of the amount of resources that pass through it as well as its potential to solve problems at levels that mattered both to the emerging private sector and by ordinary people. While USAID is currently supporting aspects of this Ministry’s work through LGED, and did single it out in terms of RTI compliance, it may wish to re-visit the MoLGRD for a post-PROGATI Project. 46 See the 4 th High Level Forum on Aid Effectiveness, Nov-Dec 2011, Busan, Korea, http://www.aideffectiveness.org/busanhlf4/. 47 Personal communication to the author by USAID’s AA for Food Security and General Counsel, summer 2012. 31 Other agencies included land registry, addressing the civil case backlog (80% of which are land disputes), or the creation of an independent prosecutorial service. Whatever it is, donors would collaborate to offer a ‘soup to nuts’ approach for that agency, involving every aspect of its operations—hiring, firing, procurement, auditing, and offering new salary structures. Seconded personnel from donor countries would work directly in that department in return for a hands-off commitment by Government to permit and encourage integrity work. Negotiating this with Government would be challenging and the multiple commitments of shared financing and responsibility across donors difficult, but the advantages would include a significantly increased change for a more positive outcome by reducing piecemeal, often duplicative and superficial approaches by multiple donors. 32 ANNEXES 33 ANNEX A: EVALUATION STATEMENT OF WORK Scope of Work for the Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Project Performance Evaluation USAID/Bangladesh Office of Democracy and Governance Program Identification Data Program Title: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Project Program Number: Task Order DFD-I-04-05-00220-00 Program Dates: October 2007 - September 2012 Program Funding: $19,085,875 Implementing Organization: Development Alternatives Inc. (DAI) Contracting Officer Representative (COR): Sherina Tabassum I. Background As the seventh most populous country in the world with the fourth largest Muslim population, Bangladesh’s stability and prosperity are important to U.S. foreign policy interests. Bangladesh is poor, vulnerable to natural disasters, and susceptible to social upheaval and political conflict. U.S. Government (USG) foreign assistance is helping Bangladesh build its democratic institutions, reduce poverty through broad-based based economic growth and improve its human capital. Since independence, Bangladesh has held democratic elections sporadically, including three successive peaceful transfers of power from 1991 to 2006. Yet, the country’s development as a democracy has been interlaced with military rule, debilitating political polarization, ineffective institutions of governance, and endemic corruption. Partisan deadlock over electoral issues resulted in the cancellation of the January 2007 parliamentary elections and the self-imposed rule by the military-backed CTG. Following the two-year state of emergency enforced by the CTG, a new chapter in the country’s history opened for democratic rule with the success of the free and fair parliamentary elections held on December 29, 2008. The difficult but successful transition back to elected government has brought about new opportunities for democratic development and a new sense of urgency in the country to reform political practices and institutions of governance. This new chapter also presented the United States with an historic opportunity to help Bangladesh improve the country’s governance, foster economic development and deny space to terrorists. Bangladesh has many formal structures of democracy, but most function in an undemocratic manner with limited transparency, accountability, and openness to public input. While democratic institutions exist including parliament, a free and vigorous print media, the judiciary, political parties, competitive elections, and free speech, consensus among the national leadership about the policy-making process is absent. National politics is about competition for power and resources for a few political elites and their followers, and only rarely about domestic issues important to citizens. Parties in power fail to distinguish between the partisan interests of their party and those of the nation. Pervasive corruption significantly limits Bangladesh’s development and erodes public support for its democratic institutions. Corruption also makes Bangladesh more vulnerable to social and political upheaval, instability, and violence and provides a breeding ground for terrorism. A key objective of the 34 United States Government’s (USG) foreign assistance in Bangladesh is to reduce corruption by promoting greater transparency and accountability in the management of public resources. USAID’s approach emphasizes transparency in public management and enhancing the watchdog capabilities of CSO and the media. In September2007, USAID awarded a task order to DAI to implement a four-year anti-corruption program known as PROGATI with a total estimated cost (TEC) of $18,185,876. The task order was subsequently modified in September 2011 to extend the program up to September 2012 and increase the TEC to $19,085,875. At a minimum, the contract specified that DAI, Inc. achieve the following principal objectives: (7) Strengthen a Parliamentary Budget Unit; (8) Provide TA to support a civil society anti-corruption coalition development, public-private partnerships and significant citizen advocacy and watchdog initiatives; (9) Increase citizen group participation in understanding and developing program-based budgets for more external oversight of GOB budgets; (10) Support the mass media to build its capacity as a public watchdog through the establishment of a Center for Investigative Journalism; (11) Support parliamentary oversight committees and parliamentary accountability reforms; and (12) Continue to support the Parliamentary Budget Analysis and Monitoring Unit during its transition to USAID’s legislative strengthening program. A number of additional objectives are included in the task order SOW to promote principles of accountability, transparency, and responsiveness to decrease the level of corruption in Bangladesh. These additional objectives are provided under the DCHA/DG BRDG IQC. The PROGATI Project anticipated the achievement of these objectives by a two pronged approach, which entails: (1) strengthening public institutions, including civil servants and elected officials to better carry out their oversight functions, and (2) strengthening CSOs, including media to generate demand for increased oversight. PROGATI was to focus on five components: media, civil society, public institutions, parliament and windows of opportunity. Policy reform, while not listed as a separate component, was to be emphasized and incorporated under each of the components. PROGATI was expected to build on recent accomplishments of past anti-corruption programs funded by other donors as well as establish synergies with partners involved in on-going USAID Bangladesh work and with other USAID objectives (e.g., population, health, nutrition, and education; economic growth; and food, disaster and humanitarian assistance). PROGATI expected to achieve the following results:  Increase the capacity of the Parliament and its members to analyze, monitor and influence national policy and budgetary priorities and also strengthen its oversight capacity;  Enhance the effectiveness of internal procedures of selected public institutions of accountability dealing with public finance monitoring and auditing, transparency and accountability and right to information;  Build new and strengthen existing civil society coalitions, networks and public-private partnerships to increase awareness on corruption issues;  Encourage opportunities for citizen participation in and oversight of the national budget and government decision-making processes;  Foster media capacity to report on transparency and accountability issues and advocate for legal reforms; and, 35  Advance the dissemination of information by selected public institutions at both national and local levels to promote citizen access to improved government information. II. Objectives of the Evaluation This external performance evaluation will assess the results/outcomes and lessons learned from PROGATI’s five-year program. The evaluation will allow the Mission to make more informed decisions as they embark on integrating USAID Forward into its programming. The evaluation will:  Evaluate PROGATI overall performance by assessing results against contractual goals and indicators;  Assess the efficacy, cost-effectiveness, and results of the PROGATI implementation approaches and management structure in meeting the objectives;  Make recommendations to USAID/Bangladesh concerning future interventions and direct programming of local organizations, including those that DAI worked with under PROGATI. The audience for this evaluation is USAID/Bangladesh, USAID/Washington leaders of USAID Forward, DAI, and existing USAID partners. III. Evaluation Questions This SOW is for a performance evaluation of the approximately five years of implementation of the PROGATI Project (October 2007 to September 2012). The evaluation should review, analyze, and evaluate the PROGATI Project along the following criteria, and, where applicable, identify opportunities and recommendations for improvement. In answering these questions, the Evaluation Team should assess the performance of both USAID and its implementing partner(s). A. Results. 1. To what extent has PROGATI been successful in achieving program objectives? (Describe what have been the most significant outcomes of activities implemented under PROGATI, explore variation in performance - may be performance has been relatively successful in some areas but less successful in others, explain the way and effectiveness of program components of PROGATI in contributing to program objectives) 2. Were there any significant unintended results (positive or negative)of PROGATI interventions? B. Program Theory and Design. 3. To what extent has the original hypothesis “Strengthening public institutions, parliament and CSOs, including the media, and promoting principles of accountability, transparency, and responsiveness will result in decreased levels of corruption in Bangladesh.” on which the PROGATI Project approach was designed proved valid? (explain validity of PROGATI Project assumptions regarding requirements for promoting anti-corruption, good governance and integrity, describe the extent to which political events during the implementation period affect the success or limitations of PROGATI interventions) 36 C. Implementation and Management. 4. To what extent has the PROGATI Project implementation approach been successful in achieving the results of the program? ( Describe effectiveness of implementation methods, indicate coordination level with various USAID/ Bangladesh technical teams i.e., Economic Growth, Population/Health/Nutrition and Education, Food Disaster and Humanitarian Assistance as well as within other DG activities, describe coordination with the various other donors’ anti-corruption activities) 5. How effective has PROGATI’s performance management system been to achieve program objectives? (Explain relevance of indicators to manage results) 6. How effective was USAID management role to implement the program successfully? 7. To what extent has the PROGATI management structure and its mix of acquisition and assistance tools (i.e., STTA, sub-contracts, sub-grants, training, etc.) been appropriate for supporting PROGATI objectives in an efficient and flexible way? D. Cost Effectiveness. 8. To what extent was targeting of program beneficiaries appropriate to support cost-effective achievement of program results? (describe if results achieved under PROGATI are being produced at a reasonable cost as compared to other programs the Evaluation Team may identify in Bangladesh with similar objectives; explain the extent PROGATI took advantage of cost-share and/or leveraging opportunities) E. Sustainability. 9. Are processes, systems, and activities in place to ensure that the results of PROGATI activities will be sustainable? (Describe evidence indicating that the GOB and other Bangladeshi partners will take ownership of PROGATI activities, describe obstacles that exist for achieving or maintaining sustainability) F. Cross Cutting Issues. 10. How well has PROGATI integrated support to gender equality, the disabled, and minorities into its program? G. Client Satisfaction (GOB, beneficiaries, other stakeholders) 11. What are the levels and areas of satisfaction or dissatisfaction with PROGATI cooperation and performance on the parts of project stakeholders? This group includes: leaders and staffs of Parliament and Parliament Secretariat; leaders and staffs in the OCAG, the IC, the Anti-Corruption Commission and partner ministries; media representatives; NGOs, civil society and citizens’ organizations; and development partners and local organizations that work on anti-corruption issues but who are not PROGATI partners. H. Program Opportunities 12. Is there evidence from PROGATI and similar programs to suggest that alternative program approaches (such as one in which anti-corruption activities are interspersed across the Mission portfolio) may have been more successful? 37 IV. Proposed Evaluation Methodology The Evaluation Team will apply a multi-method approach to this performance evaluation. This will include:  Review of background documents, including, for example: the technical SOW from the PROGATI task order, the Mid-Term Evaluation of PROGATI (March 2011); all PROGATI Annual and Quarterly Reports; a DAI TAMIS data summary on PROGATI performance vis-à-vis the project’s Monitoring and Evaluation Plan; reports from PROGATI-sponsored surveys and analytical research; lists of PROGATI grantees and subcontractors, with activity descriptions, levels of PROGATI resources and points of contact  KIIs with US-based sources (e.g., at DAI Headquarters in Bethesda, Maryland; and USAID/Washington);  KIIs or group interviews (as circumstances allow) with selected PROGATI stakeholder organizations;  A mini-survey (via personal interviews) with representatives of client organizations; and  Interviews with TI/Bangladesh, and associated analysis of data as relevant and available, to address the question regarding PROGATI impact on the TI ranking. The Evaluation Team should review the proposed methodology and alternative methods in light of the evaluation questions, timeframe and budget of the exercise, data collection requirements, the quality of existing data sources, and potential biases. The team will build on the proposed methodology and provide more specific details on the evaluation methodology in the Evaluation Work Plan (see Deliverables below). The evaluation will be participatory in its design and implementation and the evaluation methodology will be finalized through further review and discussion between USAID/Bangladesh and the Evaluation Team. The methodology narrative should discuss the merits and limitations of the final evaluation methodology. The Evaluation Team will design appropriate tools for collecting data from various units of analysis. The tools will be shared with USAID during the evaluation and as part of the evaluation report. The information collected will be analyzed by the Evaluation Team to establish credible answers to the questions and provide major trends and issues. USAID requires that evaluations explore issues of gender; thus, the evaluation should examine gender issues within the context of the evaluation of PROGATI activities. The Evaluation Team will be required to perform tasks in Dhaka, Bangladesh and also will travel to activity sites within the country. Priority locations for data collection outside Dhaka are Khulna, Rajshahi and Chittagong. V. Existing Sources of Information USAID/Bangladesh DG Office will provide documents for the desk review and contact information for relevant interviewees. In-country staff of Social Impact’s Bangladesh Democracy and Governance Program Evaluations (BDGPE) project also have established a working relationship with PROGATI staff, to support sharing of information for the evaluation. The Evaluation Team will be responsible for identifying and reviewing additional materials relevant to the evaluation. 38 VI. Deliverables Evaluation deliverables include: Evaluation Team Planning Meeting (s)– essential in organizing the team’s efforts. During the meeting (s), the team should review and discuss the SOW in its entirety , clarify team members’ role and responsibilities, work plan, develop data collection methods, review and clarify any logistical and administrative procedures for the assignment and instruments and to prepare for the in-brief with USAID/Bangladesh Work Plan - Detailed draft work plan (including task timeline, methodology outlining approach to be used in answering each evaluation question, team responsibilities, and data analysis plan): Due to USAID/Bangladesh by 22 August Dhaka time; In-brief Meeting - In-brief with USAID/Bangladesh: Within 2 working days of international team members’ arrival in Bangladesh; Data Collection Instruments–Development and submission of data collection instruments to USAID/Bangladesh during the design phase; Regular Updates - The Evaluation Team Leader (or his/her delegate) will brief the BDGPE COR on progress with the evaluation on at least a weekly basis, in person or by electronic communication. Any delays or complications must be quickly communicated to USAID/Bangladesh as early as possible to allow quick resolution and to minimize any disruptions to the evaluation. Emerging opportunities for the evaluation should also be discussed with USAID/Bangladesh. Debriefing with USAID - Presentation of initial findings, conclusions and preliminary recommendations to USAID/Bangladesh before the Evaluation Team departs from Bangladesh; Debriefing with Partners - The team will present the major findings from the evaluation to USAID partners (as appropriate and as defined by USAID) through a PowerPoint presentation prior to the team’s departure from the country. The debriefing will include a discussion of achievements and activities only, with no recommendations for possible modifications to project approaches, results, or activities. The team will consider partner comments and incorporate them appropriately in drafting the evaluation report. Debriefing with USAID/W - Presentation of evaluation findings and recommendations to USAID/W (upon USAID/Bangladesh request); timeframe will be coordinated between USAID/Bangladesh and USAID/W; Draft Evaluation Report - A draft report on the findings and recommendations should be submitted to USAID/Bangladesh within 2 weeks after departure of international team members from Bangladesh. The written report should clearly describe findings, conclusions, and recommendations. USAID will provide comments on the draft report within ten working days of submission; Final Report - The Team will submit a final report that incorporates the Mission’s comments and suggestions no later than five days after USAID/Bangladesh provides written comments on the team’s draft report. 39 The final report should meet the following criteria to ensure the quality of the report:  The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why.  Evaluation reports shall address all evaluation questions included in the SOW.  The evaluation report should include the SOW as an annex. All modifications to the SOW, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Evaluation findings will assess outcomes and impact on males and females.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. The format of the final evaluation report should strike a balance between depth and length. The report will include a table of contents, table of figures (as appropriate), acronyms, executive summary, introduction, purpose of the evaluation, research design and methodology, findings, conclusions, lessons learned and recommendations. The report should include, in the annex, any substantially dissenting views by any team member or by USAID on any of the findings or recommendations. The report should not exceed 30 pages, excluding annexes. The report will be submitted in English, electronically. The report will be disseminated within USAID. A second version of this report excluding any potentially procurement-sensitive information will be submitted (also electronically, in English) by Social Impact to USAID’s Development Experience Clearinghouse (DEC) for dissemination among implementing partners and stakeholders. All quantitative data, if gathered, should be (1) provided in an electronic file in easily readable format; (2) organized and fully documented for use by those not fully familiar with the project or the evaluation; (3) owned by USAID and made available to the public barring rare exceptions. A thumb drive with all the data could be provided to the COR. The final report will be edited/formatted by Social Impact and provided to USAID/Bangladesh 15 working days after the Mission has reviewed the content and approved the final revised version of the report. 40 VII. Team Composition/Technical Qualifications and Experience Requirements for the Evaluation Team USAID/Bangladesh wants to engage the services of a team of four local and international consultants to work on the evaluation team. The evaluation team will require: 1. An international Senior Evaluation Specialist with experience in evaluating anti-corruption programs in developing countries (Team Leader). The team leader will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to USAID/Bangladesh. At least ten (10) years of experience in evaluation management is required. Experience in conducting assessments and designing strategic responses to corruption in developing countries is required. Ability to produce highly quality evaluation report in English is essential. 2. An international Senior Sector Specialist or Evaluation Specialist, also with experiencein evaluation of anti-corruption programs in developing countries (Senior International Specialist); At least seven (7) years of experience in DG programs and some experience managing or implementing anti￾corruption programs in developing countries is required. Experience in assessing: the root causes of corruption in transitional economies and democracies; the impact of corruption on national economy and politics; barriers to anti-corruption initiatives; and strategic development is critical. Knowledge of anti-corruption literature is required. Asian/regional experience is desired. Ability to conduct interviews and discussions and write well in English is essential. 3. Two Bangladesh-based Senior or Mid-Level Evaluation or Sector Specialists (Local Specialists): Team members experience should include post-graduate level economics, social science, law, and/or public finance training. In-depth knowledge of issues relating to anti-corruption, government transparency, and civil society oversight of corruption in developing and/or transitional economies and democracies is required. Familiarity with state-of-the-art anti-corruption “best practices” methods and programming is essential. Some experience in conducting evaluations or assessments is expected. Ability to conduct interviews and discussions and write well in English is essential. Overall the team will need expertise in USAID practices and expectations in program evaluation; program design and analysis; quantitative and qualitative data collection and analysis; survey design and analysis; program issues, innovations and challenges in promotion of public sector transparency and accountability; and USAID practices and requirements in program performance measurement. The team will be supported by 1-2 interpreter/translators (as needed) through the auspices of the BDGPE project. VIII. Conflict of Interest All evaluation team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the project being evaluated. USAID/Bangladesh will provide the conflict of interest forms. 41 IX. SCHEDULING AND LOGISTICS Funding and Logistical Support The proposed evaluation will be funded and implemented through the BDGPE project. Social Impact will be responsible for all off-shore and in-country administrative and logistical support, including identification and fielding appropriate consultants. Social Impact support includes arranging and scheduling meetings, international and local travel, hotel bookings, working/office spaces, computers, printing, photocopying, arranging field visits, local travel, hotel and appointments with stakeholders. The evaluation team should be able to make all logistic arrangements including the vehicle arrangements for travel within and outside Dhaka and should not expect any logistic support from the Mission. The team should also make their own arrangement on space for team meetings, and equipment support for producing the report. Schedule The evaluation is planned to follow the schedule and Level of Effort breakdown given below. PROGATI EVALUATION SCHEDULE Task/ Deliverable Proposed Dates Review background documents & preparation work (offshore): Draft work plan submitted to SI’s technical backup for review by 8/21 and to USAID/Bangladesh by 8/22 (Dhaka time) 8/15 – 8/26 Travel to Bangladesh by expat team members 8/27 – 8/28 Team Planning Meeting at BDGPE office 8/29 In-brief with USAID/Bangladesh and prepare for field work 8/30 Data collection 8/31 – 9/10 Analysis and report drafting in-country 9/11 – 9/14 Evaluation Team submits annotated report outline and draft presentation for USAID/Bangladesh DG Team review 9/15 USAID provides comments (as needed) on report outline and draft presentation 9/16 Presentation and debrief with DG Team 9/17 Presentation to USAID/Bangladesh 9/18 Preparation for debrief meetings with stakeholders 9/18 Debrief meetings with key stakeholders, including GOB 9/19 Expat team members depart Bangladesh 9/20 SI delivers draft report to DG Team 9/27 Evaluation Team delivers presentation to USAID/W (upon DG Team request) 10/01 USAID and partners provide comments on draft 10/12 Team revises draft report 10/13 - 10/16 Social Impact edits/formats report 10/17 SI delivers final report 10/18 42 Level of Effort Note: LOE does not include LOE of: (a) the USAID staff member participating on the team; (b) quality assurance provided by BDGPE’s Chief of Party; (c) product editing and formatting from SI HQ; or (d) logistical/management support from BDGPE or SI HQ staff. Task/ Deliverable Level of Effort (LOE) Team Leader Int’l Specialist Local Eval Specialist Local Sector Specialist Review background documents & preparation work (offshore): Draft work plan submitted to SI’s technical backup for review by 8/21 and to USAID/Bangladesh by 8/22 (Dhaka time) 5 days 5 days 5 days 5 days Travel to Bangladesh by expat team members 2 2 -- -- Team Planning Meeting at BDGPE office 1 1 1 1 In-brief with USAID/Bangladesh and prepare for field work .5 .5 .5 .5 Data collection 8.5 9.5 9.5 9.5 Analysis and report drafting in-country 3 4 4 4 Evaluation Team submits annotated report outline and draft presentation for USAID/Bangladesh DG Team review 1 1 1 1 USAID provides comments (as needed) on report outline and draft presentation; Evaluation Team refines presentation 1 1 1 1 Presentation and debrief with DG Team 1 1 1 1 Presentation to USAID/Bangladesh 1 1 1 1 Preparation for debrief meetings with stakeholders 1 1 1 1 Debrief meetings with key stakeholders, including GOB 1 1 1 1 Expat team members depart Bangladesh 2 -- -- -- SI delivers draft report to DG Team -- -- -- -- Evaluation Team Leader delivers presentation to USAID/W (upon DG Team request) 1 -- -- -- USAID and partners provide comments on draft report -- -- -- -- Team revises draft report 3 2 2 2 Social Impact edits/formats report -- SI delivers final report --- TOTAL LOE by team member 32 31 28 28 TOTAL LOE: Full team 119 working days 43 X. REPORTING REQUIREMENTS The total pages of the final report, excluding references and annexes, should not be more than 30 pages. The following content (and suggested length) should be included in the report: Table of Contents Acronyms Executive Summary - concisely state the project purpose and background, key evaluation questions, methods, most salient findings and recommendations (2-3 pp.); 1. Introduction – country context, including a summary of any relevant history, demography, socio-economic status etc. (1 pp.); 2. The Development Problem and USAID’s Response - brief overview of the development problem and USAID’s strategic response, including design and implementation of the PROGATI project and any previous USAID activities implemented in response to the problem, (2-3 pp.); 3. Purpose of the Evaluation - purpose, audience, and synopsis of task (1 pp.); 4. Evaluation Methodology - describe evaluation methods, including strengths, constraints and gaps (1 pp.); 5. Findings/Conclusions - describe and analyze findings for each objective area using graphs, figures and tables, as applicable, and also include data quality and reporting system that should present verification of spot checks, issues, and outcomes(12-15 pp.); 6. Lessons Learned - provide a brief of key technical and/or administrative lessons on what has worked, not worked, and why for future project or relevant program designs (2-3 pp.); 7. Recommendations – prioritized for each key question; should be separate from conclusions and be supported by clearly defined set of findings and conclusions. Include recommendations for future project implementation or relevant program designs and synergies with other USAID projects and other donor interventions as appropriate (3-4 pp.); Annexes – to include statement of work, documents reviewed, bibliographical documentation, evaluation methods, data generated from the evaluation, tools used, interview lists, meetings, FGDs, surveys, and tables. Annexes should be succinct, pertinent and readable. Annexes should also include if necessary, a statement of significant unresolved difference of opinion by funders, implementers, or members of the evaluation team on any of the findings or recommendations. The report format should be restricted to Microsoft products and 12-point type font should be used throughout the body of the report, with page margins one inch top/bottom and left/right. 44 ANNEX B: LIST OF DOCUMENTS REVIEWED/BIBLIOGRAPHY ARD, Inc. (2009, August). Bangladesh Democracy and Governance Assessment. Ashman, Darcy. (6 April 2010). Depth of Anti-Corruption Partnerships, Baseline Study. Promoting Governance, Accountability, Transparency and Integrity (PROGATI) in Bangladesh, Research Report. Ashman, Darcy, DBA, and Md. Mafizur Rhaman MA, MS, S.M., Shafiul Islam, MA, MSS. (29 August 2011). Depth of Anti-Corruption Partnerships, An Endline Study. Promoting Governance, Accountability, Transparency and Integrity (PROGATI) in Bangladesh, Research Report. Bangladesh Enterprise Institute (2009, February). Report on Mapping CAG Functions and Reporting Channels. Dhaka. Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI). Media Sustainability Index, Bangladesh-2007. USAID Contract No. DFD-I-00-05- 00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI). Media Sustainability Index, Bangladesh-2008. USAID Contract No. DFD-I-00-05- 00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI). Media Sustainability Index, Bangladesh-2009. USAID Contract No. DFD-I-00-05- 00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI). Media Sustainability Index, Bangladesh-2010. USAID Contract No. DFD-I-00-05- 00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Semi Annual Report – 1 October 2007 to 31 March 2008. USAID Contract No. DFD-I-00-05- 00220-00. Order No. DFD-I-04-05-00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Semi Annual Report – 1 April to 30 September 2008. USAID Contract No. DFD-I-00-05-00220- 00. Order No. DFD-I-04-05-00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) 45 Semi Annual Report – 1 October 2008 to 31 March 2009. USAID Contract No. DFD-I-00-05- 00220-00. Order No. DFD-I-04-05-00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Semi Annual Report – 1 April to 30 September 2010. USAID Contract No. DFD-I-00-05-00220- 00. Order No. DFD-I-04-05-00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Semi Annual Report – 1 October 2010 to 31 March 2011. USAID Contract No. DFD-I-00-05- 00220-00. Order No. DFD-I-04-05-00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Semi Annual Report – 1 April to 30 September 2011. USAID Contract No. DFD-I-00-05-00220- 00. Order No. DFD-I-04-05-00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI); Quarterly Report – 1 October to 31 December 2011. USAID Contract No. DFD-I-00-05- 00220-00. Order No. DFD-I-04-05-00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Quarterly Report – 1 January to 31 March 2012. USAID Contract No. DFD-I-00-05- 00220-00. Order No. DFD-I-04-05-00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Quarterly Report – 1 April to 30 June 2012. USAID Contract No. DFD-I-00-05-00220- 00. Order No. DFD-I-04-05-00220-00. USAID Contractor: Development Alternatives, Inc. (DAI). Bashrach, Elinor B. (2009, April). Bangladesh Parliament Secretariat Budget Analysis & Monitoring Unit (BAMU) – Needs Assessment and Recommendations Report. Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Delivered to USAID. Development Alternatives, Inc. (2009, July-October). Survey Report: Citizens' Perception of News Media. Baseline Study for Promoting Governance, Accountability, Transparency and Integrity (PROGATI). Conducted by Bangladesh Centre for Development, Journalism and Communication (BCDJC). Development Alternatives, Inc. (DAI) (2011, June). Survey on CAG Audit Skill – Component Level Indicator 3.5.1. Document produced for review by USAID. 46 Development Alternatives, Inc. (2011). Survey Report: Citizens' Perception of News Media. Endline Study for Promoting Governance, Accountability, Transparency and Integrity (PROGATI). Conducted by Bangladesh Centre for Development, Journalism and Communication (BCDJC). Development Alternatives, Inc. (DAI). (2012, June). Bangladesh Open Government Partnership Assessment Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI). Development Consultant and Global Compliance Initiative (DCGCI). (7 December 2010). Community Satisfaction with Quality and Impact of Participation in Budget –Oversight. Submitted to PROGATI. Development Consultant and Global Compliance Initiative (DCGCI). (2011, September). Endline Survey on Citizens’ Knowledge and Perceptions of Corruption. Submitted to Promoting Governance, Accountability, Transparency and Integrity (PROGATI). Key Researchers: Dr. Rafique Uddin Ahmed and Professor Sheikh Shafiul Islam. Frammolino, Ralph. (24 April 2012). Review and Comment on OCAG Press Conference Guidelines. Media Development Consultant. USAID/PROGATI. Hasan, Manzoor. (13 December 2007). Public Sector Corruption in Bangladesh: Political and Bureaucratic. Institute of Governance Studies, BRAC University. Prepared for the Victory Day Commemorative Seminar, Bangladesh Institute of International and Strategic Studies. Johnston, Michael. (2005). Syndromes of Corruption: Wealth, Power, and Democracy. Cambridge and New York: Cambridge University Press. Management Systems International. (24 December 2004). Support for Development of the USAID/Bangladesh Anti-Corruption Strategy, Final Report. Submitted to USAID Office of Democracy, Governance and Education (DG-ED). Dhaka, Bangladesh. Millennium Partners. (6 April 2011). Bangladesh Rule of Law Assessment. Submitted to USAID/Bangladesh. Millennium Partners. (6 April 2011). Promoting Governance, Accountability and Integrity Program (PROGATI) Evaluation Report. Submitted to USAID/Bangladesh. Mozumder, Abul Kashem and Nurul Huda Sakib. (2008, June). Corruption in Bangladesh: Theoretical Analysis and Institutional Effectiveness. Asian Studies, Journal of the Department of Government and Politics, Jahangirnagar University, Savar, Dhaka, No.27 (pp 9l-101). 47 National Baseline Survey 2008: Citizens’ Perception and Knowledge of Corruption Survey Results Presentation and Questionnaire, Promoting Governance, Accountability, Transparency and Integrity (PROGATI), 3 September 2008. Promoting Governance, Accountability and Integrity Program (PROGATI): Addendum Performance Monitoring Plan (PMP). (25 March 2010). Submitted to USAID/Bangladesh. Promoting Governance, Accountability and Integrity Program (PROGATI): Addendum #3 (for Year 5) Final, Performance Monitoring Plan (PMP). (15 December 2011). Submitted to USAID/Bangladesh. Promoting Governance, Accountability and Integrity Program (PROGATI) Performance Monitoring Plan (PMP) (Final Version). (22 September 2008). USAID Contract No. DFD-I-00-05-00220-00. Submitted to USAID/Bangladesh. Rudisuhle, Donovan. (2009, December). Promoting Transparency and Accountability Through Initiatives Conducted in Collaboration with the Two Partner Ministries - Findings and Recommended Activities. Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI). USAID Contract No. DFD-I-00-05-00220-00. USAID Contractor: Development Alternatives, Inc.(DAI). Sakib, Nurul Huda. (2010). Anti-Corruption Commission Under Political Government: An Assessment of Present Regime. Department of Government and Politics, Jahangirnagar University, Savar, Dhaka. Tarana Begum. (2010). Combating Corruption in Bangladesh: Role of the Anti-Corruption Commission. USAID, Bureau for Policy, Planning, and Learning. (19 January 2011). Evaluation Policy. Wehlau, Jeanne. (2009, April). A Six-Month Plan to Strengthen the Oversight Capacity of the Office of the Comptroller and Auditor General in Bangladesh. Bangladesh: Promoting Governance, Accountability, Transparency and Integrity (PROGATI). 48 ANNEX C: DATA COLLECTION INSTRUMENTS AND APPROACH GUIDELINES FOR KIIs KIIs have been carefully selected to elicit critical information for the evaluation. Please use introductory remarks to confirm the interviewee’s level of knowledge, and sources of knowledge, about PROGATI and its program context. Also, ensure that the interviewee is aware of our plan to keep individual responses anonymous unless explicit permission has been gained from the respondent. For purposes of consistency across interviews, the evaluation team agreed to be guided by three principal questions, the first two were more relevant for PROGATI staff, implementers and beneficiaries: 1. How would you assess PROGATI’s performance against its own goals and indicator targets? 2. How would you rate the impact or effectiveness of those indicators or results (using whatever ‘yardstick’ respondents chose, not necessarily those chosen by DAI or USAID)? 3. What advice or suggestions would you give USAID on future governance and integrity-related programming? GUIDELINES FOR FGDs For all three FGDs, a skilled moderator introduced Social Impact, explained its agreement with USAID covering DG-related M&E services, outlined PROGATI’s program, its budget, goals and program components. The moderator reminded participants that USAID was interested in respondents’ opinions more broadly on the topic of what should be done to alleviate corruption in government, business and civil society in Bangladesh and assured them that confidentiality in all comments would be maintained, whether in English or Bangla. Specific questions were then asked: 1. How well do you feel USAID and other donors have diagnosed the causes, mechanisms and extent of corruption in Bangladesh? 2. What are the roles and responsibilities of various parts of Bangladeshi society in fighting corruption? 3. What can/should donors do to help Bangladesh address its problems with corruption, and how should they go about it? Each question also included secondary, follow-up questions (designed to help the moderator elicit more specificity in comments, engage those reluctant to share opinions or to re-balance a discussion). Question 1 follow-up questions: a. What do you see as the significant causes of corruption? b. Are there opportunities to address it that USAID and other donors have missed or have been reluctant to address? c. Have those donors adequately engaged Bangladeshi ‘communities of opposition’ by seeking them out and listening to their advice? Question 2 follow-up questions: 49 a. Many people assert that political leaders and parties are the underlying cause of systemic corruption and only they can stop it. Do you agree with that and if so, why? Are leaders themselves more responsible or are they ‘good people trapped in a bad system?’ b. What do you see as the role of Parliament and Government (and its agencies) for providing transparency, oversight and accountability for public funds? Would you please comment briefly on the budget process, the OCAG, the ACC and RTI? c. How would you encourage good behavior and discourage bad in the private sector’s relationship with the state? Question 3 follow-up questions: a. Some observers believe donors cannot really do much to curb corruption and, at worst, may be part of the problem by spending money without proper safeguards. Do you agree with that statement? b. What would be your priority for partners in this effort--Parliament, agencies, government departments, political parties, civil society, media, or others such as religious leaders? c. How much money should USAID and donors spend on anti-corruption and governance programs in light of other priorities such as health care, education, agriculture and climate change? What would represent too little, too much or the right amount to address corruption? Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 50 ANNEX D: FOCUS GROUP DISCUSSION RESULTS Question 1: How well do you think that USAID and other donors diagnose the causes, mechanisms, and extent of corruption in Bangladesh? Summary Points:  Most said USAID did not have a firm diagnosis of corruption in the country.48  Those most familiar with PROGATI (in Group 2) said the project seemed “unfocused.”  A single consensus existed as to the causes of corruption— politicians and political parties, whose actions dictate the standards by which the civil service and police operate. Other recognized causes include: low salaries and poverty, low accountability within the GOB, dysfunctional Parliament, lack of grassroots awareness, and the deterioration of moral values. Many examples were given from public sector hiring, the courts, education, healthcare and banking. Discussion: The background discussion of the underlying socio-economic, cultural, and political causes of corruption included more examples concerning the exercise of law, lack of compliance with the law, and the negative role played by law enforcement agencies. Two academics mentioned that the judiciary remains insufficiently independent of political influence, especially at lower levels, pointing out the backlog of civil cases that effectively denies justice to ordinary citizens in lower courts that do not enjoy reputations of impartiality and incorruptibility. The discussion also addressed the absence of commitment by political leaders (of both parties), the misuse of political authority and the absence of accountability within party structures themselves, as well as a lack of public awareness about corruption. In terms of institutions, the Anti-Corruption Commission was generally rated favorably, seen as better than its predecessor (the Bureau of Anti-Corruption), but still under political party influence. Participants cautioned that USAID has no mechanisms in place to monitor the performance of public service agencies and therefore must rely on the ACC. In terms of service delivery, it was pointed out that while some civil society groups do monitor government performance; citizens have little recourse when service delivery fails. 48 One participant in Group 1 said donors understand corruption very well, but do not acknowledge it. Otherwise, they would not build safeguards into their own programs. Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 51 Question 2: What are the roles and responsibilities of the various parts of Bangladesh society in fighting corruption? Summary Points:  The GOB needs to take corruption seriously.  The PM should be leading the fight against corruption, rather than denying its existence  MPs are silent on corruption  The ACC does a good job generally but is reactive rather than investigatory.  RTI is a good law but information needs to analyzed for it to make an impact.  The OCAG seems to be doing a good job but it deals with spending after the fact.  Inconsistent views were expressed on media. Some journalists said they themselves were not treating the issue seriously enough, citing time constraints and the complexity of the stories involved. Others said media are doing a good job given their resources and other constraints. All agreed that corporate ownership of the media skewed coverage toward select targets (while there was no mention of the Padma Bridge affair, there was considerable discussion about the Sonali Bank and Hallmark group).  Civil society activism in regions was viewed favorably, but in Chittagong some participants expressed concern that civil society efforts were too concentrated in Dhaka.  One observer expressed sympathy with the private sector as a victim of state corruption, rather than a contributor to it. Discussion: The background discussion again re-visited the consensus that the cause of corruption rests with political leaders and parties, and only they can stop it. Further, corruption is seen to increasingly reduce maneuvering room for good people trapped within it. Monitoring and supervision were mentioned as ways to increase the accountability of political leaders to voters. Participants mentioned that despite the conference of independent authority to the OCAG under Articles 127-132 of the Constitution, its behavior is constrained by rules the Government operates under, which is determined by the Finance Ministry and its capacity is limited. Rules of Business that the Government operates under are subject to the direction and control of the Finance Ministry. Laws require the periodic auditing of public accounts, but those audits are irregularly conducted, and even when submitted to Parliament, they are not subject to public scrutiny and recommendations from them are rarely implemented. There is a law requiring periodic audit of public accounts, but audits are not conducted regularly. Though audit reports are provided to parliament, they are often not open to the public. While Parliament itself has oversight authority over budgets and expenditures, it does not ensure that public funds are used for their intended purposes. Open budget hearings are not conducted and Parliament itself finds itself stalemated in passing legislation and performing other tasks. In terms of procurement, multiple bids and re-bidding is now required for major procurements, invitations are improperly advertised and procedures not adhered to during the process. In local government, Union Parishad councils and City Corporations lack independence, resources and expertise. Media is as involved in corruption as is the ruling party, either overlooking its instances or covering it inappropriately, and frequently engaging in self-censorship out of fear of threats from powerful individuals or the ruling party. Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 52 Question 3: What can and should USAID and other donors do to help Bangladesh address its problems with corruption, and how? Summary Points:  Responses to this question coalesced into a series of discussions between “skeptics” and “optimists.” o Optimists stated USAID should have moved sooner to address corruption. Recommendations centered on institutional strengthening (with many pointing to the ACC) and RTI needing to reach out to regions (one law professor wondered why court documents are not subject to the law). Parliamentary knowledge about budget procedures should be improved while the OCAG should be more focused on the results of public spending and less on the process. o Pessimists stressed the importance of grassroots awareness and the role of civil society and media in promoting improved public awareness.  A vigorous debate emerged regarding whether USAID and donors should be involved in combating corruption at all. Many expressed the opinion that corruption represents an internal Bangladeshi issue with some stating that addressing it by donors violates Bangladesh’s national sovereignty. Some expressed a concern that USAID may have its own agenda in its anti￾corruption activities.  No one wanted a direct confrontation between donors and individual politicians;  A consensus emerged that closer consultation with a “community of opposition” before designing any new initiatives is needed. One participant advised: “Take a year and work with us to design a program in which there is genuine local ownership.”  Donors must lead by example and clean up corruption in their own programs (USAID and DFID was rated highly while UNDP was rated the worst of any donor).  Encourage MPs to be anti-corruption champions;  Participants expressed no real appetite for spending large amounts of money on anti-corruption programs. There was concern that large sums would simply attract attention, infecting the anti-corruption effort itself. Programs should encourage and defend internal communities of opposition. Discussion: Most participants expressed the belief that USAID and other donors are inadequately engaged with Bangladeshi ‘communities of opposition’ by not seeking or listening to their advice. Discussants noted that USAID is spending only limited resources to address corruption. Challenges to that exist in the form of lack of access, and non-cooperation from both state and civil society. One participant stated USAID could not provide any examples of remarkable or significant success, and is not pursuing areas with adequate support or with sufficient partners. Opinions tended to coalesce around the fact that donors cannot really do much to curb corruption and could well exacerbate it by spending money without proper safeguards. Donors do not have much access in Parliament, ministries or agencies, departments, political parties or within communities of religious leaders. While USAID could work as a partner to local organizations, its ability to handle the issue is severely constrained. Partnership (with both government and NGOs) might be focused on changing mindsets, providing skills training, and involve media more frequently to help them frame the issue more effectively. Some initiatives discussed included (a) an emphasis on moral education; (b) family-based education and Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 53 family partnerships; (c) working more frequently with standing committees of Parliament; (d) clarifying the role of media in reporting on corruption issues; (e) rewarding sincere, honest and good people and organizations; and (f) constantly evaluating donor programs to ensure they lead by example. SUMMARY Because the FGDs were both focused and free-flowing, generalizations remain difficult, but three common themes were identified: 1. The need to engage with local communities of opposition more effectively and in a more long-lasting manner; 2. The continuing return to political parties and structures as the primary cause of corruption; 3. The importance of donor willingness to listen and to be flexible (as well as modest) in programming options. Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 54 ANNEX E: LIST OF PERSONS INTERVIEWED FOR KIIS BANGLADESH Anti-Corruption Commission Brigadier General (Ret.) M H Salahuddin, Director General Asian Development Bank Mr. Firoz Ahmed, Public Management Specialist, Bangladesh Resident Mission The Asia Foundation Mr. Russell Pepe, COP, PRODIP Bangladesh BD Union Parishad Forum (BUPF) Mr. Mahbabur Rahman Tulu, President Bangladesh Enterprise Institute (BEI) Mr. Shaeb Enam Khan, Research Director (and former PROG Project Director) Mr. Farooq Sobhan, President Bangladesh NGOs Network for Radio and Communication (BNNRC) Mr. Syed Kamrul Hasan, Coordinator, Knowledge Management for Development (KM4D) Mr. AHM Bazlur Rahman, Chief Executive Officer (CEO) Bangladesh Parliament Secreatariat Mr. Md. Shafiqul Islam, Deputy Secretary Mr. Md. Mahfuzur Rahman, Secretary In-Charge Bangladesh Women Chamber of Commerce & Industry (BWCCI) Ms. Selima Ahmad, Founder President Ms. Farhana Akhtar (with PROGATI project from March-June 2012) Md. Mojibur Rahman (Acting CEO & Head of Program) Syed Sultan (with Progati Project from June 2009 till July 2012) Chittagong University Professor Nizam Ahmed Democracy International Inc. Mr. David Dettman, Chief of Party, Democratic Participation and Reform Project (DPR) Democracy Watch Ms. Taleya Rehman, Founder Executive Director Department for International Development (DFID) Mr. Richard Butterworth, DFID Governance Advisor, Dhaka Development Alternatives, Inc. (DAI) Mr. Jeremy Kanthor, Sr. Development Specialist (Governance) Foreign Aided Projects Audit Directorate (FAPAD) Mr. A. K. M. Jashim Uddin, Director General Grameen Alo Ms. Ferdousi Begum Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 55 International Business Forum of Bangladesh (IBFB) Mr. Mahmudul Islam Chowdury, Ex-President Khan Foundation Mr. Murshid Alam, Campaign Coordinator of PROGATI Project, Sr. Program Officer Mr. Noor Taslima Jahan, Assistant Director Program, Project Director PROGATI Project Ms. Nawshin Khan, Research Associate Management and Resource Development Initiative (MARDI) Mr. Hasibur Rahman Mukur, Executive Director Ministry of Foreign Affairs Ambassador Wali-ur Rahman, Executive Director, Bangladesh Institute of Law and International Affairs (BILIA), Former Secretary, Ministry of Foreign Affairs, Special Envoy of the Prime Minister Sheikh Hasina (1997-1998) Ambassador (Ret.) Muhammad Zamir, Former Secretary, Ministry of Foreign Affairs and Former Chief Information Commissioner, Information Commission News Network Mr. Shahiduzzaman, Editor Tetra Tech ARD Professor Ataur Rahman, Member, ARD DG Assessment and former Seconded to UNDP/Afghanistan Mr. Jerome Sayre, Chief of Party, (SDLG) Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Ms. Beth Cunningham, Deputy Chief of Party Mr. Dennis Gallagher, Chief of Party Mr. Rick Gold, former Senior Technical Advisor Mr. Manash Mitra, Former Parliament Component Manager Mr. Md. Pervez, Former Office Manager Rupantar Mr. Aminul Ehsan, Communications Director USAID/Bangladesh Ms. Sherina Tabassum, Governance Advisor, Office of Democracy and Governance The Hunger Project-Bangladesh Badiul Alam Majumdar, PhD, Vice President and Country Director Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 56 ANNEX F: LIST OF FGDS AND PARTICIPANTS Group Moderator Title Date Location Participants Femal e Male Background 1 Ataur Rahaman Professor, ret., Dhaka University and member of USAID/Bangladesh’s Governance Assessment in 2004 12 Sept 2012 JATRI’s conference facilities; Dhaka 3 8 Familiar with PROGATI 2 Delwar Hussein Professor; Chair of the Department of International Relations, Dhaka University 13 Sept 2012 JATRI’s conference facilities; Dhaka 4 9 Senior community leaders and experts not familiar with PROGATI 3 Rezaul Islam Professor of Social Work, Dhaka University 15 Sept 2012 Chittagong Club; Chittagong 5 15 Regional leaders and experts Participants49 Group 1 1. Dr. Rajal Chatterjee 2. Dr. Owen Lippert 3. Dr. Lily Gomes 4. Nabila Rahman 5. Abu Basher 6. Golam Arshad 7. Md. Abdul Momen 8. AQM Nasir Leddin 9. (Illegible) 10. Zannatul Ferelus 11. Dr. Dalim Brumin Group 2 1. Mirza Hassan 2. M. Jahangik 3. Rafayet Ana 4. Fahima Sultana 5. Shameem Akter Jahan 6. M. A. Halier 7. Md. Rahman 8. Prof. Golan Rahman 9. Siddiqe Rahman 10. Mohammad Sirajul Islam 11. Dr. Mohammad Mamdie Hossain 12. (Illegible) 13. Unknown 49 Please see Annex I for complete attendance list and correct spellings of names. Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 57 Group 3 1. Adv. Qumrul Islam Saggad 2. Mustafa Nayeem 3. Mohammed Shah Nowaz 4. Sandan Mohammad Zobayan 5. Md. Malkutur Rahman Munir 6. Reida Azad 7. Shampak Nahs 8. Dr. Khurshid Jamil 9. Prin .Shamsuggiamanheldy 10. (Illegible) 11. Raisul Hug Bahar 12. Mahfuzul Hoque Shah 13. Dr. Abduleh Al Faruque 14. Mohammad Ali Asgar Chowdhy 15. Nazimuddin Shyamol 16. Anindya Tito 17. Mr. Ahmed 18. Ad. Rehana Begum Raun 19. Syeda Mahfuga Begin 20. Eugene Rodrigues 21. Prof. Jahajir Chowdhy Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 58 ANNEX G: DISCLOSURE OF ANY CONFLICTS OF INTEREST Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 59 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 60 I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 08/15/12 Name Owen Lippert Title Senior Evaluation Specialist Organization Bangladesh Democracy and Governance Program Evaluations (BDGPE), implemented by Social Impact, Inc. Evaluation Position? Team Leader Tea member: I am a team member Evaluation Award Number(contract or other instrument) Contract # AID-OAA-I-10-00003, Task Order # AID-388-TO-12-00001 USAID Project(s) Evaluated(Include project name(s), implementer name(s) and award number(s), if applicable) PROGATI; DAI, Inc. I have real or potential conflicts of interest to disclose. Yes N I have no substantive real or potential conflict of interest. See notes. If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. 1. I have no relatives in any way associated with DAI. 2. I and all my relative, friends, and colleagues have no financial interest, direct or indirect, in DAI. 3. I have not been involved with any evaluation of PROGATI for DAI or any other organization. The only potentially perceived conflict is that in 2009 I was contracted by East-West Management Institute, which was a sub-contractor to DAI on a bid to USAID for a judicial reform RFP in Pakistan, to accompany Rick Gold of DAI on a pre-bid study mission for 10 days in Pakistan. EWMI paid my way. (Mark Dietrich mdietrich@ewmi.org of EWMI would have the details.) USAID ended up canceling the bid. In later conversations with Rick, he said he was having to go to Bangladesh on several occasions re. the PROGATI project. Just what he was doing for or with them, I never inquired. Rick has since left DAI and is working for MSD in D.C. 5. EWMI does do some work in anti-corruption but only as a sub--to the best of my knowledge and though I have done work for them on and off over the last ten years, I have not been engaged by them for over 18 months. During my times in Dhaka, I was aware of the PROGATI project and had met, at USAID partners' meetings, several of the people involved with it. Nonetheless, I only knew of it in outline and have no pre-conceived views on their performance. Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 61 Name Saiful Islam Title Deputy Chief of Party Organization Bangladesh Democracy and Governance Program Evaluations (BDGPE) Evaluation Position? Team Leader Te member Evaluation Award Number(contract or other instrument) BDGPE; PROGATI Performance Evaluation USAID Project(s) Evaluated(Include project name(s), implementer name(s) and award number(s), if applicable) I have real or potential conflicts of interest to disclose. es No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 62 information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 14 August 2012 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 63 ANNEX H: LETTER FROM ALEXIOUS BUTLER TO MAHFUZUR RAHMAN, PARLIAMENT SECRETARIAT Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 64 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 65 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 66 ANNEX I: FGDS’ ATTENDANCE LISTS Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 67 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 68 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 69 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 70 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 71 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 72 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 73 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 74 Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 1 ANNEX J: PROGATI TARGETS AND ACHEIVEMENTS TABLE 3: PROGATI Targets and Achievements Project Years One through Five TARGET ACHIEVEMENT TYPE OF COMPONENT INDICATOR AND GOALS** Yr 1 Yr2 Yr 3 Yr 4 Yr 5 Total Yr 1 Yr 2 Yr 3 Yr 4 Yr 5* Total % of Targets Achieved 1. Strengthening media to serve as an effective public watchdog Number of media CSOs and/or support institutions assisted 0 4 8 8 0 20 0 7 6 8 0 21 105.0% Output Number of government media relations staff trained 0 15 300 224 6 545 0 28 351 258 21 658 120.7% Output Person-days government media relations staff trained 0 0 600 448 36 1,084 0 0 1,629 524 42 2,195 202.5% Output Journalists trained 0 0 570 420 106 1,408 0 0 679 469 124 1,652 117.3% Output Person-days journalists trained 0 0 1,950 1,280 140 3,370 0 0 1,635 1,699 144 3,478 103.2% Output Non-state news outlets assisted 0 0 75 50 30 155 0 99 126 85 39 349 225.2% Output 2. Strengthening civil society to support and promote anti-corruption reforms People affiliated with NGOs receiving anti￾corruption training 56 492 0 544 213 1,305 41 479 0 758 185 1,463 112.1% Output Person-days people affiliated with NGOs receiving anti￾corruption training 0 0 0 2,191 469 2,660 0 0 0 2,297 504 2,801 105.3% Output Anti-corruption measures implemented 0 13 11 9 6 39 0 4 10 11 13 38 97.4% Output Promoting Governance, Accountability, Transparency and Integrity (PROGATI) Performance Evaluation 2 CSO Advocacy campaign supported 6 120 377 310 N/A 813 2 98 378 316 N/A 794 97.7% Output Persons participating in CSO advocacy campaigns 0 0 95,450 83,500 N/A 178,950 0 0 130,206 71,134 N/A 201,340 112.5% Output CSO that engage in advocacy and watchdog functions supported 8 84 120 90 134 436 8 158 353 103 105 727 166.7% Output Positive modifications to enabling legislation/regulation for civil society accomplished 0 NA N/A N/A 0 0 0 NA N/A N/A 0 0 0% Outcome CSOs improved internal organizational capacity 0 0 10 10 N/A 20 0 0 8 2 N/A 10 50.0% Outcome 3. Strengthening public institutions oversight capacity Mechanism for external oversight of public resource use supported 0 6 0 1 3 10 0 1 0 3 1 5 50.0% Output Government Officials receiving anti￾corruption training 38 106 0 200 160 504 37 30 0 445 164 676 134.1% Output Person-days government officials receiving anti￾corruption training 0 0 0 590 390 980 0 0 0 1218 471 1689 172.3% Output 4. Strengthening Parliament’s oversight capacity Members of Parliament and Secretariat staff trained on budget transparency and accountability issues 26 94 50 50 10 230 34 14 29 87 9 173 75.2% Output Members of Parliament using budget analysis services (BAMU and community in practice) 0 50 50 50 21 171 0 0 107 80 187 109.4% Output