October 2014 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Harry Blair, William Cartier, Akhter Hussain, and Naim Mostofa of Social Impact, Inc. Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project Cover Photo: Photograph of Ward Shava planning meeting courtesy of SDLG from RDRS. PERFORMANCE EVALUATION OF THE STRENGTHENING DEMOCRATIC LOCAL GOVERNANCE (SDLG) PROJECT MACRO-LEVEL ADVOCACY AND LOCAL-LEVEL PARTICIPATION DATE: October 2014 USAID Contract AID-OAA-I-10-00003 Task Order AID-388-TO-12-00001 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACKNOWLEDGEMENTS The evaluation team was most fortunate in receiving all needed assistance from the Social Impact staff and the SDLG staff in Dhaka, and the partner NGOs who assisted with field logistics on its visits to Khulna and Rajshahi divisions. In particular, the team would like to thank Saiful Islam, Deputy Chief of Party at the Dhaka Social Impact office, and Md. Pervez, Operations Manager, for their support throughout the team’s work. At the SDLG office, the team wishes in particular to thank Jerome Sayre, Chief of Party, Zarina Rahman Khan, Deputy Chief of Party, and Abu Md. Mohsin, Team Leader for Citizen Participation. Finally, thanks also to Rumana Amin, the Contracting Officer’s Representative at USAID for the Bangladesh Democracy and Governance Programs Evaluation project, and Sherina Tabassum, the Contracting Officer’s Representative for the SDLG project itself. CONTENTS Contents ............................................................................................................................................................. v Acronyms ........................................................................................................................................................... 1 Executive Summary .......................................................................................................................................... 3 1. Introduction ............................................................................................................................................... 8 2. The development Problem and USAID’s Response ......................................................................... 8 4. Evaluation methodOLOGY and Limitations .................................................................................... 12 5. Findings and Conclusions ..................................................................................................................... 15 6. Lessons Learned .................................................................................................................................... 35 Recommendations ......................................................................................................................................... 35 ANNEXES ....................................................................................................................................................... 39 Annex I: EVALUATION SCOPE OF WORK ........................................................................................ 40 Annex II: DOCUMENTS REVIEWED ....................................................................................................... 52 Annex III: EVALUATION INSTRUMENTS .............................................................................................. 55 Annex IV: INTERVIEW LIST ...................................................................................................................... 68 Annex V: Tables and Figures ....................................................................................................................... 74 Figure 1: SDLG Goals and Components .............................................................................................................................74 Table 1 Field Visit Plan ...............................................................................................................................................................74 Figure 2: SDLG Sample LGUs Visited ....................................................................................................................................75 Table 2: SDLG Evaluation Questions and Conclusions in Brief ......................................................................................75 Table 3: SDLG Spending on Support to LGAs March 2011 to December 2013 (in US$) ......................................77 Table 4: Comparison of the Percent of Union Parishad Respondents who Report Being Aware of UP Efforts to Raise Revenue (PMP 4.2) .....................................................................................................................................................77 Table 5: Comparison of the Percent of Paurashava Respondents who Report Being Aware of PS Efforts to Raise Revenue (PMP 4.2) ..........................................................................................................................................................77 Table 6: Comparison of the Percent of Union Parishad Respondents who Agree that the UP Manages their Funds with Transparency and Accountability (PMP 4.2) ..................................................................................................78 Table 7: Comparison of the Percent of Paurashava Respondents who Agree that the PS Manages its Funds with Transparency and Accountability (PMP 4.2)...............................................................................................................78 Table 8: Own Source Revenue Expenditure Patterns in 2012-13 .................................................................................78 Table 9: Comparison of Citizens’ Evaluations of the Quality of Road Construction, Repair, and Maintenance in Union Parishads (PMP 3.5) ...................................................................................................................................................79 Table 10: Comparison of Citizens’ Evaluations of the Quality of Maintaining Market Places for Citizens in Paurashavas (PMP 4.10) .............................................................................................................................................................79 Annex VI: Performance Management Plan for SDLG: Targets and Actual Results, 2011-2013 .. 80 SDLG Performance Indicators – Results of 2011 (Year 1), 2012 (Year 2) and 2013 (Year 3)........................ 80 Annex VII: Footnotes on Survey Based Indicator Values for 2012 (3.4, 3.5, 4.2, and 4.5) ........... 85 Annex VIII. Evauation Design Matrix ....................................................................................................... 88 Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 1 ACRONYMS ADB Asian Development Bank AL Awami League BDGPE Bangladesh Democracy and Governance Programs Evaluation (project) BUF Bangladesh Urban Forum BUPF Bangladesh Union Parishad Forum CiG Citizens in Governance COR Contracting Officer’s Representative CTG Caretaker Government CUS Center for Urban Studies DG Democracy and Governance (sector of USAID) DLGP Democratic Local Governance Program EOP End of project GAF Governance Advocacy Forum GiZ Deutsche Gesellschaft für Internationale Zusammenarbeit GOB Government of Bangladesh ILLG Improving Local Level Governance (program) IQC Indefinite quantity contract JICA Japanese International Cooperation Agency LG Local government LGA Local government association LGD Local Government Department LGED Local Government Engineering Department LGSP Local Government Support Project LGU Local government unit LOE Level of effort LOP Life of project M&E Monitoring and Evaluation MAB Municipal Association of Bangladesh MP Member of Parliament NCSLG National Constituency for Strong Local Government (program) NILG National Institute for Local Government OSR Own source revenue PMP Performance Management Plan PNGO Partner Non-Governmental Organization PPC Policy and Program Coordination Bureau PPL Policy, Planning and Learning Bureau PROGATI Promoting Governance, Accountability, Transparency and Integrity (program) PS Paurashava (municipal government) RCT Randomized Control Trial RDRS (Formerly Rangpur Dinajpur Relief Service) now the acronym is the name SDC Swiss Agency for Development and Cooperation SDLG Strengthening Democratic Local Governance (program) Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 2 SOW Scope of Work TA Technical assistance TO Task Order TOR Terms of reference UGiiP Urban Governance Infrastructure Improvement Project UNDP United Nations Development Programme UP Union Parishad US Unnayan Shamannay USAID United States Agency for International Development UZP Upazila Parishad UzPAB Upazila Parishad Association of Bangladesh V-AID Village Agricultural and Industrial Development Program VGD Vulnerable Group Development program VGF Vulnerable Group Feeding program Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 3 EXECUTIVE SUMMARY This evaluation aims to provide USAID/Bangladesh with an informed assessment of the Mission’s Strengthening Democratic Local Governance (SDLG) project over its first 39 months from December 2010 through February 2014, at a cost of $11.6 million. SDLG covers 450 township-size union parishads (UPs), 50 municipalities or paurashavas (PSs), and 100 county￾level upazila parishads (UZPs) for a total of 600 local government units (LGUs) Of the 600 LGUs, 200 were to have partnered in previous USAID local government projects. SDLG has five program components. Component 1 and supported research, policy dialogue, and advocacy aiming to reform local government laws, along with introduction and demonstration of innovative practices in targeted LGUs. Component 2 aimed to improve the capacity of Bangladesh’s local government associations (LGAs), to improve their financial sustainability, their capacity to provide with training and technical assistance, and to advocate for policy reform. These two components consumed about 20 percent of SDLG’s budget. Component 3 intends to improve LGU transparency and effectiveness in service delivery in their ten compulsory functions, and in additional basic services. Component 4 adopts an existing model of “citizens’ forums” to promote participation in planning, budgeting and oversight. In SDLG these were termed Citizen in Government (CiG) groups, at the ward (neighborhood) level to set planning priorities, and channel citizen service needs to LGUs. Components 3 and 4 were implemented by nine partner non-governmental organizations (PNGOs). These two components absorbed 80 percent of SDLG’s budget. SDLG’s Component 5 focused largely on including women and youth in local governance leadership roles. In implementing the different activities, SDLG was also tasked to maintain close contact with Government of Bangladesh (GOB) institutions, including the Local Government Division (LGD), “to ensure cooperation and buy-in from the GOB”. Reflecting the recommendations of evaluations of UAID’s earlier local government projects, SDLG was also to collaborate closely with other donor programs, particularly the WB’s LGSP. According to the Task Order, SDLG would provide demonstration sites and models: for this reason, it placed emphasis on “creation and dissemination of best practices”. Finally, it was also to work closely with USAID’s own programs in other sectors like health, environment, economic growth and food security. The objectives of the evaluation scope of work (SOW) are to assess SDLG’s actual results against targets; to gauge SDLG’s implementation tools and management in meeting project goals; and to make recommendations for future programs. There are six evaluation questions:  Effectiveness in achieving objectives;  Efficiency as the best approach, as opposed to alternative project approaches;  Management and administration in working with the PNGOs, other donors, LGUs, citizens, and the Government of Bangladesh (GOB);  Sustainability after the end of the project (EOP);  Relevance to current development circumstances in Bangladesh; and  Cross-cutting impact in addressing gender and youth issues. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 4 The evaluation team’s methodology comprised document review; key informant interviews in Dhaka; semi-structured interviews with individual key informants and with LGU and CiG groups during two field visits, and analysis of results of three surveys contracted by SDLG. Several LGU clusters were chosen for the team’s visits. Two constraints emerged: time did not permit visits to control LGU sites, and ongoing UZP elections meant excluding many areas from the sample LGUs. Also, elected officials in many of the LGUs visited had absented themselves to engage in electioneering, which limited the team’s findings concerning SDLG’s UZP activities. FINDINGS FOR COMPONENTS 1 (Roles and Authorities of Local Governments AND 2 (Advocacy and Capacity Building of Local Government Associations) Effectiveness: SDLG focused its policy research effort mostly on Upazila issues, creating a new LGA (Upazila Parishad Association—UzPAB) and placing less emphasis on working with LGAs representing the UPs and paurashavas. Instead, SDLG concentrated on an advocacy endeavor to influence policymakers to implement the Upazila law. These efforts succeeded in getting pledges into the ruling party’s campaign manifesto, which have yet to be acted upon. The cutbacks in SDLG’s budget had a significant impact on the project’s activities in advocacy. Management and Administration: SDLG gave core support to the Municipal Association of Bangladesh (MAB) and the Bangladesh Union Parishad Forum (BUPF), while pressing them to increase their dues-collection and improve their internal organization. SDLG budget cutbacks foreshortened these activities, and caused conflicts with the LGAs. SDLG consulted with other donors, and with relevant GOB offices, but these contacts did not lead to involvement with either donor projects or the GOB. Sustainability of LGAs: SDLG’s emphasis on LGA membership dues bore some results, with BUPF increasing its dues-paying membership from eight to 23 percent and UzPAB from nine to 20 percent, as MAB experienced a slight growth from 41 to 43 percent. But while MAB will likely survive SDLG’s EOP and the termination of its support, BUPF and UzPAB (the Upazila Parishad Association of Bangladesh) expressed pessimism. Relevance: SDLG’s research priorities are relevant to UzPAB’s expressed needs, though less so to BUPF’s and MAB’s. There was no apparent link between the research and advocacy work in Components 1 and 2, and the field activities in Components 3 and 4. Again, progress in this direction was forestalled by project budget cuts, as SDLG preferred to maintain its field presence in Components 3 and 4, to the detriment of activities in support of policy research, advocacy, and collaboration with other donor agencies and with other USAID sector programs. CONCLUSIONS FOR COMPONENTS 1 AND 2 Effectiveness: SDLG’s research and advocacy promotion engage with national policymakers, but this high-level lobbying does not appear to have thus far translated into policy change. The budget cutbacks had a negative impact on relations with LGAs and partner research organizations. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 5 Management and Administration: SDLG managed well the relations with PNGOs and the fieldwork with partner LGUs. However, SDLG did not build partnerships with other donors in research or advocacy efforts, in particular with LGSP—which was a specific TO requirement. Nor did SDLG engage effectively with the GOB, in particular with the LGD senior bureaucracy. Sustainability: Owing to difficulties in reaching agreements with the LGAs, and then a major project budget cut, SDLG was unable to carry out the LGA capacity development activities, for example, to develop their capacity to provide training services for member local governments. And, while SDLG’s efforts to increase LGA dues-paying did produce more revenue, the improvement will probably not be enough to enable BUPF and UzPAB Secretariats to survive. Relevance: To attempt to promote policy reform in all three levels of local governance is relevant but overly ambitious. The research supported by SDLG was at best exploratory, summarizing and reiterating earlier research, without delving deep into LGU management issues. It failed to address effectively the TO requirements for documenting LGUs’ innovative service delivery solutions and disseminating them through studies and policy advocacy events. FINDINGS FOR COMPONENTS 3 Transparent and Effective Service Delivery by Local Governments), 4 (Citizen Participation in Local Decision Making) and 5 (Windows of Opportunity) Effectiveness: SDLG did well in meeting its operational targets for these two components, to judge by the project’s Performance Management Plan (PMP) reports. The survey results are less positive, indicating that in some cases (PS integrity in managing funds and citizen evaluation of public services in UPs and PSs) the control group LGUs showed more improvement than the treatment group. Most interviewees saw the training and technical assistance as useful. Efficiency (Alternatives): The main efficiency issue is the decision to work across three levels of local government and in such a large number of LGUs. SDLG was not able to address some of the Task Order requirements effectively by spreading budget resources so thinly. Management and Administration: The administration of the project field activities was adequate, and overall LGU officials were satisfied with the support. The PNGOs appear to have been effective in their role as training providers and facilitators. Yet, despite this good management of field activities, SDLG was unable to engage other donors or GOB agencies, to “upload” program innovations into other projects or into policy. Sustainability: Despite widespread enthusiasm for the program among the LGUs and CiG members, the team found little evidence that the CiG groups had built the capacity to continue practicing the skills they had acquired. This is not surprising, in the sense that SDLG’s LOP allowed only 12 to 18 months of full-fledged field implementation before it ended in December 2013. Moreover, as observed above, the absence of linkages to other donor programs or GOB agencies reduced the potential dissemination and replication of its program activities. Relevance: SDLG’s interventions were relevant to the LGUs’ major “demand side” governance problems, related to weak accountability and responsiveness. However, SDLG’s Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 6 capacity development, less than 10 days in duration, was unable to address complex service delivery management issues. Cross-Cutting Issues: In Component 5, SDLG ensured that all CiG groups included one￾third women member and sponsored several leadership training programs for women. As for youth, members under 25 years old constituted 12 percent of the overall CiG groups. CONCLUSIONS FOR COMPONENTS 3, 4 and 5 SDLG’s Major Achievement: Combining Revenue Generation with Citizen Participation: Bringing together revenue generation and citizen participation was SDLG’s most noteworthy innovation. Activities to improve OSR were sufficiently attractive to lead LGUs to overcome their reticence in working with the CiG groups, and soon elected local officials came to see the CiGs as a “helping hand” for the local administration, by reaching down to the ward level. Low Sustainability of Project Interventions: SDLG’s startup left little time for field implementation– 18 months at most and generally less than that. This was simply not enough time to begin impacting local political culture, as the opinion surveys showed. Thus, even within the target LGUs, the evaluation team has doubts about sustainability of interventions. Project Design and Budgeting: Even without the cuts in the SDLG budget at the end of year 2, the Task Order was overly ambitious in requiring SDLG work in 600 LGUs. This meant that SDLG was able to address a narrow range of governance processes, and with a relatively low amount of training per LGU. In applying the budget cuts, SDLG opted to maintain this large set of LGUs, which kept the program from offering “demand driven” support to LGUs. Policy Linkages: The fact that SDLG was ineffective in channeling its innovations into policy meant that the major project innovation in UPs and paurashavas (linking the CiGs to elected Councils to promote better governance) had no broader impact beyond the “demonstration” LGUs. Add-on components: When an add-on feature like gender links directly to a mainstream USAID theme, it can be incorporated into new projects straightforwardly. Thus SDLG had no trouble including women’s participation in its programming for Components 3 and 4. RECOMMENDATIONS 1. Concentrate Resources. The evaluation team recommends that USAID rethink the design of subsequent local government projects. USAID should prioritize, focusing on either rural LGUs (either UPs or Upazilas) or urban LGUs (paurashavas). Also, the project could use fewer treatment LGUs. 2. Ensure Continuity of Interventions. For any effort promoting change in political culture to build up sufficient momentum to sustain impact after EOP, the project must last longer Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 7 than the 39 months of SDLG’s lifetime. A five- or six-year effort would be much better, and also ensure that successor local governance projects work in the same set of LGUs. 3. Build In a Service Sector Focus. The evaluation team recommends that the next project focus on decentralizing service delivery in one or more of these sectors. It should have sufficient resources for intensive and sustained capacity development for management improvements and support for citizen participation and accountability mechanisms to pressure LGUs to be more responsive. 4. Incorporate Change Management Interventions. The focus of the local government project should be expanded to include piloting of “supply side” management solutions in specific service sectors. Work in Upazilas could incorporate change management and process reengineering approaches, working directly with the GOB line ministries. 5. Look at IT Solutions. In designing the next RFP, USAID should look at how UPs and Upazilas can improve how they manage between levels, for example through better collection and utilization of information. This could be coordinated with the GOB, perhaps through the PMO’s A21 or the Governance Innovation Unit. The project could look at using governance and management indicators to benchmark LGU performance. 6. Collaborate with GOB in Project Implementation. The discussion of “management solutions” in the previous recommendation suggests, too, that future local government projects need to dedicate more effort to engagement with the GOB, recognizing that government agencies have increasing fiscal, human and technical resources. The sustainability of project interventions will best served through direct involvement with the GOB, in both project design and implementation, to ensure buy-in from relevant ministries and agencies. 7. Engage/Coordinate/Collaborate More with Other Donors. The team suggests that USAID explore how to complement and extend the impacts of other donor programs in the field of local government. This may require much more preparation by USAID prior to issuance of the RFP, to better assess opportunities and reach agreements with potential partner programs on specific modalities and objectives of collaboration. 8. Separate Contract Mechanisms for LGA Support. USAID should leave out LGA support/advocacy from a future project, and look instead for a separate contract mechanism under USAID Forward. These might include direct grants, conditional on pre-grant enquiries and appropriate capacity development prior to approval of funding. However, this is probably not the main priority, in the context of reduced budgets for DG programs. 9. Continue to Use Rigorous Evaluation Approaches. The evaluation team recommends that USAID continue to use RCT approaches to evaluate the impacts of its interventions in strengthening local government. The team recommends that USAID support another application of SDLG’s survey of treatment and control LGUs, in 2015 or 2016, to examine the sustainability and mutation of project interventions. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 8 1. INTRODUCTION The project under review in this final performance evaluation, Strengthening Democratic Local Governance (SDLG), is the latest in a series of USG-sponsored decentralization initiatives stretching back to 1952 and the Village Agricultural and Industrial Development program (V￾AID) launched that year in what was then Pakistan. More recently there have been four local governance support efforts beginning with the Democracy Partnership (1997-2002), National Constituency for Strong Local Government project (NCSLG, 2001-2005), the Improving Local Level Governance Program (ILLG, 2002-2011), and the Democratic Local Governance Program (DLGP, 2005-2008). SDLG has been the largest in terms of total geographic coverage, providing assistance to 600 local government units (LGUs). SDLG has also been the most complex of these local governance projects, including three levels of LGUs and a component focusing on nationwide local government associations (LGAs) and national policy reform. This report presents an evaluation of the 39 months from December 2010 to February 2014 of the SDLG program.1 SDLG’s goals and objectives can be summarized as a development hypothesis holding that strengthening the advocacy capability of LGAs at the national level can influence local government policy reform, which along with improving transparency and citizen participation at the local level will foster more responsive local governance and improved public service delivery. The project was implemented by Tetra Tech ARD of Burlington, Vermont, working with nine partner non-governmental organizations (PNGOs). 2. THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE Throughout their history, local government in colonial Bengal/East Pakistan/Bangladesh have remained weak, underfunded, subject to bureaucratic control from above, unaccountable to their citizens, and manipulated as a patronage enterprise by the political party in power.2 Although the Constitution of Bangladesh calls for elected councils at all three tiers—union, upazila, and district—directly elected councils have been in place on a regular basis only at the union level since the country won independence in 1971. In the 1980s under the military government of General Ershad, upazila parishads (UZPs), or county-like units, which now number almost 500, began operating through directly-elected chairs (elections occurred twice), 1 In February 2014, SDLG was extended at a reduced level of funding until March 2015, but our SOW called for an evaluation of SDLG’s work in the first 39 months only, and accordingly, the evaluation team will not devote attention to the extension. 2 This interpretation has been abundantly documented over several decades, for example in Siddiqui (2000) and Khan (2009). Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 9 but this system was abolished upon the return of democratic governance at the national level in 1991 by a national government and parliament intent on maintaining central power. More recently, the Caretaker Government (CTG) of 2007-2008 promulgated an ordinance restoring Upazila Parishads (UZPs) with directly-elected chairs and, for the first time, two vice￾chairs, one of whom must be a woman. The Awami League (AL) that took power in 2009 modified the ordinance so as to reduce the power of the UZPs, requiring them to take “guidance” from the Member of Parliament (MP). In addition, the government failed to establish UZP authority over the offices of the GOB agencies grouped together in the Upazila. Thus enfeebled, the UZPs have struggled to exercise any real power over Upazila management and service delivery. Elections for chair and vice-chair took place in 2009 accompanied by much enthusiasm, and at the time of this evaluation in 2014, new UZP elections were under way, in a context of intense inter-party competition and widespread violations of electoral procedures. The country’s 487 Upazilas are divided into 4,571 union parishads (according to the 2013 Upazila Manual published by the Local Governance Department—LGD). Every union is governed by a UP, divided into nine wards, each electing a UP member; there also reserved seats for women UP members, representing three wards each, which brings the total number of councilors to twelve, plus one directly-elected chair. The present system has been in place since 1983 with minor changes in 2009. UPs have brought service delivery closer to the citizenry through UP “complexes” that house government workers in health, agricultural extension, and the like. But as in the UZP, these officials belong to their line ministries, and elected UPs have no authority over them. For their budgets, the UPs have been dependent on government grants, which increased somewhat over the last decade with the Local Government Support Program (LGSP), a World Bank initiative. UPs also generate OSR through taxes and fees, but these sources of revenue have been restricted in their scope. As in other countries, there are also urban bodies with their own jurisdictions and service responsibilities: paurashavas (PSs), of which there were 319 total at last count, as well as 11 larger municipalities organized as city corporations. As befits urban areas, the paurashavas provide many more public services (e.g., water, sewage, roads, and streetlights) than the UPs, receive larger government grants, hire more municipal employees, and have more revenue￾generating power. But given the rapid urbanization now occurring in Bangladesh, the PSs are increasingly strapped for funds. They are governed by a directly-elected mayor and a council consisting of elected ward representatives.3 According to the SDLG Task Order, USAID designed the project to address the longstanding weaknesses of LGUs noted at the beginning of this section. SDLG was a relatively large project by USAID-DG standards: it worked in six of the country’s seven divisions; in 22 of its 64 districts; 100 of its 480 Upazilas; 50 of its 309 municipalities; and 450 of its 4,500 unions. Of the 600 partner LGUs, 200 were to have participated in previous USAID local government 3 The number of wards varies according to a pourashava’s population, and thus, the number of reserved seats for women would also vary, though it equates to roughly one-third of the open seats. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 10 projects. At the same time, it provided support to three national LG associations (LGAs): the Bangladesh Union Parishad Forum (BUPF); the Municipal Association of Bangladesh (MAB); and the newly established Upazila Parishad Association of Bangladesh (UzPAB). SDLG’s budget was initially funded at $19.3 million over the life of the project (LOP). However, the recent USG sequestration imposed a reduction of about 40 percent to $11.6 million at the start of Year 3 (January 2013).4 This large budget supported four major program components, along with a fifth component to fund emerging “windows of opportunity.” The components fit together as shown in Figure 1 (see Annex V). A brief description of components 1 through 5 is provided below. A more in depth discussion of each component is included in the Statement of Work contained in Annex I.  Component 1 activities supported policy-relevant research, dissemination, and policy dialogue to increase LG roles and authorities. This included operational research on innovative practices in LGUs, especially in service delivery, and promotion of inter-LGU partnerships. SDLG emphasized research and advocacy to address gaps, ambiguities, and conflicts in the regulatory framework.5  Component 2 activities develop the capacity of the three LGAs as membership bodies, as training providers, and in their policy advocacy. These activities were intended to link to SDLG’s Component 1 policy research and advocacy. Much of this work focuses on building LGA membership and dues collection and internal democracy through elections of the executive bodies.  Component 3 activities aimed to increase the transparency and effectiveness of LGUs through a combination of (1) training and TA for local officials and (2) the potential impact that the CiG groups created through Component 4 would have on elected councils and standing committees.  SDLG Component 4 activities supported increased citizen participation in LG decision￾making through the Citizens in Government (CiG) groups. In this way, there was a direct link with Component 3 activities. In practice, the CiG groups were the linchpin of Components 3 and 4. The CiG groups worked with SDLG to promote ward shavas to set planning priorities6 and UP-wide open budget meetings, to support the LGUs’ work in tax assessment and collection, and to encourage better service delivery through direct participation in UP Standing Committees and direct contact with elected members.7 4 Subsequently, $1.3 million was restored to SDLG’s budget in February 2014 for an extended phase to last until March 2015. The evaluation team did not include this extension in our evaluation, as per note 1 above. 5 In particular, the Local Government Acts affecting UPs, PSs,and UZPs.(GOB 2009). For a summary of the acts, see Panday (2011b).. 6 The cover photograph for this evaluation report shows a ward shava meeting in process ion one of the SDLG partner Union Parishads 7 The CiG groups were formed initially in each ward of each UP and PS included in the SDLG project, when by consensus or a show of hands at a public meeting organized by SDLG’s implementing NGO 10 citizens were chosen. These sets of 10 people each then chose three among them to represent the ward at the LGU level. In Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 11  The Component 5 activities sought to identify opportunities for innovations in transparent and participatory public administration in local government to support the other SDLG program components. This component focused largely on efforts to include women and youth in leadership roles in local governance. Components 3 and 4 cost far more to implement than 1 and 2. While Components 1 and 2 together comprised 18 percent of total programming funds through December 2013, Components 3 and 4 consumed 82 percent. Funding for Component 5 was quite minor and was included in the other four Components. The Task Order aimed to contribute to the Mission’s Development Objective 1, “Citizen Confidence in Governance Institutions Increased”, its IR 1.4, “More responsive elected local government”, and Sub-IRs 1,4.1 and 1.4.2. By expanding the roles of local government, people will be able to locally determine their own priorities and by improving service delivery, they will also see that local governments are more effective—and in this way they will have greater confidence in their democratic institutions. 3. EVALUATION PURPOSE & QUESTIONS 3.1 EVALUATION PURPOSE The purpose of this evaluation is to provide USAID/Bangladesh with an informed assessment of the Mission’s SDLG project. USAID will use the evaluation to draw lessons for the selection, design, and implementation of future projects. The evaluation will help USAID to gain a better understanding of the sustainability of SDLG program outcomes and to plan for sustainable projects. 3.2 EVALUATION QUESTIONS USAID identified three principal objectives for the evaluation, as outlined in the Scope of Work (SOW), which is provided in Annex I: These three objectives are:  Review the SDLG program’s overall performance by assessing actual results against targeted results;  Assess the efficacy of the SDLG implementation tools and management structure in meeting the objectives;  Make recommendations to USAID/Bangladesh concerning future programming in the local governance sector. this way a CiG central group of 27 (three including at least one woman from each of the nine wards) was selected for each UP. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 12 The SOW also sets six themes to evaluate through a series of questions. The areas of interest are: 1. Effectiveness: To what extent has the SDLG program been successful in achieving its planned objectives? Were there any unexpected outcomes? What added value has USAID brought to the local governance sector in Bangladesh through SDLG? 2. Efficiency: Is there evidence from the implementation of SDLG to suggest that alternative program approaches may have been more successful? 3. Management and Administration: How effective and flexible has the SDLG management been in working with implementing partners, other donors (e.g. World Bank LGSP) and beneficiaries, such as local government units, locally elected officials, citizens, and GOB? 4. Sustainability: How sustainable are SDLG program activities, and what measures could have been taken to enhance sustainability? 5. Relevance: To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh, and will they provide sufficient guidance for appropriate programmatic and technical assistance decisions? 6. Cross-cutting Issues: To what extent were gender and youth effectively addressed by SDLG’s interventions in the targeted areas? 4. EVALUATION METHODOLOGY AND LIMITATIONS The data gathering phase of this evaluation took place during February-March 2014. The team consisted of four members: Harry Blair, Visiting Fellow at the Political Science Department at Yale University and the Team Leader for this evaluation; William Cartier, Chief of Party for the BDGPE project and Evaluation Methodologist for the team; Akhter Hussain, Professor of Public Administration at Dhaka University and Senior Local Governance Specialist for the team; and Md. Naim Mostafa, Local Evaluation Specialist for BDGPE and the team. FIVE ELEMENTS OF DATA COLLECTION PLAN The team developed an Evaluation Design Matrix (presented as Annex VIII) around the six evaluation questions, listing the assumptions it was making about each question and the ways it would address each one in its field visits, Dhaka interviews and document review. The elements of the team’s data collection plan are presented below. Document Review: The team reviewed SDLG’s project documents and reports to inform evaluation planning and background research. The project’s performance reports over the LOP were the most useful in this process, but the commissioned studies such as annual work plans, opinion surveys, Performance Management Plan (PMP) reports, and the SDLG task order were also very valuable to this component. The list of documents reviewed is provided in Annex II. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 13 Key Informant Interviews in Dhaka: In addition to relevant DG staff at the USAID Mission in Dhaka and staff at the SDLG office, the evaluation team interviewed GOB officials and staff of other prominent donors in the local governance field. The team was also able to meet several of the researchers involved with SDLG’s commissioned studies and interviewed a number of staff at the three LGA offices in Dhaka. A list of all key informants the team met with is available as Annex IV. Key Informant Interviews during Field Visits: The evaluation team took field trips to the Khulna and Rajshahi Divisions. On each trip the team interviewed elected officials, government servants, and CiG groups. A list of those seen is also part of Annex IV. Semi-Structured Interview Instruments: The evaluation team developed two open-ended questionnaires for its field visits and pilot-tested each during its first two days with field visits to a UP, a PS, and a UZP. The protocols are provided in Annex III.  Individuals – LGU chairs for UPs, PSs, and UZPs; secretaries for UPs and PSs; and Upazila Nirbahi Officers (UNOs, the senior executive officer for an upazila).  Groups – elected council members, standing committees, Citizen-in-Government groups, women members of these bodies. Survey Data: The team also reviewed the three opinion surveys commissioned by SDLG, These three surveys, each employed a Randomized Control Trial (RCT) approach to include about 4,000 respondents in the treatment groups and almost 1,500 in the control groups:  A baseline study conducted in March-April 2011 before Components 3 and 4 began their actual field implementation;  A midline survey in December-January 2012-13; and  A final survey as work with the LGUs was coming to an end in December 2013. Following guidance from the Monitoring and Evaluation (M&E) office at the USAID Mission, the evaluation team only referenced the surveys to comment on the PMP measures employed by the SDLG project. SELECTION OF SAMPLE LGUS FOR FIELD VISITS The team organized the field work in geographical clusters of UPs, PSs, and UZPs, dividing the team in order to cover a larger number of partner LGUs. Because all four team members traveled for field visits, the team was able to split into two two-person sub-teams for data collection, except for the visit to the first UP/PS/UZP cluster where the team tested data collection tools and reinforced a systematic approach across sub-teams. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 14 In choosing the clusters of partner LGUs, the first step was to exclude clusters undergoing upazila elections on just prior to or during field work.8 For the remaining clusters, the team chose at random using SDLG’s Local Government Capacity Index and the Score Cards (A/B/C).9 The selected clusters included two Upazilas with higher-performing paurashavas and two Upazilas with lower-performing paurashavas. In each upazila, the team selected a union parishad (either a high-performing or low-performing UP according to the SDLG data) at random. Figure 2 (see Annex V) summarizes the final site selection. In one of the selected Upazilas (Sahjadpur in Sirajganj District), there was enough time to choose an additional UP.10 Figure 3 also located in Annex V is a map showing the field sites where the evaluation team gathered data. LIMITATIONS Small Sample Size: Due to time constraints, the team was able to visit only four SDLG Upazilas, and these four Upazilas may not have been representative of all SDLG’s sites. To obtain a truly valid sample, the team would have had to include all three levels of LGUs and areas supported by all nine project implementing NGOs. This implies a 27-cell matrix (3 x 9 = 27) to just have a single entry in each cell – an effort far beyond the level of effort (LOE) available. So, based on these limitations, the team designed a sampling scheme that it believed provided a reasonably complete picture of SDLG. In other words, while the evaluation team cannot say that the sample size was statistically representative of the universe of SDLG LGUs, they believe that the sampling design found enough variation to be illustrative of that universe. Selection Bias: As noted in Figure 3 (see Annex V), the team’s field visits were clustered in Rajshahi and Khulna divisions, suggesting some selection bias, given the lack of geographic diversity and the fact that the sites were chosen based on geography and whether or not they had an ongoing election (rather than randomly). This should not impart any serious bias since these two divisions included more than half of the total SDLG-supported district sites (12 of 22 sites), upazila sites (56 of 100 Upazilas), union sites (253 of 450 unions), and municipalities (30 of 50 municipalities). 11 Upazila Parishad Election Campaign Interference: As previously mentioned, the team’s first filter for choosing field visit sites was to eliminate all project Upazilas where campaigns 8 The electoral turnover rate among Bangladeshi LGU chairs and mayors has been quite high in the past. Many chose not to contest, and large numbers were defeated at the polls. It thus seemed reasonable to assume that the same could likely be true for UZP chairs and vice-chairs. 9 SDLG developed an elaborate index to gauge LGU capacity but then when project funds shrank found it necessary to switch to a simpler score card system. 10 It should also be noted that one of our selected pourashavas (Jibannagar) is crossed out and another (Chuadanga) is substituted. This was because we found that the mayor, a Jama’at member, was in hiding at the time of our visit, so we selected another PS in the same district. 11 It is virtually impossible to completely eliminate selection bias in any kind of sample, but the team believes it has done its best to do so in this evaluation, given the context of cost and time constraints. For a couple of good surveys of the problems involved, see Vella (1998), also Collier and Mahoney (1996). Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 15 would be in progress for the five-phase elections taking place. As it turned out, this “filter” did not fully filter election interference, as elected officials in non-campaigning areas were in many cases deeply involved in nominating candidates, campaigning for (or against) candidates in other constituencies, and lobbying with party officials visiting from Dhaka. Thus, the team was unable to meet with several UZP chairs, PS mayors, UP chairs, LGU council members, and even UNOs. In some cases it was possible to meet with vice-chairs or acting mayors, which was helpful, but not nearly as useful as meeting with the actual officials in charge of the LGUs. 5. FINDINGS AND CONCLUSIONS The findings and conclusions are presented in two sections: one for the national-level Components 1 and 2 and the other for its local-level Components 3 and 4. Each section follows the SOW’s evaluation questions, presenting the findings in response to all the questions and then the conclusions stemming from those findings. Recommendations drawn from findings and conclusions are presented separately at the end of the report. To help manage the array of questions and answers, Figure 4 in Annex V clearly states the essence of each question and the team’s answers for Components 1 and 2 and Components 3 and 4. In perusing this section of the report, reviewing the PMP indicators located in Annex VI will provide useful context for the reader. At the end of the section, the report addresses Component 5, which focuses on gender and youth inclusion in the program. 5.1 FINDINGS FOR COMPONENTS 1 & 2 SDLG’s Component 1 objective is to support advocacy initiatives for reforms to expand roles, authorities, and resources for local governments. The objective of Component 2 is to increase the capacity of local government associations to advocate and lobby on behalf of their member LGs for policy changes to empower local governments and advance democratic decentralization. Thus, SDLG provided support to the two existing LGAs (BUPF for UPs and MAB for paurashavas) and assisted in the creation of UzPAB representing UZPs. 5.1. Effectiveness: Achieving Results Effectiveness in Integrating Components 1 and 2: The team’s evaluation of SDLG effectiveness in implementing Components 1 and 2 begins with the SDLG Task Order requirements, which state, “BUPF and MAB should take the lead in advocating for reform,” and “new opportunities should also be explored to foster legal and policy reforms separate from the MAB and BUPF activities.” The first finding is that SDLG was not effective in integrating these components so that MAB and BUPF were engaged with the policy research agenda and able to independently take up the findings and use them for their own advocacy activities. Rather, both SDLG staff and MAB and BUPF officials told the Team that the project mainly focused on the “new opportunities” and gave priority to upazila issues. SDLG explained that this happened in part because of budget cuts, which required them to prioritize actions. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 16 SDLG partnered with two research organizations: Unnayan Shamannay and the Center for Urban Studies (CUS). Researchers interviewed from these organizations stated that the LGAs did not have a role in defining the research agenda. They said that both the research products and policy dialogue activities were presented as an initiative of SDLG’s/USAID’s. This was confirmed by officials from all three LGAs, who explained that they did not have input into the research agenda or the TORs. They were invited to the policy activities but did not have an important role in the events.12 BUPF has an eight-point agenda which highlights the tendency of the GOB to curtail UPs’ existing authorities through “unlawful circulars.” BUPF officials interviewed said that most of their issues were not incorporated into the SDLG policy advocacy activity. However, SDLG notes that three of the BUPF demands were incorporated into the project’s advocacy work. The management of SDLG also recognized that cuts to the project budget in 2013 created conflicts with the partner LGAs, BUPF in particular, which undoubtedly contributed to BUPF’s negative perceptions. MAB, for its part, has a 16-point reform agenda. A MAB officer stated that SDLG support did not advance this policy agenda further. The evaluation team’s review of the program research and activity reports confirms that SDLG did not take up these policy agendas. The SDLG management said that the difficult relationship with MAB caused difficulties in agreeing on a common policy advocacy strategy. Another key informant from one of SDLG’s partner research organizations was critical of the SDLG research effort, saying, “SDLG ignored the work already done by others,” in this case referring to MAB’s own efforts with the Local Government Commission under the CTG, and subsequent advocacy initiatives under the AL government post-2008 in the framework of the Bangladesh Urban Forum. Effectiveness of Linking Research to Policy: Under the rubric of research, the SDLG Task Order required a range of activities including research on LG reform; investigation into decentralized service delivery and in particular, health and education; support for innovative practices; and partnership building among targeted LGs. A review of the research products shows that SDLG’s research and advocacy activities were focused on clarifying “gaps,” conflicts, or obstacles to full implementation of the three LG Acts passed by the CTG and later amended and ratified by the AL government in 2009-2010.13 This focus on implementation of legislation was confirmed in the evaluation team’s interviews with SDLG staff. 12 A review of the various SDLG outputs (policy research reports, advocacy materials and policy dialogue report) supports the views expressed by the researchers and LGA representatives. In the national policy events themselves, the proceedings were introduced by SDLG; the moderators were from the research organizations, and the research findings were presented by the researchers. The LGAs had the role of commentators or panelists. The regional policy events, however, did give a higher profile to LG representatives, who were given opportunities to make presentations on LG issues, but LGA representatives were not included at that level either. 13 The three pieces of legislation are: the Local Government (Union Parishad) Act 2009; Upazila Parishad Act 2009; and the Pourashava Act 2009. See GOB (2009) and Panday (2011b). Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 17 Despite the TO requirement that SDLG document LGU practices “that will demonstrate their improved ability to provide services to their constituents”, the research products showed no in-depth research into service delivery. In SDLG reports and interviews with SDLG staff, Unnayan Shamannay, CUS, MAB, BUPF, and UzPAB, the evaluation team found no operational research to demonstrate or document innovation in service delivery and to take the findings into policy.14 One MAB official said, “CUS did some initial fact-finding in paurashavas, [but] after a long time passed there were [no] results, summaries, or products that would be useful for us.” According to BPF and MAB, the research done by SDLG and Unnayan Shamannay on UPs also reiterated the findings of other research, with the important exception of an audit and accountability study.15 SDLG notes that the budget cuts in 2013 limited its progress in this area. MAB representatives stated that SDLG was “too focused” on the Upazila issues and “didn’t give time to the municipal policies” or to support initiatives directly with the LGD on urban issues. One partner research organization observed that SDLG was not integrated into ongoing policy initiatives on urban governance such as the Bangladesh Urban Forum or the ABD funded UGiiP project with paurashavas. The same organization also noted that municipal governance and management is a technical policy field and requires in-depth, focused research and engagement with the central government. This organization further observed that SDLG in spreading its resources across three levels of LG, “could only scratch at the surface” of the issues. UzPAB representatives consider that SDLG and Unnayan Shamannay were effective in getting their priority issues to decision-makers, using both public forums and closed door meetings with ministers and MPs. Reviewing the SDLG reports, the evaluation team concurs with this view. The founders of this consulting group are senior opinion-makers and gave SDLG access to policymakers at the national level. The Unnayan Shamannay researchers claim that SDLG was able to have the Awami League party incorporate elements of one of the SDLG policy briefs on decentralization into its January 2014 parliamentary election manifesto, which states, “More authority and responsibility will be delegated to the Zila Parishad, upazila parishad, and union parishad through democratic reorganization of the present centralized administrative structure.”16 However, at time of writing, according to key informants interviewed, this statement had not yet translated into new policy reforms on local government. GOB representatives and donor agencies questioned the SDLG approach to advocacy on upazila issues. Two senior local government officers interviewed stated that the upazila legislation had created “conflict in the bureaucracy” that would take decades to work out with each Ministry having to rethink its processes. Further, one officer stated, “Trying to force it through by transferring the revenue fund won’t help matters.” Representatives of two donor 14 Components 3 and 4 did develop a highly innovative approach to improving service delivery with its combination of revenue generation and citizen participation, as noted later in this report. 15 Sobhan, et al. (2013). 16 It was not possible for the evaluation team to establish attribution for the AL decision on the election manifesto. While it is true that SDLG did engage with senior GOB and AL officials in 2012 and 2013, so did many other donor programs. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 18 agencies echoed this view, noting that fully implementing the existing upazila legislation would need “close engagement” with LGD and the line ministries. Unexpected Outcomes: The most salient unexpected outcome of the research and advocacy was the conflictive relationship between SDLG and the two existing LGAs during the first and second years of the project. This conflict made it difficult for SDLG to reach an agreement with BUPF and MAB on the grants, which also impeded effective integration of the LGAs into SDLG’s research and advocacy activities. The situation as aggravated by the SDG budget cuts, which were felt more strongly in Components 1 and 2 activities. Value Added: In Components 1 and 2, the UzPAB executive leader and the Unnayan Shamannay researchers told the evaluation team that SDLG had brought Upazila issues to the fore at a time when the GOB was not interested in public dialogue on how to advance the decentralization agenda. The research products and advocacy outputs were unique and timely—only one other donor organization, UNDP, was conducting research on questions related to Upazilas. And, only SDLG was carrying out advocacy on these issues (UNDP worked directly with LGD). While these efforts have not yet resulted in policy reform, they do appear to have provided some value that could later result in policy reform. With respect to LGA strengthening, despite the difficulties with LGA leadership, SDLG did add considerable value, in a variety of ways. It pressed upon the association leadership the need for sustainability through better communications and outreach to members, and through internal democracy. It also produced some innovative concepts for association strengthening, for example, the proposal for a communications strategy via cel phones (as all LGU leaders use cel phones intensively).17 However, owing to budget cutbacks, in 2013 SDLG was not able to advance further in this direction, 5.1.2. Efficiency: Alternative Approaches LGAs as Trainers: BUPF officials were critical of SDLG management, saying that they did not understand why SDLG “continued support for NGOs” but cut off support to the LGAs. The officials pointed out that early in the SDLG program they asked to be considered as a service provider for the training to UPs under Component 3 of the program. BUPF had recently received funding from the USAID-funded PROGATI program to train 25 “master trainers,” and they proposed to use these same trainers in SDLG. BUPF officials said that SDLG declined, arguing that BUPF did not have the necessary technical and management capacity. SDLG managers reiterated this view about BUPF in the interviews with the evaluation team. Coalitional Strategies: Several of the ongoing policy initiatives on decentralization in Bangladesh involve coalitions. For example, the Governance Advocacy Forum (GAF) led by the Wave Foundation is a coalition of 30 NGOs which addresses decentralization issues and which 17 See SDLG. 2013. “Digital & Email Strategy for LGA Promotion. Strengthening Democratic Local Governance (SDLG) Project”. Dhaka: Unitrend Ltd. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 19 existed at the time SDLG was starting program activities. In the urban sphere, another policy initiative is the Bangladesh Urban Forum (BUF), initiated in 2011, with GOB ministries and departments, donor agencies, national and international NGOs, universities, and MAB. In reviewing SDLG advocacy activities, there is no evidence of involvement with these initiatives. 5.1.3. Management and Administration SDLG Partners: Early on in the SDLG program, SDLG management and USAID made it known to the LGAs that continued support would require LGA leadership’s commitment to internal democracy through elections; increased membership; and increased revenues through membership fees. The MAB and BUPF executive board members describe this policy decision as “abrupt,” but they said that they agreed to the conditions and began to take the measures required by SDLG. For their part, SDLG management said that the two partner LGAs were slow to respond to the funding conditions, in particular, MAB, which took more than a year to organize its internal elections. SDLG management also emphasized that “this strategic emphasis [on internal democracy and sustainability] ran counter to the centralized nature of the existing associations and…raised the prospect of an end to their heavy reliance on donor funds”.18 SDLG management notes that despite the delays, SDLG continued to provide core support to the LGAs to cover their secretariat costs and also paid for internal elections, regional and national conferences, and advocacy events involving the LGAs. SDLG also provided training to BUPF and MAB on board leadership, membership building, and dues collection. Some of the training with MAB was suspended because of lack of commitment. SDLG management also said that the delays in meeting the requirements and the difficult relations with BUPF and MAB leadership delayed considerably the negotiation and approval of grants to these LGAs. For their part, MAB and BUPF officials are critical of both SDLG and USAID handling of the relations with the LGAs. They appreciated the core support provided by SDLG for the operation of the secretariats. The problem was the support for program activities, especially advocacy and the establishment of training programs. One official stated, “We understood the new conditions, but we didn’t understand why the grant negotiations were delayed so long—it is as if they didn’t trust us.” He pointed to the fact that after almost two years of constant meetings on the terms of grant support, SDLG announced to them that it was removing the program activities from the proposed grants. SDLG project staff explained that the decision was made in response to the funding restrictions caused by the broader USG sequestration issue. Other donors: The SOW for this evaluation asks with respect to project management, “How effective and flexible has SDLG management been in working with implementing partners, other donors and beneficiaries…and GOB?” The TO stated, “Donor coordination is especially important as USAID will through this SDLG project work with a relatively small subset of local governments,” (page 7). It also said that collaboration with the major WB funded LGSP project 18 This clarification comes from SDLG. 2014. “Comments of the Performance Evaluation of the SDLG Project from the SDLG Team”. Dhaka: SDLG. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 20 “is essential.” A Secretary from the LDRG&C Ministry and the management team of LGSP reported that they had not met with SDLG to coordinate activities; rather, according to the Secretary, SDLG limited its interaction to just “sharing” its own plans for work with the UPs with LGD, without proposing any joint activities. SDLG management told the evaluation team that they had not attempted to coordinate their activities with LGSP; they portrayed the World Bank project in negative terms. With respect to other donors, SDLG managers pointed to the early initiative to create a “friends of SDLG” group, but this initiative did not bear fruit. Subsequently, SDLG participated in the local consultative group on LG, in which it reported on its activities. The bilateral agencies interviewed said that there were no concrete proposals from SDLG for collaboration. An ADB official said they were open to cooperation with smaller donor programs, “But this would take long-term planning and coordination—it couldn’t be done at the spur of the moment.” He said “It would need to be agreed with the project management teams,” referring to GiZ’s work on governance practices within the larger ADB UGIIP project and UNDP’s applied research work within the framework of the LGSP as examples of effective cooperation. The sole example of successful SDLG collaboration with other donor agencies was the efforts to link the SDLG partner LGUs to the World Bank-funded Horizontal Learning project. The HL project accepted five partner Upazilas and their respective UPs into the program, with purpose of disseminating their innovations with CiGs. This happened almost three years into the project, however, and will require several more months before it comes to fruition. Government of Bangladesh: Policy research by Dr. P. K. Panday contracted by SDLG in 2011 argued for engagement with the GOB to resolve issues in implementation of the Upazila Law, recommending that “Advocacy is necessary with the LGD for taking an initiative to sit with the line ministries for issuing a circular directing their field level officials to send all files to the UZC for approval .”19 This conclusion of course repeats the injunction in the Task Order for SDLG to engage effectively with national policy-makers. However, SDLG’s advocacy work was more focused on legislators and party leaders than the bureaucracy. The evaluation team asked the LGD secretaries whether SDLG had provided opportunities for working directly with the LGD or other departments —both replied they had not interacted with SDLG. When asked whether there had been follow up meetings with SDLG one Secretary replied, “Not to my knowledge. We don’t know what the USAID project is doing with the Upazilas.” 5.1.4. Sustainability of LGAs Project Activities: With respect to Component 2, as Table 1 (see Annex V) shows, SDLG made significant contributions to the three LGAs, covering a large proportion of their core costs (secretariat) and funding project costs with the LGAs (internal elections, policy events, meetings, etc.), as well providing small grants for LGA activities. In addition, SDLG spent an estimated $199,567 for staff working with the LGAs. 19 See Panday (2011a: 90). Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 21 As a result, and despite the difficult relations between SDLG and the LGAs and the complaints from the LGAs about the change in the rules of the game, the team found that SDLG did contribute to LGA internal democracy; the higher rate of LGU membership affiliation and increased financial sustainability. Over the last year of the SDLG project, the rates of LGU dues-paying membership in the three LGAs increased from 41 percent to 42.9 percent in MAB; from eight percent to 22.7 percent in the case of BUPF; and from nine percent to 20.3 percent in UzPAB, according to data assembled by SDLG. The UzPAB and BUPF presidents recognized that the improvements in membership affiliation and dues collection were due to SDLG assistance through training for association capacity development. From interviews with the three LGAs, the evaluation team heard that MAB had taken actions to ensure its continued survival by obtaining free office space in an LGED building and raising its membership dues. MAB was also finishing implementation of a grant from DANIDA and was looking for additional opportunities for small grants. MAB officials expressed confidence that it would be able to make the transition from USAID core support, albeit with “some difficulties.” In contrast, both BUPF and UzPAB state that they anticipate difficulties in maintaining their secretariat functions, without which it will be impossible to retain their membership. The UzPAB president, too, was pessimistic about the future of the association, saying, “SDLG made some progress in identifying the main problems, but the support should continue for another four or five years. We will try with our own resources but it will be very difficult.” The evaluation team also asked SDLG about sustainability with respect to training provision by the LGAs and the GOB. The SDLG management team said that neither the LGAs, nor the GOB had the capacity to provide quality training for a large number of local governments in the relatively short time period of the project. Thus, SDLG’s decision was to design its own training program and to train partner NGOs to implement it. 5.1.5. Relevance SDLG was effective with UzPAB in linking research and advocacy to this LGA’s express priorities. US’s research focused on the implementation of two important provisions of the Upazila Act: the effective transfer to Upazila Council control over 17 GOB departments, including the spending of development and revenues funds; and the collection of land taxes directly by the upazila. The choice of these issues was supported by the UzPAB executive body. The other LGAs did not question the relevance of SDLG’s policy advocacy, but instead they wanted a closer fit with their own policy agenda and support for their own advocacy activities. The evaluation team’s review of the SDLG research products compared to other research on LG issues shows that in union parishad and municipal governance, the research and advocacy activities tended to focus on a set of general issues that have been amply discussed in past years. The SDLG project managers and the Unnayan Shamannay and CUS researchers all consider these issues still relevant. The donor agencies agree that decentralization and local government capacity development continue to be critical priorities. However, several donor officials suggested that working on upazila issues would require a sectoral approach. For their Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 22 part, LGD officials suggested that to be more relevant (and effective), USAID’s capacity development programs have to engage more in the technical, financial, and administrative details of a broader range of LG governance and management topics — the “nitty gritty,” as one LGD Secretary termed it. From the perspective of LGD, the SDLG’s thematic focus on revenue enhancement and participatory planning was too narrow, especially in paurashavas and Upazilas, which compared to UPs face a wider range of challenges in governance and management. 5.2 CONCLUSIONS FOR COMPONENTS 1 & 2 5.2.1 Effectiveness of Integration of Components 1 and 2 In the case of BUPF and MAB, SDLG did not involve the associations sufficiently in the definition of research priorities or in the use of the research in function of LGAs’ advocacy activities. In the case of the Upazilas, SDLG and Unnayan Shamannay did incorporate some of UzPAB’s priorities, especially the need to eliminate MP interference and to give Upazilas management control over their funds. 1. Effectiveness of Linking Research to Policy SDLG mainly focused on implementation of the three LG laws. The interviews with the LGAs and the review of SDLG documents suggest that SDLG gave priority to Upazila issues in its research agenda and advocacy. While senior GOB bureaucrats, donor agencies, and even SDLG's own researchers say that it would be productive to engage directly with LGD and other GOB ministries, SDLG did not take this route, involving LGD only tangentially at best. While effective decentralization requires clarity on “roles and responsibility,” it also depends on finding a way to manage between levels of government. For this reason, other donors are working directly with LGD on the implementation of the UP, paurashava and Upazila laws. SDLG's research and advocacy ignored this aspect, including the possibilities for working with the GOB within ongoing, multi-donor initiatives, which would have provided ample room to focus on improving specific aspects of LGU management, including service delivery innovations. 2. Management of Research and Advocacy SDLG did not carry out effective research and advocacy activities in two of the tasks required by the TO under Component 1 Task C – Innovative Practices. SDLG failed to document and disseminate innovations for policy advocacy and reform; and Task D (Partnership Building), where the activities supported by SDLG were at time of writing still incipient. While SDLG argues that this is because of budget cuts in 2013, the evaluation team saw no progress towards these two TO requirements prior to the budget cuts coming into force. By opting for a high-level lobbying strategy with Unnayan Shamannay, SDLG was able to get its foot in the door with senior officials. In the absence of an immediate decision by government, and also now absent continued funding for Unnayan Shamannay, however, the momentum appears to have been lost. Nor did SDLG build partnerships with donor organizations in its policy research and advocacy activities, which might have carried forward the SDLG initiatives. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 23 Neither SDLG nor Unnayan Shamannay engaged other donors in the design of any of the policy research and advocacy strategies, nor did they coordinate with other, large-scale projects funded by multi-lateral institutions, in particular LGSP, as required by the TO. Referring to union parishads, SDLG's interventions ignored the challenge and opportunity posed by the impending conclusion of the WB’s LGSPII. The evaluation team considers that this was a critically important, missed opportunity for SDLG to engage with LGD, the LGSPII management team, other donors, and BUPF on making block transfers to UPs a permanent policy of the GOB. 4. Sustainability SDLG's approach to training, and its reluctance to use the BUPF or other LGAs for the delivery of training, was based on effectiveness, not sustainability considerations. The trade-off was evident—SDLG was effective in using the partner NGOs to deliver training, but this training provision is not sustainable. These NGOs may well return to LG issues but only if another project provides the necessary funds. At a time when several other important donors in Bangladesh are withdrawing from local governance programs, this may not happen soon. While MAB will continue to survive, BUPF and UzPAB face an uncertain future; BUPF did not prepare for the day when USAID support would end, and UzPAB is still an incipient organization. Again, all of the LGAs expressed surprise and dismay that USAID would not want to ensure the sustainability of its investments in developing their organizational capacities. On the question of the sustainability of advocacy, SDLG’s main partner, Unnayan Shamannay, conducts research and advocacy according to the client’s wishes. It has no mission or mandate to continue efforts in this direction. CUS will continue to work in the area of urban governance and management. The SDLG initiatives did not connect with any other advocacy initiatives led by Bangladeshi organizations, such as the GAF or the BUF. Progress in this sense was cut short by the SDLG budget reduction, which gave preference to Components 3 and 4 activities. 5. Relevance After reviewing the SDLG project documents, research products, and advocacy activity reports, the evaluation team concludes that the SDLG TO design was overly ambitious in attempting to support high quality, operationally relevant, and durable policy change interventions across all three levels of local government. From the interviews with LGAs, LGD officials and donor representatives, the team concludes that while the issues addressed by SDLG are relevant in principle, in remaining at a general level, without engagement with the partner LGAs and GOB agencies, they do not generate specific proposals that could be taken up by policy actors. The work by SDLG and its partner Unnayan Shamannay on Upazila issues was the most relevant of the advocacy activities and continues to be so. Most of the key informant interviews with donors and government officials highlighted the question of how to decentralize service delivery at the Upazila level as one of the most critical aspects of government effectiveness in Bangladesh today. However, from the interviews with GOB and donor agencies, the evaluation Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 24 team concludes once again that SDLG lacked engagement with government agencies (LGD and other ministries) on this topic. By supporting the immediate transfer of funds to upazila control, without working through the details of service delivery in each sector, LGD’s senior managers saw SDLG’s activities as reflecting UzPAB’ s interests but not necessarily favoring a rational, planned process of policy change. For the senior bureaucrats in the Bangladesh civil service, such a wholesale transfer of power – in their words a “big bang” approach – is anathema. FINDINGS FOR COMPONENTS 3 & 4 1. Effectiveness 1a. Achieving Objectives: The objective of Component 4 (“citizen participation in local decision making”) was “to ensure transparency and accountability in public decision making [that would] lead to improved service delivery,” while that of Component 3 (“transparent and effective service delivery by local governments”) was “to enable [local governments] to be effective and transparent service providers at local level.” Accordingly, the team interprets Component 4 to be aimed at producing the mechanisms that will (with additional support from SDLG) help bring about improvement in Component 3. More participatory government will support better service delivery. For Component 4, the PMP data in Annex VI show SDLG establishing CiG groups in all 500 LGUs (UPs + PSs) and participatory planning/budgeting implemented in 490 of them (PMP 4.1).20 Moreover, SDLG met the LOP target of supporting five local mechanisms supported for citizens to engage local governments (PMP 4.4). For the two opinion questions (PMP 4.2), findings are somewhat mixed, though both show gains. Tables 2 and 3 (Annex V) show modest increases in citizen awareness of LGU revenue raising efforts for UPs and PSs respectively, while Tables 4 and 5 show a larger growth in citizen trust in LGU integrity. In the latter PS case, however, the “yes” answers were also improving, even more in the control municipalities (from 16 percent to 58 percent answering “yes”) than in the treatment towns (17 percent to 57 percent), implying that the “treatment” was irrelevant, or at least that other factors operating in both groups overshadowed the SDLG interventions in the treatment group.21 For Component 3, SDLG or exceeded all three PMP coverage targets (numbers of LGUs receiving USAID assistance in increasing capacity, number receiving assistance with revenue generation, and numbers of individuals receiving USAID training – PMPs 3.1, 3.2, 3.3). Revenue generation did go up, by 217 percent in the treatment unions as against 122 percent in the control units, while the treatment municipalities registered a 117 percent increase vs. a decline of 19 percent in the control paurashavas. 20 The LOP target for participatory planning/budgeting (PMP 4.3) is shown as 600, which would include all 100 UZPs, but SDLG wasn’t working along these lines in the UZPs. Therefore, the team assumes there must have been a typographical error here. 21 Interestingly, the endline survey showed in a logistical regression that whether or not a respondent was in the treatment or control group had virtually no effect in predicting whether he/she had confidence in local elected officials (Sabet and Haque, 2014: 51). Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 25 The increased OSR generated by SDLG provided considerable discretionary income to be allocated by the LGUs. Data on how that money was spent in SDLG’s PSs and UPs is presented in Table 6. The largest single item is “Development Projects,” which represent service delivery, and likely some of the “Office, et al.” category would include service delivery as well. But the “Salaries” line indicates that a good portion of the additional resources went to increasing the pay of locally elected officials, particularly in the UPs, where aside from the secretary they are the only officials on the payroll.22 Satisfaction with public services proved a somewhat different story, as becomes clear in PMP indicators 3.5a and 3.5b (citizen satisfaction with service delivery in UPs and PSs respectively), shown in Tables 7 and 8 (Annex V). UP respondents, asked about quality of roads, answered “good” or “excellent” 53 percent of the time in the baseline survey, a number that climbed up to only 57 percent in the final survey, far below the targeted 30 percent over baseline (the control group registered a slightly larger increase, from 48 percent to 53 percent). In the paurashavas, citizens were asked about marketplace quality. The baseline-to-final “good” or “excellent” answers improved from 49 percent to 55 percent, this time well below the increase among control group respondents, which went from 58 percent to 80 percent. The greater increase within the control LGUs implies that public services improved more outside of the SDLG project area than within it.23 In its field visits, the team found that CiG groups were formed in every UP and PS (PMP 4.1). Respondents from LGU councils and CiG groups as well as chairs/mayors and LGU secretaries reported uniformly that the CiG groups were instrumental in mobilizing attendance at the ward shavas and open budget meetings (PMP 4.3), and further that the CiG members themselves played an active role in the discussion at those assemblies. Finally, all respondents said that the PNGOs had provided training and TA relating to the ward shavas, open budget meetings, standing committees, and LGUs themselves (PMP 4.4). All the LGUs visited reported receiving SDLG assistance in capacity building (PMP 3.1) and in particular, receiving assistance in increasing their own source revenue (PMP 3.2) – indeed, this was the most important aspect of the whole project in the eyes of the vast majority of respondents, as will be discussed in more detail below. And, all those the team talked with received at least some training from the PNGO project officers (PMP 3.3). Looking at SDLG’s work with the Upazila level, upon reviewing the program documents, reports and PMP, and in the interviews with SDLG management and PNGOs, it was found that SDLG was unable to roll-out its training for the targeted Upazilas until Year 3; indeed, the 22 In addition to paying elected officials, the PSs employ a number of workers to provide services, so the “Salaries” line may well have included them also. 23 The small size of the control group in the pourashavas (n=44 in Table 8) may well be a factor here. It should be noted that in the baseline survey, 70 percent of respondents answered the question in both treatment and control groups, but in the endline survey, it was only 28 percent and 37 percent, respectively. Other factors also may have been at work here. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 26 design and piloting of the training materials for what SDLG termed “the most contentious and complex” of the three levels of local government was not completed until the end of Year 2. SDLG was not able to look at the impact of its work on citizen satisfaction with Upazila services at all—which is unfortunate, as the Upazila is the most important service delivery organization in the GOB. The SDLG Upazila surveys were limited to self-reporting by elected officials. In some processes, like budgeting, the survey shows is a tendency among treatment Upazilas to consult more widely. In other processes, like the measures to increase OSR the result is not clear; indeed the control Upazilas showed better performance than treatment LGUs. The last SDLG quarterly reports for 2013 also recognize that it will take some time to see major differences in Upazila governance and management as a result of SDLG interventions. 1b. Unexpected Outcomes: The evaluation team did not find any outcomes unexpected by SDLG interventions or the PNGOs. 1c. Added Value for Local Governance Sector: In developing its combination of helping LGUs increase local revenue and creating participatory input through the CiG mechanism, SDLG did provide added value to the LGUs themselves and to donor understanding of local governance. Its main governance innovation was the CiG understood as a “helping hand” for the local Council. It allowed them to increase their local OSR by strengthening the “outreach” of the elected Council members to their ward constituencies and better communicating the need for a local tax base.. The CiG groups also acted as “eyes and ears” for the Council, identifying and attending to critical issues like child marriage, domestic violence, trafficking and access to GOB safety net programs. Without taking anything away from this innovation, it still falls far short of the Task Order requirement to provide “comprehensive experimental programs to strengthen local government planning and management capacity… in quality and quantity of the compulsory duty services, as well as additional services, as appropriate, such as disaster mitigation and environmental conservation”. Nor were there any innovations generated with respect to the use by LGUs of “rigorous and practical indicators (and accompanying baseline measurements)… to measure performance changes, such as surveys, tracking local revenue generation, etc.”, which are also mentioned in the TO. The large number of partner LGUs covered by SDLG meant that the interventions were quite limited in terms of the LOE dedicated to capacity development in each partner LGU. With less than 10 days of formal training for each LGU, the project was severely constrained in both the depth of the capacity development, and in the number of functions/processes that it could cover in any LGU. Despite the TO requirements, it was unable to provide “demand driven” capacity development, for example to support, document and replicate innovative practices of individual LGUs outside of the three functional areas covered its standard package of training. This low level of capacity development support might explain some of the inconclusive results of the SDLG surveys. One question asked if there had been ward meetings; 41% of people in treatment LGUs said yes, as against 29% in control LGUs. However, when the paurashava residents were asked whether services had improved, there was no discernable difference Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 27 between treatment/control LGUs in basic urban services such as water, drainage, or solid waste management. While one interpretation might be that the perception of improvement will take time, another, more plausible explanation is that SDLG did not provide any “supply side” capacity development to the partner paurashavas improve those services. Indeed, this same point was mentioned by two of the paurashava Chairs interviewed by the evaluation team. 2. Efficiency With respect to efficiency in the project’s overall approach, in designing SDLG, USAID might have decided to focus project resources on either 1) a smaller set of local governments; 2) a single tier of local governments; or 3) LGUs within a single Division or a set of adjacent Districts. If the logic of SDLG was to create a “demonstration” of local government innovations of sufficient interest and impact to be taken up by the national government and other LGUs, this objective was made more difficult by the project approach. The dispersion of activities across a large number of LGUs led to a very “shallow” set of interventions in each of the participating LGUs, particularly in PS and Upazilas. It is not clear to the evaluation team why a group of 600 LGUs was a better “demonstration” than 300, 200 or even 100 LGUs. If there had been an effective link to the GOB for taking up innovations, a small number of LGUs would have sufficed. By focusing resources on a smaller sub-set of LGUs, within a single tier and/or a set of LGUs within a smaller geographical area, SDLG might have been able to free up resources for deeper interventions in improving governance and management, especially in the more complex LGU administrative structures of the paurashavas and Upazilas. 3. Management and Administration Implementing Partners: As specified in the TO, SDLG solicited and recruited nine domestic NGOs to implement Components 3 and 4, and SDLG assigned each organization a specific area to work in. The evaluation team interviewed staff from four of the nine PNGOs (one in each of the four districts visited). All four had considerable previous experience with field-based projects in their respective regions.The team met with the project coordinators (who managed overall SDLG implementation for their PNGOs) and the project officers (who dealt directly with the LGUs and CiGs) and learned about the training imparted by SDLG staff in a training-of-trainers (ToT) process that enabled the project officers to then train council members and CiG members, as well as to provide follow-on advice and guidance to them. The four PNGOs the team visited all had considerable experience implementing donor-assisted projects in different sectors. They appeared to adapt to SDLG with no difficulty. The project officers were impressive, committed to their SDLG work and to the LGU and CiG members they worked with. The best judges of their performance were the LGU chairs and councilors, and the CiG members, who uniformly expressed high satisfaction with the quality of their Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 28 training and the follow-up visits.24 The only negative comment the evaluation team heard was the frequently expressed complaint that SDLG’s activities had ended too soon and should continue longer to achieve results. One, related point about the PNGOs was their role in sustaining the CiGs. The interviews with CiG members and LGU officials coincided in highlighting the continuing role of the PNGO staff in convening and organizing the activities supported by SDLG, even in the third year of the project. In all but one of the UPs, the informants reported that the CiG meetings in the wards were organized in this way: PNGO staff contact the CiG leaders by telephone, agree on meeting logistics, and then call the members. The PNGOs often provided refreshments. Other Donors: It was not possible to ascertain if links were in place between other donor programs and SDLG at the field level among the sites the team visited. With respect to SDLG linkages with donor projects, the evaluation team met with the World Bank and the LGSP project management. While the TO requires SDLG to “strengthen and complement” LGSP, this did not happen in practice; there was no substantive collaboration. According to LGSP management and LGD officials, the linkages were limited to occasional “sharing meetings”. SDLG did not take up any of the activities suggested in the TO such as working with UPs receiving the extended block grant under the LGSP project “to pilot new implementation techniques”. For their part, the UNDP and ADB also reported that they had no collaboration with SDLG in their respective projects with UPs, Upazilas and paurashavas. Other USAID Sector Programs: The Task Order was explicit in the requirement for SDLG to explore opportunities with other USAID programs in health, environment, agriculture, disaster prevention or and food security. However, for the period covered by this evaluation, the evaluation team found that SDLG had made no progress in this respect. Locally Elected Officials: With only one exception among all the UP chairs, PS mayors, and elected council members at both levels, locally elected officials uniformly found SDLG’s training and CiG groups both helpful, as recounted elsewhere in this report. Thus, SDLG was perceived as being effective. Unfortunately, given the Upazila elections, it was not possible to reach a sufficient number of Chairs or Vice-Chairs to analyze their views on SDLG effectiveness. PNGO project officers reported experiencing considerable initial difficulty in presenting and selling the SDLG project to elected LGU councils. The selling point proved to be SDLG’s revenue generating potential. The project officers found their audience eagerly receptive to both the training they offered about local revenue generation and the idea of a CiG group that would assist in increasing the revenue to be collected. In effect, the councils were willing to accept the CiG concept and the risk of rivalry in order to increase local revenues. However, the councils soon found the CiG groups to be immensely in motivating participation in planning/budgeting; in identifying eligible recipients for social safety net programs; in mobilizing 24 Follow-up visits have often been a shoRCToming of field-based projects, but that was not the case here. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 29 people to participate in the ward shava and open budget meetings; and in acting as information gatherers about neighborhood needs such as clogged drains, broken streetlights, etc. Citizens: Apart from CiG group members, the evaluation team did not interview citizens directly on field visits. But the three opinion surveys sponsored by SDLG paint a picture of how citizens received and perceived the program. Details are reported above in subsection 1a. Government of Bangladesh: The Task Order states that SDLG will “build upon and deepen the existing relationship with the Local Government Division”. The specific requirements for Components 3 and 4 set out as expected results, “increased information available on participatory planning and community engagement to key actors including GOB officials at the local and national levels” as well as its “adoption and use…in local governments outside the purview of the SDLG program”. And “lessons learned … should be channeled into the national policy reform dialogue”. From interviews with SDLG management and GOB officials, and through a review of program reports, the team finds that SDLG achieved little or no impact in this respect. The senior officials in LGD were not aware of SDLG activities and reported having met only occasionally with project management to share information about activities. Overall, the evaluation team found none of the engagement with the GOB required by the TO. No GOB agency was involved in the planning of SDLG; the GOB was not consulted on selection of LGUs; there was no substantive collaboration with the GOB’s flagship program for UPs (LGSP); the GOB training institute, NILG, had no role in the design, testing or application of training materials; there was no collaboration with the UNDP/GOB Upazila Governance Program; and SDLG did not collaborate with the ADB/GOB UGIIP focused on paurashavas. 4. Sustainability Project Activities: The team’s timing was good for looking into sustainability of the interventions in local government units as the formal SDLG project had ended in December 2013, two months before this evaluation. This timing allowed the team to check on post￾project interest in continuing the project’s work. In all of the locations visited, the team found uniform enthusiasm for continuing the CiG groups’ activities. The chairs/mayors, council members, and secretaries appreciated the increased revenue and the neighborhood information generated by the CiG groups. CiG members themselves enjoyed the enhanced self-esteem and local status the project gave them, as well as their role in selecting visible local improvements funded by the LGUs.25 Aside from the enthusiasm, however, the team did not find much evidence that the CiG groups had developed a capacity to carry on in making and keeping schedules, crafting agendas, organizing meetings, and providing logistical support. In their defense, it must be said that the CiGs actually had little time to build these skills. Though it formally launched in December 2010, SDLG had to recruit staff, locate offices, solicit and vet PNGO proposals, get the PNGOs approved by the GOB’s NGO Affairs Bureau, and 25 The Insight study funded by SDLG (Unitrend, 2012) documented a quite similar finding among the CiG members. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 30 train the project officer trainers. The project officers then had to train the LGU members and set up and train the CiG groups. By the time the project officers completed this necessary work, it was summer and early fall 2012. The result was that only 12 to 18 months remained after all the training had been completed for field implementation with the CiGs in place working with the LGUs before project end in December 2013. Thus, the annual participatory planning/budget cycle and the OSR effort could only run for one or in some cases one-and-a-half rounds, and the neighborhood information generation was similarly restricted. This was enough time to build up significant momentum, as is clearly reflected in the keenness and even passion so widely expressed for the project, but likely not enough to enable it to continue on its own. In fact, as we discussed above, the evaluation team found in the field interviews with CiG members and LGU officials that given the short period for institutionalization of the new processes created with the CiG groups, the PNGOs still had a crucial role in organizing and sustaining CiG activities, convening and organizing many CiG activities. Measures to Enhance Sustainability: As we have discussed above, the evaluation team found that SDLG did not develop an ongoing relationship with GOB institutions like the LGD or NILG that would have remained in place after the project ended, to continue to provide LGUs with quality capacity development training and technical assistance. SDLG management has stated in the comments on the first draft of this evaluation report, that “SDLG’s primary sustainability objective was sustainability of results in the field”, with the idea that the councils and communities would continue the practices. As was discussed above, it remains to be seen if this has happened in practice; it would require another evaluation survey to assess SDLG’s results in this regard. SDLG’s explanation of “sustainability of field results” is not entirely consistent with the Task Order’s depiction of SDLG as a “demonstration project”, which clearly required the project to carry out specific activities to ensure a connection with GOB policy, and complementarities with other large donor programs that would sustain the SDLG innovations and extend them to other LGUs throughout the country. The evaluation team has found that these activities were not implemented, and as a result, this sustainability strategy through national-level linkages is not to be found in SDLG’s program interventions in Components 3 and 4. Related to this same point is the absence of linkages between the innovations generated in the program Components 3 and 4 and the policy advocacy activities carried out under Components 1 and 2. In the absence of GOB buy-in to the SDLG project, it did not have a ready audience for its governance and management results—as one senior LGD Secretary observed about SDLG, “it’s an NGO project”. 5. Relevance For Components 3 and 4, there is no question that supporting local governance in general and SDLG’s particular approach to it will have continuing relevance in Bangladesh. As explained above in the discussion of “value-added”, SDLG’s interventions produced an important innovation in governance, through the CiG groups, which as a result led to a clear improvement Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 31 in LGU performance (in this case, increased tax revenues of UPs and paurashavas). Other SDLG interventions like participatory budgeting, or supporting service monitoring committees, while not innovative in the Bangladeshi context, are relevant for improving local democratic governance. With respect to Upazilas specifically, the evaluation team questions looked at whether SDLG’s interventions were relevant to the current situation of Upazila Councils in the context of the recent Upazila law. An analysis of the key sections of the law and a review of recent studies on local government in Bangladesh, including SDLG’s own policy documents, shows that the law sets up a conflict between the elected Council and the multitude of line ministries and agencies represented in the Upazila administration. Unlike decentralization policy in other countries, the Upazila law did not provide for internal reorganization or “reengineering” of processes within the line ministries and agencies. The Upazila continues to be just a physical infrastructure housing numerous de-concentrated GOB agencies, with a loose supervisory function carried out by the UNO. SDLG chose to train the Upazila Councils on three topics: service monitoring; planning; and financial management, with an emphasis on own source revenue, which are all relevant in principle. With respect to Upazila governance and management, SDLG might have considered how to make better use of the large amounts of information collected in health, education, social welfare, reproductive health, and agriculture, for Upazila oversight, benchmarking of services, etc. This would have been a very relevant intervention for the current Upazila legal framework, giving the elected Councils a tool to press for greater political accountability of the Upazila administration. Admittedly, it would have required greater buy-in from the GOB and focusing budget resources in fewer LGUs to support in-depth capacity development interventions. One notable absence in SDLG’s work with LGUs was improvement of information systems for decision-making, which was highlighted in the TO. In this sense, the evaluation team considers that SDLG lost an opportunity to respond to growing interest within the GOB generally in finding IT solutions to improve management. While it was not necessary to hinge the entire project on information systems, for example, SDLG might have explored opportunities with the PMO’s Access to Information (A2I) program, which is developing web portals for Upazilas. This would have been in line with the TO requirement to “help identify complementary areas of training and longer-term reforms or innovations that would bring further improvements”. FINDINGS FOR COMPONENT 5 Component 5, officially titled “Windows of Opportunity” but often referred to as “Cross￾Cutting Issues” in SDLG’s reports, focused on women and youth. SDLG generally treated this as an add-on to the four main project components, as the term “windows of opportunity” implies and as is evident in the evaluation team’s SOW, where Component 5 is not listed as such along with the other components. Cross-cutting issues received little attention in the PMP Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 32 reporting.26 Even so, significant attention was devoted to women at both national and local levels. For instance, women UZP vice chairs received specific training as did MAB and BUPF national women’s committees. At the local level, all CiG groups were required to include women as one-third of their executive committee members, a goal that SDLG met. The project also trained women LG council members and in particular standing committee members in the UPs and PSs. There was no target for youth inclusion, but SDLG emphasized recruiting younger members, such that almost 12 percent of the committees consisted of citizens less than 25 years old.27 In addition, SDLG sponsored the creation of several videos and radio programs featuring women and youth. The interview team interviewed 150 people in the site visits: 120 men and 130 women. The team was able to meet with women LGU and CiG members separately from their male counterparts at each LGU site visited, and a number of the women interviewed mentioned (in all cases with significant enthusiasm) the training they had received. Generally, they were more articulate than the male Council and CiG group members in explaining the objectives and contents of training. As for youth, while the team did not specifically ask to meet with younger CiG members, in most cases one or two members under 30 (which seemed the more common denominator of “youth” where the team visited) showed up to talk and were as enthusiastic about the project as their elders. CONCLUSIONS FOR COMPONENTS 3 AND 4 1. Citizen Voice for Improving LGU Governance SDLG’s Major Achievement: The decision to focus on strengthening citizen voice through the CiGs, and to use the CiGs to support the LGUs in their OSR strategies was SDLG’s most outstanding achievement and a noteworthy innovation in the field of local governance in Bangladesh. As noted above, initially the revenue generation incentive was sufficiently attractive to lead LGUs to accept what they perceived initially as a threat of increased citizen involvement in the council’s business, and within quite a short timeframe, elected local officials came to see that the CiGs were useful in extending the reach of Councils to their constituents in the wards. During the project implementation period SDLG’s participating LGUs (UPs and PS) experienced higher growth of OSR than the control LGUs, which can be attributed to the campaigns led by Councilors and CiG members, and to SDLG training of officials on strategies for enhancing OSR. As noted above, after the project’s preparatory phases were put into place, only 18 months remained in LOP for actual field implementation, i.e., just one or one-and-a-half budget cycles. In spite of this, it was sufficient to raise OSR dramatically, as is clear from the data for PMP 3.4 (Annex VI), which show a much larger increase for the treatment sample than for its control counterpart. 26 Only one PMP Performance Indicator refers to women (1.4 Number of advocacy actions conducted by women elected representatives in local government), and none to youth. 27 These figures were reported by the PNGOs. The proportion of youth among CiG groups varied between seven percent and 17 percent among the nine PNGOs. The team had no means of verifying these numbers. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 33 2. SDLG’s Use of Randomized Control Trials On aspect of SDLG which should be highlighted is its use of Randomized Control Trails (RCT) approaches in the design of its PMP, which included a baseline measurement and two-follow – up surveys over the life of the project. Much of what we can say about the effectiveness of the project is owed to the data collected through surveys over the life of the project. In this sense, SDLG is to be commended for its rigorous use of RCT methods in designing and implementing its PMP. Unfortunately, the team only had access to the data from the final survey during the report writing stage and was unable to look at the data in depth. However, it is certainly a rich source of information for USAID and should be utilized in subsequent project design. On the down side, the short project length meant that SDLG could not fully exploit this resource. 3. Consequences of a Short Project Life While there were discernable impacts in own source revenue, SDLG did not show a clear “governance payoff”, expressed in greater trust in the treatment LGUs, or greater satisfaction with services. Indeed, citizen perception of improvement in actual service delivery shows little difference between treatment and control for the UPs (Table 7 in Annex V, also PMP 3.5a in Annex VI). The data also show a greater rise for the control than the treatment sample for the PSs (Table 8 in Annex V, also PMP 3.5b in Annex VI). The political culture question in the surveys, on citizen voice in ward shava meetings (PMP 4.5) produced muddled results; there was no indication that the treatment sample had made more progress than the control. These disappointing results on the SDLG governance indicators do not necessarily indicate project failure, but rather that the timeframe was too short to create and nurture a “demand side” CiG mechanism that could promote and sustain serious change in local political culture. It was in effect a misguided approach to the RCT concept to think that such a change could come so quickly and be expected to show up in opinion surveys conducted at short intervals. 4. Low Level of Capacity Development Support Another explanation for the ambiguous survey results with respect to improvements in governance between the SDLG treatment and control LGUs might also be explained by the low levels of capacity development. SDLG’s interventions consisted of three short training courses on service monitoring, planning/budgeting, and financial management/OSR), with follow up facilitation by PNGO staff to motivate CiG members and Council members to implement the proposed policy/process changes. However, there was little or no attention on the “supply side”, i.e., the development of capacity within the local government administration itself. While this is understandable in the case of the UPs, with their scanty administrative structure, it is less so with respect to paurashavas and Upazilas, each with a complex administration. 5. Sustainability of SDLG Field Programs While the CiG involvement in the LGUs had gotten off to an excellent start and the evaluation team observed enthusiasm for carrying on the project from both LGU officials and CiG members, the team did not see evidence that CiG activities had become institutionalized to the Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 34 extent necessary to continue without outside TA and logistical support. The LGU officials and CiG members reported that the NGOs still had a major role in convening and organizing CiG meetings and activities. The cause was SDLG’s short LOP, which had not had time to build momentum within the CiGs for them to continue on their own. Without a follow-up survey after the end of the project, it is not possible to determine the degree of sustainability of the SDLG interventions. As was mentioned above, the short project time frame made it impossible to examine the long-term impacts of these interventions, especially with respect to changes in political attitudes and behavior. Thus, in such a short project period, the Randomized Control and Treatment (RCT) approach to impact evaluation used by SDLG to generate data for its PMP indicators was not able to achieve its full potential. The evaluation team agrees with SDLG’s comment on the first draft of this report, in the sense that “sustainability of field results can be determined objectively through a survey”. 6. PNGOs as Implementers Related to sustainability, is the question of SDLG’s use of NGOs as implementing partners in Components 3 and 4. The four PNGOs interviewed appeared to be effective implementers for SDLG, a judgment based on what the evaluation team learned from PNGO project officers and, more importantly, from the LGUs and CiG groups that were the recipients of training and follow-on guidance, and the achievements registered by treatment LGUs in terms of revenue generation. However, the decision to use PNGOs had negative implications for sustainability of the SDLG program interventions. This is in no way a criticism of the partner NGOs, who appeared to have performed well for SDLG. Rather, in practical terms, the expertise and capacity of NGOs developed under SDLG is lost to LGUs once the project is ended. In the absence of a private market for technical services to LGUs, in countries like Bangladesh governments play this role, through training institutes and other programs, sometimes led by line ministries (LGD) or training institutes (NILG). Granted, they have capacity issues; for this reason, USAID Forward objectives include developing sustainable institutional capacity and strengthening national systems. 7. Exiguous Engagement with GOB The SDLG Task Order was explicit in spelling out USAID’s intention that the activities under this “demonstration project” would be taken up into GOB policy and widely replicated among a larger group of LGUs, saying “the Contractor shall maintain close contact with…GOB institutions, including the Local Government Division (LGD) throughout the life of this program to ensure cooperation and buy in from the GOB”. The evaluation team concludes that this did not happen. In the responses to the first draft of this evaluation report, SDLG noted that “in the third year and onwards” it started to interact with LGD, however, in practical terms, as it comes at the end of the project, it is difficult to assess its potential contribution to SDLG’s overall objectives. As a result, there has been no uptake into policy or wider replication within non-project LGUs of SDLG’s successes in effectively engaging communities in local governance and developing Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 35 capacities of community members and Councils to make local government more responsive. The evaluation team considers this to be a major, missed opportunity for USAID/Bangladesh. 6. LESSONS LEARNED In this section, the team has tried to distill from its many findings and conclusions what it believes are the essential lessons to be learned from SDLG’s experience.  Supporting research and advocacy for implementation of new LG laws is a worthy objective, but it requires deep and sustained engagement with GOB ministries and departments.  Short-term local governance programs may deliver impressive performance in LGU management processes in a brief project, but the sustainability of broader governance practices might require longer projects, or several projects working in the same group of LGUs over time.  The use of RCT approaches to measure impacts in local government projects is promising, but they need to be done in the context of longer project periods, or successive project interventions in the same LGUs.  In managing budget cuts to ongoing LG projects IPs should take care not to eliminate entirely those activities that will channel “demonstrations” and “models” into policy. Otherwise, the project’s broader development objectives could well be lost, despite successes in the field.  Working in LGUs with more complex administrative structures like paurashavas and Upazilas may require more attention to “supply side” governance improvements involving management and staff.  Local government projects that intend to have broader impacts should incorporate strategies for early and deep engagement with government institutions, involving them in project decisions. Otherwise the program interventions could well stay at the “demonstration” stage. RECOMMENDATIONS RECOMMENDATIONS FOR FUTURE PROGRAMMING 1. Concentrate Resources The evaluation team recommends that USAID rethink the design of subsequent local government projects. To be able to influence policy change and reform, the team advises not spreading resources across all levels of local government. Rather, USAID should prioritize, focusing on either rural LGUs (UPs or Upazilas) or urban LGUs (paurashavas), according to its overall project orientation. Also, the project should consider carefully the number of LGUs required for a “demonstration”. A workable RCT approach could use as fewer treatment LGUs. Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 36 2. Ensure Continuity of Interventions For any effort promoting change in political culture to build up sufficient inertial momentum to sustain impact after EOP, the project must last longer than the 39 months of SDLG’s lifetime. A five- or six-year effort would be much better, but even that would in all likelihood represent just a beginning. Such an approach would take some re-jiggering of the 4- or 5-year LOP system, but it could be made to happen, perhaps with successor projects in which the RFP would require working in the same LGUs as the previous project. The point is to ensure continuity in the same LGUs; little is gained over time if each new project implementer moves on to a new set of partner LGs. In other words, a subsequent RFP should require the project to work in a sub-set of the existing SDLG 600 partner LGUs, whether UPs, paurashavas or Upazilas, chosen for their success in SDLG and/or through a field assessment to identify innovative leaders among the 600 SDLG LGUs. 3. Build in a Service Sector Focus Given USAID’s priorities in health, food security, WASH, and agriculture, the evaluation team recommends that the next local governance project focus on decentralizing service delivery in one or more of these sectors. In line with the other recommendations presented here, it should focus on a smaller set of LGUs, perhaps with a narrower geographical focus, to ensure that sufficient resources are available for intensive and sustained capacity development for management improvements, and to promote more “contagion” with adjacent LGUs. It should combine support for citizen participation and accountability mechanisms to strengthen both the “supply and the “demand” sides of local governance. In emphasizing responsiveness in service delivery, it should focus on LGUs with significant service functions (paurashavas and Upazilas) rather than UPs. 4. Incorporate Change Management Interventions The governance focus of the local government project should be expanded to include piloting of “supply side” management solutions. Work in Upazilas could incorporate change manage￾ment and process reengineering approaches, working with line ministries to experiment with governance improvements. Returning to the previous recommendation, it should be noted that this is intensive in training, facilitation, mentoring, etc., well beyond the 10 days of training provided by SDLG. Experience from decentralization programs in other countries suggests that it could require a full time trainer/facilitator in each Upazila or paurashava to lead the change management process, supported by experts in process reengineering, and in close collaboration with GOB counterparts. 5. Look at IT Solutions Related to the previous point, in designing the RFP, USAID should look carefully at how UPs and Upazilas can improve management/governance between levels, particularly through better use of information on plans, budgets, and service provision levels and standards. What is notable about the SDSLG project in this aspect is its lack of attention to IT solutions, which Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 37 have a huge potential for bringing local governments into a performance management approach, and also incorporating citizen access and oversight. This might involve piloting innovative IT solutions (e.g. a simple information management systems that can piggy back on the existing Union Information Service Centers), or better access to existing data sources (i.e., service data collected by Upazila offices of ministries and agencies) and its use for decision-making and accountability. 6. Engage with GOB in Project Implementation The history of local government reform—indeed, any policy reform—in Bangladesh suggests that without taking the bureaucracy on board it is virtually impossible to implement policy change. The discussion of “management solutions” in a previous recommendation suggests, too, that future local government projects need to dedicate more effort to engagement with the GOB, recognizing that government agencies have increasing fiscal, human and technical resources—indeed, LGD’s own budget has grown by almost 10% annually in real terms since 2009. The sustainability of project interventions will best served through direct involvement with the GOB, in both project design and implementation, to ensure buy-in from relevant ministries and agencies. If a future local government projects, it is recommendable to work with LGD and NILG in the design of the project, and in the implementation of capacity development and training. Evidently, given the legitimate concerns about NILG’s technical capacity, the project would have to include a capacity development component for NILG itself, perhaps drawing on existing NGO expertise and staff for assistance in design, in training of trainers and in monitoring training quality. If the project includes Upazilas, it should also engage line ministries and agencies responsible for service delivery, to lead, design and implement the required change management strategies for Upazila governance, perhaps even the PMO’s Governance Innovations Unit. This would require prior engagement and negotiation between USAID, the PMO and the respective ministries, before issuance of an RFP. The Task Order should set out in detail the project counterparts in line ministries, LGD, etc., the specific service/processes that will be reengineered, and the modalities for GOB participation, which might include colocation in ministries and/or Upazila complexes. Local government projects are particularly conducive to the use of benchmarking governance and management improvements, comparing LGU performance over time. In engaging LGU and GOB actors, subsequent projects could incorporate governance and management indicators, developed collaboratively with the LGU leaders and in the case of Upazilas, with the GOB (LGD and line ministries). 7. Engage/Coordinate/Collaborate More with Other Donors The Task Order requires the Contractor to coordinate closely with other donors, and specifically LGSP. Our findings show that this did not happen. The team suggests that USAID Final Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project 38 could profitably explore how to complement and extend the impacts of other donor programs in the field of local government, such as:  For union parishads, the World Bank’s LGSP and the Swiss Development Corporation.  For paurashavas, the Asian Development Bank’s UGiiP, JICA, GiZ.  For upazila parishads, the UNDP and A2I in the PMO. Whatever the focus of any future project, effective engagement with other donors (and with the GOB) would require more upfront work by USAID prior to issuing the RFP/RFA. The team recommends that USAID undertake a detailed assessment to identify opportunities for cooperation with other projects, and negotiate with the respective donors and the GOB to specify the exact modalities and contents of cooperation with USAID’s selected implementing partner. These details should then be incorporated into the RFP/RFA and the respective Task Order. 8. Separate Contract Mechanisms for LGA Support USAID might consider leaving out LGA support/advocacy from a future project, and look instead for a separate contract mechanism under USAID Forward. These might include direct grants, conditional on pre-grant enquiries and appropriate capacity development prior to approval of funding. Such a grant should maintain the established performance criteria of internal democracy through elections of the national and regional executives, and a minimum level of membership and fees collection to sustain the association. Such a grant might be implemented in cooperation with another international organization (Commonwealth Local Government Forum), or an association of local governments from a neighboring country (the Philippines or India). 9. Continue to Use Rigorous Evaluation Approaches The evaluation team recommends that USAID continue to use RCT approaches to evaluate the impacts of its interventions in strengthening local government, with a long-term focus on how these interventions change not just management processes, but also governance variables, including people’s knowledge of local politics, attitudes towards local governments, perceptions of service delivery, and patterns of political engagement and participation. In particular, the evaluation team recommends that USAID support another application of SDLG’s survey of treatment and control LGUs, in 2015 or 2016, to examine the sustainability and mutation of project interventions. The results of such a study should be invaluable to USAID/Washington and Missions worldwide in providing a rare picture of what contributes to sustainability in its local governance work. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 39 ANNEXES Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 40 ANNEX I: EVALUATION SCOPE OF WORK Scope of Work for the Strengthening Democratic Local Governance (SDLG) External Performance Evaluation USAID/Bangladesh Office of Democracy and Governance Program Identification Data Program Title : Strengthening Democratic Local Governance (SDLG) Program Number: EPP-I-00-04-00035-00, Task Order No: AID-388-TO-11-00001 Program Dates : Start Date: 12/21/2010- End Date: 3/30/2014 Program Funding: $ 19,248,165 Implementing Organization: Tetra Tec ARD Contracting Officer Representative (COR): Sherina Tabassum I. Background While the constitution of Bangladesh calls for a strong locally-elected government at each administrative level, what primarily exists is a weak form of sub-national government, structured in a way that allows it to play a minimal role in development. According to national law, each administrative unit of local governance (Union Parishad, Upazilla Parishad, municipalities, etc.) was to consist of elected representatives in charge of public offices and services. In addition, these local bodies were to have taxation authorities. The Union Parishad (UP) level is the one level that has consistently had elected representatives. And, it was only in January 2009, based on the legal framework established by the Caretaker Government (CTG) that governed Bangladesh from January 2007 – December 2008 that three leadership positions at the Upazilla-Parishad level were directly elected for the first time. In most levels of local government, little potential exists for generating revenue from permitted sources. The majority of resources from the central government flow to the various line ministries to programs managed by the line ministry staff, over which local government has minimal influence. In addition, at various times, local government structure changes depending on who is in power. The end result provides few opportunities for Bangladeshis to influence funding and decision-making at the local levels. Despite these challenges, there has historically been a vigorous national debate about what should be the main level of elected local government and their related authorities. Many rural Bangladeshis view the Upazilla Parishads level as closest to the people, and it is generally regarded more favorably than other elected and posted government officials. As part of the wave of reforms pushed by the CTG, several ordinances focused on establishment of a more robust legal framework for local government that provided for increased authorities, particularly for Upazilla Parishads. Specifically, it made the chairman and two vice chairman directly elected by voters, increased funds they would manage, and incrementally changed the Upazilla relationship to line ministry officials posted at the Upazilla level. A setback to these reforms occurred in April 2009, when the new administration enacted a law that failed to provide the newly elected Upazila officials with many of the authorities outlined in the CTG ordinance. Now according to the current law, Upazila elected Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 41 officials must seek advice from the Member of Parliament of that locale in decision-making. In practice, this undermines the Upazila councilors who are also directly elected by the people like MPs. The response to the new law among locally elected representatives and key constituencies including the two national local government associations, the Bangladesh UP Forum (BUPF) and Municipal Association of Bangladesh (MAB), was fast and vigorous. A heated debate has been underway, and in October 2009, a writ petition was filed by the BUPF to Bangladesh’s High Court to deem the law unconstitutional. Through USAID’s assistance, both BUPF and MAB have made significant progress in building a national constituency for local governance reform. The associations now have elected Executive Committees, a secretariat, and their own staff. The BUPF includes about 1,000 paid members, while the MAB has brought all 309 municipalities under its fold. These associations have also begun to provide a platform for local governments to advocate for greater autonomy in local decision-making and for a larger share of the national budget. Women’s participation at the local level has increased substantially over the last 15 years. A major change occurred in the rural areas in the mid-1990s when one-third of the Union Parishad seats were reserved for women. Immediately, the number of elected women officials grew to approximately 12,000. More recently, one of the three newly-elected positions to the Upazilla Parishad is reserved for a woman; however, the guidance from the Ministry of Local Government and Rural Development designated no authorities for these elected women. The USAID/Bangladesh Mission overall strategy on local government has focused on the promotion of a legal and policy framework conducive to effective, participatory local governance. First, USAID funded a program from 2001-2005 through ARD, Inc. entitled the National Constituency for Strong Local Government. This was followed by a two-pronged effort to support local governance starting in 2005 through activities in 134 elected rural and urban governments implemented by RTI International and the Bangladeshi organization, Rupantar, through the Democratic Local Governance Program (DLGP) and the Improving Local Governance by Strengthening Union Parishad and Creating Citizens’ Awareness (ILLG) program, respectively. The DLGP concluded in 2008, but ILLG, which ended in March 2011, remained active and expanded its scope and worked with a total of 214 local governments (210 Upazilas and four Municipalities). Interventions under ILLG encouraged national dialogues on decentralization policies to increase access to more resources and autonomy for local governments. In April 2008, USAID conducted an internal (USAID/Washington) evaluation of the Democratic Local Governance program, which included the RTI and Rupantar programs. The evaluation sought to provide insight on what has worked and what has not and offer recommendations for the next phase of local government activities. The assessment findings and recommendations have greatly informed the design of the SDLG program. The most significant achievements cited in the report included: 1) improvements in service delivery as a result of training and increased interactions between Union Parishads and citizens that helped to establish community priorities; 2) an increase in revenue generation in 105 Union Parishads by over 50 percent; 3) establishment of 85 (at the time of the report) Citizens Forums that enabled citizens for the first time to participate in local decision-making in a structured and constructive way; and 4) the impact and growing capacity of the BUPF and MAB to generate support/interest and advocate for local governance reform. Some of the weaknesses cited included the need to improve measurements on the quality and quantity of service delivery as well as the need for improved coordination with other donors and within USAID/Bangladesh on local governance issues and programming. Donor coordination is especially important as USAID has and will, through this SDLG program, work with a relatively small subset of local governments. Therefore, USAID’s main comparative advantage is in providing demonstration sites Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 42 or models for improved service delivery and citizen participation, in which the creation and dissemination of best practices is paramount. In addition, USAID efforts to provide support to the local government associations will continue to be instrumental in building a national constituency for reform and advocacy. It is important that the SDLG program not only build on the USAID/Bangladesh Mission’s past and current local government strengthening programs, but also seek synergies with other USAID and donor programs. The program is also designed to take into account the salient aspects of the USAID/Bangladesh Mission programs implemented under different Objectives. SDLG Result Framework Component 1 – Roles and Authorities of Local Governments Policy reform for greater roles and authorities for local governments depends on the will and commitment of policy makers. Constructive policy debate and sharing of experiences are critical to increase policy makers’ understanding of local government issues. In addition, sub-national governments in Bangladesh can and should participate more actively in this debate on the development of policies and legal reforms that affects their roles and resources. With support from USAID under DLGP, the BUPF and MAB have been relentlessly advocating for policy reform. Local governments’ roles change not only as result of legislative and regulatory actions but also through the gradual expansion of pilot practices or innovations initiated in response to felt needs or citizen demands. For example, the Mission’s work along with other donors on providing greater fiscal autonomy to the Union Parishads resulted in allocation of small direct block grants under the annual development budget from the Local Government Ministry directly to the Union Parishads. Although the amount of that grant was less than one percent of the annual development budget, that was a major breakthrough in the traditional policy and practice of channeling funds to UPs through the district and sub-district administrations of the central government. Subsequently, with the loan support from the World Bank and grants from some other donors, the Ministry of Local Government increased the amount of extended block grant for UPs under the LGSP. Component 3 – Transparent and Effective Service Delivery by Local Governments Component 2 – Advocacy and Capacity Building of Local Government Associations Component 1- Roles and Authorities of Local Governments Strategic Objective: More effective and responsive democratic institutions and practices Component 4 – Citizen Participation in Local Decision Making Component 5- Windows of Opportunity (crosscutting) Program Goal: Improve transparency and participatory public administration at the sub-national level and enhance legal and policy reform at the national level in order to promote and expand decentralization Figure 1 RESULTS FRAMEWORK Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 43 The objective of this component is to plan and carry out specific advocacy activities to encourage the national government to adopt legal and policy reforms to expand roles, authorities and resource allocation for local governments so that they can provide better service delivery. Component 2 – Advocacy and Capacity Building of Local Government Associations Capacity building is crucial for both the BUPF and the MAB to play an even greater role as the leading advocacy groups for local governments in the country. These two national level associations currently represent approximately one-third of all the local government units in Bangladesh and are poised to take the lead in promoting local governments’ concerns. Despite great success in advocating for reform and building a national constituency for local government, the capacity of the BUPF and MAB is mixed. Although the organizations do have a growing membership base, they are far from being financially self￾sustaining. Further, the associations continue to sort through challenging issues related to management, organization and general operations (legal, administrative, etc.). In addition, with elected Upazila Parishads in place, there may be a need for the formation of a new association that will represent the sub-national government at this level. Currently, the Upazila elected officials are raising their voice for policy change, but their advocacy is ad hoc. A forum of Upazila Parishads representing their collective voice could join the policy advocacy efforts carried out by BUPF and MAB. The objective of building capacity of the local government associations is to increase their ability to advocate and lobby on behalf of the sub-national governments that bring about broad policy changes that empower local governments and advance democratic decentralization. Component 3 – Transparent and Effective Service Delivery by Local Governments Successful decentralization depends on the ability of local governments to deliver effective public services in a transparent and efficient manner. To that end, local government elected officials and managers need effective management tools to expand and improve service delivery in cost-efficient ways. There is a wide array of potential in-country capacity that can be drawn on and/or mobilized, including research institutes, NGOs and the private sector. In addition, Information and Communications Technologies may offer cost-saving and effective approaches for improved performance of local government managers and administrators. The principal emphasis of this component is focused on achieving an immediate impact on local government management capacity through on-the-job training and technical assistance. The program should also help identify complementary areas of training and longer-term reforms or innovations that would bring further improvements. The role of the program is to generate new ideas and innovations through discussion, debate and consensus building. Lessons learned in this component should be channeled into the national policy reform dialogue. The objective of building the capacity of the targeted local governments is to enable them to be effective and transparent service providers at the sub-national level. D. Component 4 – Citizen Participation in Local-Decision Making Transparency in public management and in the use of public funds is fundamental to citizens’ trust and confidence in their local governments. Transparency can only be established by promoting and requiring the use of participatory planning by local governments and an active involvement of the citizenry in the affairs of the elected councils. USAID/Bangladesh Mission assessments and observations indicate that Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 44 initiating participatory processes is important to bridge the gap between elected officials of local governments and their constituents. The objective of increasing citizen participation in local decision-making is to ensure transparency and accountability in public management will lead to improved service delivery. Gender Perspective of the program: The SDLG activity supports women and men to enhance their leadership and management skills. Leadership training for women local representatives and women local council candidates is a priority for this project. Training for the newly elected women Vice Chairs of the Upazila Parishads as well as the Union Parishad representatives in their roles and responsibilities and other related issues is a major focus of this contract. The dynamics of women’s and men’s relationships as well as the complexities and inter-relatedness of their roles in improving governance at the local level cannot be simply addressed by any single activity. However, ensuring that women receive the leadership and management skills necessary to be successful at the local level is essential in promoting women as capable and reliable leaders. Likewise, they should be recruited and trained for local government jobs. The program promotes the concept of expanded roles and responsibilities for women through various seminars, workshops and conferences. Youth Perspective of the program: Bangladesh has a large youth population. There are approximately 45.7 million youth between the ages of 10 to 24, representing around 32% of the total population. Youth have enormous potential to play a pivotal role in promoting local level democracy by helping the local governments serve their constituents effectively. The program tries to involve the youth in the Citizens’ Forum, which will not only provide them with the opportunity to play a watchdog role but also help build them as future leaders. II. Objectives of the Evaluation The objective of the performance evaluation is to measure the development outcomes of the program with a view to drawing lessons learned for the selection, design, and implementation of future projects. The performance evaluation will also assess the relevance and sustainability of the program outcomes. The evaluation will:  Assess SDLG program’s actual results against targeted results;  Assess the efficacy of the SDLG implementation tools and management structure in meeting the objectives;  Make recommendations to USAID/Bangladesh concerning future programming with Local Government. The audience for this evaluation is USAID/Bangladesh, USAID/Washington leaders of USAID Forward, other USAID missions, The Tetra Tec ARD, relevant stakeholders such as BUPF, MAB, Elected Local Government, Community leaders, community people and existing USAID implementing partners etc. III. Evaluation Questions The evaluation should review, analyze, and evaluate the SDLG program by answering the following evaluation questions, and where applicable, identify opportunities and make recommendations for future programming with Local Government. In answering these questions, the Evaluation Team should assess Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 45 both the performance of USAID and that of the implementing partner(s). The evaluation questions, in order of priority are: 7. Effectiveness: To what extent has the SDLG program been successful in achieving its planned objectives? To what extent has the program resulted in unexpected outcomes (positive or negative)? The Local Governance sector in Bangladesh receives the largest amount of development assistance. What added value has USAID brought to the sector through SDLG? What has SDLG done differently? 8. Sustainability: How sustainable are SDLG program activities, and what measures could have been taken to enhance sustainability? 9. Efficiency: Are the objectives being achieved economically by the project intervention? (Comparison: resources applied – results)? Is there evidence from the implementation of SDLG to suggest that alternative program approaches may have been more successful? 10. Relevance: To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh, and will they provide sufficient guidance for appropriate programmatic and technical assistance decisions? 11. Management and Administration: How effective and flexible has the SDLG management been in working with implementing partners and beneficiaries, such as local government units, locally elected officials, citizens, and GOB? 12. Cross-cutting Issues: To what extent were gender, youth, and disability issues addressed by SDLG’s interventions in the targeted areas? IV. Proposed Evaluation Methodology The detailed methodology of this evaluation will be described by the evaluation team in the Work Plan; this will include presentation of an evaluation matrix that will explicitly link evaluation questions and sub￾questions to particular data collection approaches and data sources. In general, the evaluation will apply a mixed-methods approach, with an emphasis on comparative field￾based case studies of Local Government Units. Some quantitative analyses may be featured, for example, in the review of SDLG’s performance monitoring data or in the analysis of the program’s efficiency. The qualitative side of the evaluation will be incorporated to address several questions (regarding program relevance, management and administration, and sustainability, for example). In addition, the field data collection will involve intensive case study visits, organized around a set of semi￾structured individual interviews and group discussions. Individual interviewees will include: members of Local Government bodies, staff from Local Government ministries, staff of Donor organizations working with Local Government, Local opinion leaders, general community people, etc. The team will welcome suggestions from USAID as well as The Tetra Tech ARD and other evaluation stakeholders, for additional data sources at the community level. Discussion groups will include balanced numbers of men and women; in addition, as appropriate to local circumstances sex- or age-segregated discussion groups will be used to promote free discussion by women, men, and youth. The evaluation team will analyze the information collected to establish credible answers to the questions and provide major trends and issues. USAID requires that evaluations explore issues of gender; thus, the evaluation should examine gender issues within the context of the evaluation of SDLG activities. Methodological limitations and challenges for this evaluation are expected to include:  Ensuring adequate representation of interview and rapid appraisal sources vis-à-vis the full scope of SDLG activities and outcomes; and Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 46  Taking systematic actions to counter any biases in (a) reporting by data collection sources and (b) interpretations of collected data by the evaluation team. The methodology narrative should discuss the merits and limitations of the final evaluation methodology. The evaluation team will design appropriate tools for collecting data from various units of analysis. The tools will be shared with USAID during the evaluation and as part of the evaluation report. The evaluation team will be required to perform evaluation tasks in Dhaka, Bangladesh and also will travel to activity sites within the country VI. Existing Sources of Information USAID/Bangladesh DG Office will provide documents for the desk review that are not available outside. The list of available documents is presented in Annex A. The list is not exhaustive and the Evaluation Team will be responsible for identifying and reviewing additional materials relevant to the evaluation. The USAID/DG office will also help the evaluation team with contact information for relevant interviewees. VII. Deliverables All deliverables are internal to USAID and the evaluation team unless otherwise instructed by USAID. Evaluation deliverables include: Evaluation Team Planning Meeting (s) – essential in organizing the team’s efforts. During the meeting (s), the team should review and discuss the SOW in its entirety, clarify team members’ roles and responsibilities, work plan, develop data collection methods and instruments, review and clarify any logistical and administrative procedures for the assignment and prepare for the in-brief with USAID/Bangladesh; Work Plan - Detailed draft work plan (including task timeline, methodology outlining approach to be used in answering each evaluation question, team responsibilities, and data analysis plan): Within 5 working days after commencement of the evaluation; In-brief Meeting - In-brief with USAID/Bangladesh: Within 2 working days of international team members’ arrival in Bangladesh; Evaluation Design Matrix – A table that lists each evaluation question and the corresponding information sought, information sources, data collection sources, data analysis methods, and limitations. The matrix should be finalized and shared with USAID/Bangladesh before evaluation field work starts. It should also be included as an annex in the evaluation report. Data Collection Instruments – Development and submission of data collection instruments to USAID/Bangladesh during the design phase and after the evaluation is completed; Regular Updates - The Evaluation Team Leader (or his/her delegate) will brief the BDGPE COR on progress with the evaluation on a weekly basis, in person or by electronic communication. Any delays or complications must be quickly communicated to USAID/Bangladesh as early as possible to allow quick resolution and to minimize any disruptions to the evaluation. Emerging opportunities for the evaluation should also be discussed with USAID/Bangladesh. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 47 Debriefing with USAID - Presentation of initial findings, conclusions and preliminary recommendations to USAID/Bangladesh before the international team members depart from Bangladesh. Debriefing with Partners - The team will present the major findings from the evaluation to USAID partners (as appropriate and as defined by USAID) through a PowerPoint presentation prior to the team’s departure from the country. The debriefing will include a discussion of achievements and activities only, with no recommendations for possible modifications to project approaches, results, or activities. The team will consider partner comments and incorporate them appropriately in drafting the evaluation report. Draft Evaluation Report - – The Evaluation team will analyze all data collected during the evaluation to prepare a draft Performance Evaluation Report and submit the report within 10 working days on after the departure of international team members from Bangladesh. The draft report must be of a high quality with well-constructed sentences, and no grammatical errors or typos. The report should answer ALL the evaluation questions and the structure of the report should make it clear how the evaluation questions were answered. The draft report must meet the criteria set forth under the final report section below. USAID will provide comments on the draft report within ten working days of submission. The Evaluation Team will in turn revise the draft report into a final Performance Evaluation Report, fully reflecting USAID comments and suggestions, within five working days of receipt of the written comments; Final Report: The Evaluation Team will submit a final Performance Evaluation Report that incorporates Mission comments and suggestions no later than five working days after USAID/Bangladesh provides written comments on the draft Performance Evaluation Report. The format of the final report is provided below. The report will be submitted in English, electronically. The final report should meet the following criteria to ensure the quality of the report:  The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why.  Evaluation reports shall address all evaluation questions included in the scope of work.  The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Evaluation findings will assess outcomes and impact on males and females.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 48  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. The format of the final performance evaluation report should strike a balance between depth and length. The report will include a table of contents, table of figures (as appropriate), acronyms, executive summary, introduction, purpose of the evaluation, research design and methodology, findings, conclusions, lessons learned and recommendations. Where appropriate, the evaluation should utilize tables and graphs to link with data and other relevant information. The report should include, in the annex, any dissenting views by any team member or by USAID on any of the findings or recommendations. The report should not exceed 30 pages, excluding annexes. The report will be submitted in English, electronically. The report will be disseminated within USAID. A second version of this report excluding any potentially procurement-sensitive information will be submitted (also electronically, in English) to Development Experience Clearinghouse (DEC) for dissemination among implementing partners and stakeholders. All quantitative data, if gathered, should be (1) provided in an electronic file in easily readable format; (2) organized and fully documented for use by those not fully familiar with the project or the evaluation; (3) owned by USAID and made available to the public barring rare exceptions. A thumb drive with all the data could be provided to the COR. The final report will be edited/formatted by Social Impact and provided to USAID/Bangladesh 15 working days after the Mission has reviewed the content and approved the final revised version of the report. VII. Team Composition/ Technical Qualifications and Experience Requirements for the Evaluation Team The evaluation team will include and balance several types of knowledge and experience related to program evaluation. Individual team members should have the technical qualifications as described below: 1. Team Leader: An international Senior Evaluation Specialist (Program Development Specialist – Senior) with experience in evaluating Local Government programs in developing countries. The Team leader will provide leadership for the Team, finalize the evaluation design, coordinate activities, arrange meetings, consolidate individual input from Team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to USAID/Bangladesh. S/he should also have the number of years of experience and level of education required under the Program Development Specialist – Senior labor category. Experience in conducting assessments and designing strategic responses to Local Government in developing countries is required. Ability to produce highly quality evaluation report in English is essential. 2. Evaluation Methodologist: At least 5 years of experience in designing and conducting field￾based evaluations and assessments in the democracy and governance sector. Experience in conducting assessments and designing strategic responses to Local Government in developing countries is required. Relevant experience in Bangladesh preferred. 3. National Team Member: A national Senior Sector Specialist should have working experience with Local Government in Bangladesh. At least ten (10) years of experience in democracy and governance programs and some experience managing or implementing programs related to Local Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 49 Government in developing countries is required. Ability to conduct interviews and discussions and write well in English is essential. 4. National Team Member: A national senior or mid-level evaluation specialist should have at least 7 years of experience in designing and conducting field-based evaluations and assessments in the democracy and governance sector. Relevant experience in Bangladesh preferred. The proposed team composition will include one team leader and two/three team members. USAID strongly encourages the team to have one member from the LTTA staff for this Evaluation. All positions will be considered key staff and will require USAID approval. Overall the team will need expertise in USAID practices and expectations in program evaluation; program design and analysis; quantitative and qualitative data collection and analysis; survey design and analysis; program issues, innovations and challenges in promotion of public sector transparency and accountability; and USAID practices and requirements in program performance measurement. VIII. Conflict of Interest All evaluation team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the project being evaluated. USAID/Bangladesh will provide the conflict of interest forms. IX. SCHEDULING AND LOGISTICS Work is to be carried out over a period beginning from mid-February 2013, with field work completed in March, 2014 and final report and close out concluding o/a May, 2014. Funding and Logistical Support The proposed evaluation will be funded and implemented through the BDGPE project. Social Impact will be responsible for all off-shore and in-country administrative and logistical support, including identification and fielding appropriate consultants. Social Impact support includes arranging and scheduling meetings, translation services, international and local travel, hotel bookings, working/office spaces, computers, printing, photocopying, arranging field visits, local travel, hotel, and appointments with stakeholders. The evaluation team should be able to make all logistic arrangements including the vehicle arrangements for travel within and outside Dhaka and should not expect any logistic support from the Mission. The team should also make their own arrangement on space for team meetings and equipment support for producing the report. Please note that business calendar days and LOE days do not match up, as the evaluation team will not be working full-time on this evaluation while in the U.S. Extending calendar time past just the number of days of LOE allows SI time to review the evaluation report for quality and to provide comments back to the team to be addressed. This extra time does not cost USAID extra money, it simply improves the quality of the documents submitted. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 50 Task/ Deliverable Proposed Dates Business Calendar Days* LOE Days (estimated, days) Team Leader Evaluation Specialist National National Review background documents & preparation work (offshore): Draft work plan submitted to SI by 2/14 and to USAID/Bangladesh by 2/21 (Dhaka time) 1/27-2/20 15 3 3 3 3 Travel to Bangladesh by expat team member 2/20-2/22 3 2 - - Team Planning Meeting hosted by BDGPE 2/23 1 1 1 1 1 In-brief with USAID/Bangladesh 2/24 .5 .5 .5 .5 .5 Meet with SDLG/Tetra Tech ARD staff 2/24 .5 .5 .5 .5 .5 Data collection 2/25-3/15 16 16 16 16 16 Analysis and product drafting in-country Final work plan due to USAID (draft due from team to SI by 2/23) 2/26 Counted above Counted above Counted above Counted above Counted above Evaluation Team submits draft presentation to SI on 3/11 and then to the USAID/Bangladesh DG Team for review on 3/12; data collection continues after submission 3/12 Counted above Counted above Counted above Counted above Counted above USAID provides comments (as needed) on report outline and draft presentation 3/13 Counted above Counted above Counted above Counted above Counted above Presentation and debrief with DG Team and USAID/Bangladesh 3/16 .5 .5 .5 .5 .5 Debrief meetings with key stakeholders, including GOB 3/16 .5 .5 .5 .5 .5 Expat Team members depart Bangladesh 3/17-3/18 2 2 - - - Produce draft report. Team leader delivers draft report to SI/BDGPE on 15 Nov. 3/19-4/11 18 SI reviews draft report and provides 6 6 3 3 comments back to the team to address; delivers to USAID on 5/2 4/12-5/2 15 USAID and partners review draft and provide comments by 5/15 5/3-5/15 10 - - - - Team revises draft report and submits to SI by 5/16; SI comments back to the team and then once final, completes copy editing; finally, SI submits to USAID on 6/6 5/16-6/6 15 3 3 - - TOTAL 96 35 31 25 25 *Please note that this assumes a 6-day work week in the field and a 5-day work week before and after fieldwork. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 51 X. REPORTING REQUIREMENTS The total pages, excluding references and annexes, should not be more than 30 pages. The following content (and suggested length) should be included in the report: Table of Contents Acronyms Executive Summary - concisely state the project purpose and background, key evaluation questions, methods, most salient findings and recommendations (2-3 pp.); 1. Introduction – country context, including a summary of any relevant history, demography, socio-economic status etc. (1 pp.); 2. The Development Problem and USAID’s Response - brief overview of the development problem and USAID’s strategic response, including design and implementation of the SDLG program and any previous USAID activities implemented in response to the problem, (2-3 pp.); 3. Purpose of the Evaluation - purpose, audience, and synopsis of task (1 pp.); 4. Evaluation Methodology - describe evaluation methods, including strengths, constraints and gaps (1 pp.); 5. Findings/Conclusions - describe and analyze findings for each objective area using graphs, figures and tables, as applicable, and also include data quality and reporting system that should present verification of spot checks, issues, and outcomes (12-15 pp.); 6. Lessons Learned - provide a brief of key technical and/or administrative lessons on what has worked, not worked, and why for future project or relevant program designs (2-3 pp.); 7. Recommendations – prioritized for each key question; should be separate from conclusions and be supported by clearly defined set of findings and conclusions. Include recommendations for future project implementation or relevant program designs and synergies with other USAID projects and other donor interventions as appropriate (3-4 pp). Annexes – to include statement of work, documents reviewed, bibliographical documentation, evaluation methods, data generated from the evaluation, tools used, interview lists, meetings, focus group discussions, surveys, and tables. Annexes should be succinct, pertinent and readable. Should also include if necessary, a statement of differences regarding significant unresolved difference of opinion by funders, implementers, or members of the evaluation team on any of the findings or recommendations. The report format should be restricted to Microsoft products and 12-point type font should be used throughout the body of the report, with page margins one inch top/bottom and left/right. Annex-A List of Document (would be provided by USAID) 1. SDLG program document 2. SDLG PMP 3. SDLG performance report Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 52 ANNEX II: DOCUMENTS REVIEWED ARD (Associates in Rural Development) (2001) GOLD: USAID Support to Philippines Local Governance, Governance and Local Democracy Project 1995-2001. Compact disk should be available from TetraTech ARD. Asian Development Bank. 2012. The Urban Governance and Infrastructure Improvement Project in Bangladesh: Sharing Knowledge on Community-Driven Development (Manila: ADB, August). Baldwin, Kate, and Shylock Muyenga. 2014. Impact Evaluation of Supporting Traditional Leaders and Local Structures to Mitigate Community-Level Conflict in Zimbabwe, Final Report (Arlington, VA: Social Impact, August). Blair, Harry, Michael Calavan, Md. Azizur Rahman Siddique, and Naim Mostafa. 2012. Evaluation of the Improving Local Level Governance Project in Bangladesh: Combining Traditional Folk Arts with Democratic Local Governance (Dhaka: Social Impact, November). Blair, Harry. 2013. “Participatory budgeting and local governance,” in Joakim Öjendal and Anki Dellnas, eds., The Imperative of Good Local Governance: Challenges for the Next Decade of Decentralization (Tokyo: United Nations University Press), 145-178. Center for Urban Studies. 2011. Strengthening Democratic Local Governance: An Investigation into the Roles and Authorities of Local Governments in Bangladesh (Dhaka: CUS, 27 November). Center for Urban Studies. 2012. Research Report on Strengthening Democratic Local Governance: An Investigation into the Roles and Authorities of Local Governments in Bangladesh (n.p., April). Collier, David, and James Maloney. 1996. “Insights and pitfalls: Selection bias in qualitative research,” World Politics 49 (1), 56-91. Data International. 2011. SDLG Baseline Survey (Dhaka: Data International, June). GOB. 2009. Local government acts for Union Parishads, Paurashavas, and Upazila Parishads, accessed 22 June 2014 at <<<< http://www.dwatch-bd.org/lgacts.html>>. Kabir, Mahfuz. 2012. Leveraging the Periphery: Effective Decentralization at Upazila and Union Parishad (Dhaka: US, August). Keshishian, Mike, and Maureen Taft-Morales. 2008. Evaluation of the USAID/Bangladesh Local Government Activity (n.p., 7 October). Khan, Mohammad Mohabbat. 2009. Decentralization in Bangladesh: Myth or Reality? (Dhaka: A H Development Publishing House). Nijera Kori. 2009. Annual Report 2008-2009 (Dhaka: Nijera Kori). Accessed at . Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 53 Panday, Pranab Kumar. 2011a. Working of the Upazila Parishad: Selected Case Studies (Final Report) (Dhaka: SDLG, 12 December). Panday, Pranab Kumar. 2011b. “Local government system in Bangladesh: How far is it decentralized?” Lex Localis – Journal of Local Self-Government 9 (3), 205-230. Rahman, Hossain Zillur, and S. Aminul Islam. 2002. Local Governance and Community Capacities: Search for New Frontiers (Dhaka: University Press Limited). Sabet, Daniel M., and Monzurul Haque. 2014. SDLG Endline Survey Report (Dhaka: Org-Quest Research, April). Sabet, Daniel, and Mijanur Rahman. 2013. SDLG Midline Survey Report (Dhaka: Data International, April). SDLG. 2011-2012. Work Plans, 2011, 2012, 2013 (Dhaka: SDLG). SDLG. 2011-2013. Strengthening Democratic Local Governance (SDLG) in Bangladesh: Semi￾Annual Progress Performance Report, Numbers 1-5 (Dhaka: SDLG). SDLG. 2013. Strengthening Democratic Local Governance (SDLG) in Bangladesh: Quarterly Progress Performance Report, Numbers 6-7 (Dhaka: SDLG). Siddiqui, Kamal. 2000. Local Governance in Bangladesh: Leading Issues and Major Challenges (Dhaka: University Press Limited). Sobhan, Farooq, et al. 2013. Local Government Audit and Accountability Systems: A Framework for Analysis (Dhaka: Bangladesh Enterprise Institute, March). Swiss Cooperation Office. 2009. Strategy Note for the Domain Local Governance (Dhaka: Swiss Cooperation Office, April). Swiss Development Corporation. 2013. Swiss Cooperation Strategy Bangladesh 2013-2017 (Dhaka: SDC, November). Unitrend. 2012. SDLG Insight Mining: Topline Findings, version 1.1 (Dhaka: Unitrend, November). Unnayan Shamannay. 2013. Research and Policy Advocacy on Local Governance: Role of International Donors and Local NGOs (Dhaka: US, 10 October). Unnayan Shamannay. 2013. Overlapping Sources of Revenue in LGU Laws, revised (Dhaka: US, 10 October). USAID. 2014. Local Systems: A Framework for Supporting Sustained Development (Washington: USAID, April). Vella, Francis. 1998. “Estimating models with sample selection bias: A survey,” Journal of Human Resources 33 (1), 127-169. World Bank. 2011. Project Appraisal Document on a Proposed Credit in the Amount of SDR 185.80 Million (US$290 Million Equivalent) to the People’s Republic of Bangladesh for a Second Local Governance Support Project (31 October). Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 54 World Bank. 2013. Project Appraisal Document on a Proposed Credit in the Amount of SDR 266.6 Million (US$410 Million Equivalent) to the People’s Republic of Bangladesh for a Municipal Governance and Services Project (26 November). Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 55 ANNEX III: EVALUATION INSTRUMENTS This annex consists of protocols for the team’s individual interviews (UP chairs & PS mayors; UZP chairs & UNOs; UP secretaries & PS executive officers) and then group interviews (UP and PS council members; CiG group members). Revised version Interviewer: AH HB NM WC PROTOCOL FOR INDIVIDUAL INTERVIEWS (UP CHAIR/VICE-CHAIR/MUNICIPAL MAYOR/DEPUTY) UP/PS name ______________________________ Date of interview ______March 2014 Respondent name___________________________ Position_________________________ 1. Background (how long have you been in the position, what did you do before that, etc.) 2. Did you participate in the SDLG training workshop(s)? If so, what did you think of the training? Was it useful (e.g., in explaining the new local government laws, in other ways)? What was the most useful topic? How has SDLG done with follow-up to the training? Did the Program Officer make regular visits? If so, was s/he helpful with good advice and guidance? 3. When did the CiG program begin here (how many budget cycles has it gone through)? In the time that you’ve been working with the CiG groups, what has been your experience? To what extent were they helpful and in what ways? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 56 4. One major task for the CiG groups has been to support the UP/PS planning process. How has this worked? What have they done? 5. How have the Standing Committees been doing? Do you find them useful? Which ones are most important? Have the CiG groups been helpful here? 6. Has UP/PS revenue generation improved? Has SDLG been helpful here? 7. One SDLG objective was to improve service delivery. Do you think it has improved? If so, how? In what sectors (health, education, agriculture)? Can you give some examples? What does “transparency” mean to you? Do you think local governance has become more “transparent”? 8. Has the CiG process had any effect in improving candidate selection for social safety net programs (VGD, VGF, widow’s allowance, etc.)? How about the management of these programs? 9. Has women’s participation in UP/PS governance increased? If so, what are women doing now that they didn’t do earlier? Youth? 10. What has happened after the end of SDLG in December 2013? Does the CiG system seem sustainable to you? What has SDLG done to encourage it to continue? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 57 11. Is your union parishad/paurashava a member of BUPF/MAB? What is the value of these organizations? 12. What do you think is the Most Significant Change that SDLG has played a role in? 13. Improvements. Every donor-supported program can be improved. SDLG has been a “pilot program” reaching about one-tenth of all UPs in Bangladesh and about one-sixth of all the paurashavas. If USAID (or some other donor) would like to support an expansion of the program to other LGUs in the future, what changes would you suggest? What improvements could be made in the program? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 58 Revised version Interviewer: AH HB NM WC PROTOCOL FOR INDIVIDUAL INTERVIEWS (UPAZILA PARISHAD CHAIR/VICE-CHAIR, NIRBAHI OFFICER) UP/PS name ______________________________ Date of interview ______March 2014 Respondent name___________________________ Position_________________________ 1. Background (how long have you been in the position, what did you do before that, etc.) 2. Did you participate in the SDLG training workshop(s)? If so, what did you think of the training? Was it useful (e.g., in explaining the new local government laws, in other ways)? What was the most useful topic? How has SDLG done with follow-up to the training? Did the Program Officer make regular visits? If so, was s/he helpful with good advice and guidance? 3. In general, would you say that the UP/PS chairs have improved their performance, now that they have taken the SDLG training? Do they know more about their jobs? Or is it hard to see much difference in them? 4. One major for SDLG has been to support the UP/PS planning process, beginning with the ward shava planning meeting and continuing with the annual open budget meeting and final UP decisions on budget. This should have resulted in your receiving annual plans that are better prepared. Have you seen any improvement in these plans? 5. Has this upazila participated in the Upazila Resource Team (URT) program, which has included training from NILG? If so, has it been useful? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 59 6. Has your UZP joined any of the national UZP associations? If so, what benefit do you see in the one you joined? What about political partisanship in these organizations? 7. Improvements. Every donor-supported program can be improved. SDLG has been a “pilot program” reaching about one-fifth of all UZPs in Bangladesh. If USAID (or some other donor) would like to support an expansion of the program to other LGUs in the future, what changes would you suggest? What improvements could be made in the program? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 60 Revised version Interviewer: AH HB NM WC PROTOCOL FOR INDIVIDUAL INTERVIEWS (UP SECRETARY/PS ACCOUNTANT) UP/PS name ______________________________ Date of interview ______March 2014 Respondent name___________________________ Position_________________________ 1. Background (how long have you been in the position, what did you do before that, etc.) 2. Did you participate in the SDLG training workshop(s)? If so, what did you think of the training? Was it useful (e.g., in explaining the new local government laws, in other ways)? What was the most useful topic? How has SDLG done with follow-up to the training? Did the Program Officer make regular visits? If so, was s/he helpful with good advice and guidance? 3. Have you been working with the CiG groups? If so, how many budget cycles has it been through? What has been your experience? To what extent were they helpful and in what ways (e.g., tax collection)? 4. Has UP/PS revenue generation improved? Which components have improved and by how much? Has SDLG been helpful here? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 61 5. What has been the balance in UP/PS revenues between grants from ADP or other outside sources and locally generated revenue? Has this balance changed since the SDLG program began? If so, how? 6a. [if this is a union parishad]. Could you tell us about your UP’s participation in the World Bank’s LGSP (Local Government Support Program)? How much funding does it provide, and how is it used? How does their evaluation component work? Are there any bonus aspects for good performance? 6b. [if this is a paurashava]. Did your paurashava participate in the UGiiP program (Urban Governance and Infrastructure Improvement Program) funded by the Asian Development Bank? If so, how much funding did the program provide, and when (phase 1 and/or phase 2)? How was the money used? 7. Do citizens have access to your accounts; can any citizen see the books? Have any citizens asked to see the books, either members of the CiG or ordinary citizens? 8. Improvements. Every donor-supported program can be improved. SDLG has been a “pilot program” reaching about one-tenth of all UPs in Bangladesh and about one-sixth of all the paurashavas. If USAID (or some other donor) would like to support an expansion of the program to other LGUs in the future, what changes would you suggest? What improvements could be made in the program? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 62 Revised version Interviewer: AH HB NM WC PROTOCOL FOR GROUP INTERVIEWS (UP/PS ELECTED MEMBERS) UP/PS name ______________________________ Date of interview ______March 2014 Respondents names (1) _______________________ (2) ______________________________ (3)__________________________________ (4) ________________________________ (5) _________________________________ (6) ________________________________ 1. Introductions & Background (how long have you been a UP/PS member, what is your occupation) 2. Did you participate in the SDLG training workshop(s)? If so, what did you think of the training? Was it useful (e.g., in explaining the new local government laws, in other ways)? What was the most useful topic? How has SDLG done with follow-up to the training? Did the Program Officer make regular visits? If so, was s/he helpful with good advice and guidance? 3. When did the CiG program begin here (how many budget cycles has it gone through)? In the time that you’ve been working with the CiG groups, what has been your experience? To what extent were they helpful and in what ways? 4. What do you think has been the Most Significant Change in UP/PS governance since the SDLG program began in 2012? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 63 5. One major task for the CiG groups has been to support the UP/PS planning process. What have they done? 6. How have the Standing Committees been doing? Which ones do you belong to? How many of them are active on a regular basis (periodic meetings, active monitoring of sectoral activities)? Do you find them useful? Have the CiG groups been helpful here? 7. Participation. Would you say that ordinary citizens have become more engaged in local government since SDLG began its work? If so, how? Participation in ward shava meetings? In the open budget meeting? What kind of participation (entering the discussions, or just attending)? 8. Are ordinary citizens making more demands on you and on the UP than before? If so, of what kind? 9. Has UP/PS revenue generation improved? Has SDLG been helpful here? 10. One SDLG objective was to improve service delivery. Do you think it has improved? If so, how? In what sectors (health, education, agriculture)? What does “transparency” mean to you? Do you think local governance has become more “transparent”? 11. Has the CiG process had any effect in improving candidate selection for social safety net programs (VGD, VGF, widow’s allowance, etc.)? How about the management of these programs? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 64 12. Has women’s participation in UP/PS governance increased? If so, what are women doing now that they didn’t do earlier? Youth? 10. What has happened after the end of SDLG in December 2013? Does the CiG system seem sustainable to you? What has SDLG done to encourage it to continue? 14. Improvements. Every donor-supported program can be improved. SDLG has been a “pilot program” reaching about one-tenth of all UPs in Bangladesh and about one-sixth of all the paurashavas. If USAID (or some other donor) would like to support an expansion of the program to other LGUs in the future, what changes would you suggest? What improvements could be made in the program? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 65 Revised version Interviewer: AH HB NM WC PROTOCOL FOR GROUP INTERVIEWS (CITIZENS-in-GOVERNANCE MEMBERS) UP/PS name ______________________________ Date of interview ______March 2014 Respondents names (1) _______________________ (2) ______________________________ (3)__________________________________ (4) ________________________________ (5) _________________________________ (6) ________________________________ 1. Introductions & Background (how long have you been a CiG member, what is your occupation?) 2. Did you participate in the SDLG training workshop(s)? If so, what did you think of the training? Was it useful (e.g., in helping you to become a good CiG member, in explaining what LGUs are supposed to do)? What was the most useful topic? How has SDLG done with follow-up to the training? Did the Program Officer make regular visits? If so, was s/he helpful with good advice and guidance? 3. When did the CiG program begin here (how many budget cycles has it gone through)? In the time that you’ve been working with the CiG groups, what has been your experience? To what extent were they helpful and in what ways? 4. What was your experience in working with the UP/PS? Were the chair and the members welcoming right away, or did it take some time to get integrated with them? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 66 5. Do you think the CiG participants have been able to make an impact on how the UP/PS does it work? If so, how would you describe that impact? 6. What do you think has been the Most Significant Change in UP/PS governance since the SDLG program began in 2012? 7. One major task for the CiG groups has been to support the UP/PS planning process, beginning with the ward shava planning meeting and continuing with the annual open budget meeting and final UP decisions on budget. How has this worked? 7. How have the Standing Committees been doing? Are you a member of one or more of them? How many of them are active on a regular basis (periodic meetings, active monitoring of sectoral activities)? Do you find them useful? Do you think the CiG members have made an impact with them? How? 8. Participation. Would you say that ordinary citizens have become more engaged in local government since SDLG began its work? If so, how? Participation in ward shava meetings? In the open budget meeting? What kind of participation (entering the discussions, or just attending)? 9. Has UP/PS revenue generation improved? Has SDLG been helpful here? 10. One SDLG objective was to improve service delivery. Do you think it has improved? If so, how? In what sectors (health, education, agriculture)? Have public servants become more Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 67 “accountable” to the citizenry? What does this term “accountable” mean to you? What does “transparency” mean to you? Do you think local governance has become more “transparent”? 11. Has women’s participation in UP/PS governance increased? If so, what are women doing now that they didn’t do earlier? Youth? 12. Has the CiG process had any effect in improving candidate selection for social safety net programs (VGD, VGF, widow’s allowance, etc.)? How about the management of these programs? 13. What has happened after the end of SDLG in December 2013? Does the CiG system seem sustainable to you? What has SDLG done to encourage it to continue? 14. Improvements. Every donor-supported program can be improved. SDLG has been a “pilot program” reaching about one-tenth of all UPs in Bangladesh and about one-sixth of all the paurashavas. If USAID (or some other donor) would like to support an expansion of the program to other LGUs in the future, what changes would you suggest? What improvements could be made in the program? Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 68 ANNEX IV: INTERVIEW LIST Dhaka interviews Organization Individuals interviewed Titles USAID Mission Rumana Amin Sherina Tabbasum Jason Smith Billy Woodward COR, BDGPE COR, SDLG DG Deputy Director DG Officer SDLG Jerome Sayre Zarina Rahman Khan Abu Md Mohsin COP, SDLG Deputy COP, SDLG Team leader, Citizen Participation MAB Shahmim Al Razi Md. Azmat Ullah Khan Secretary General, MAB President, MAB ADB M. Rafiqul Islam Elma Morsheda Senior Project Officer Project Officer JICA-UDCC Arika Munekata Advisor, Local Governance DFID Richard Butterworth Senior Governance Advisor GiZ Hans Joachim Hermann Principal Advisor World Bank Zahed H. Khan Christopher T. Pablo Senior Urban Specialist Senior Urban Development Officer DFATD (Canadian High Commission) Peggy Thorpe First Secretary (Development) TAF Hasan Mazumdar Russell Pepe Country Representative COP, PRODIP Unnayan Shamannay Shaheen ul Alam Mahfuz Kabir Project Coordinator Consultant SDC/Sharique Sohel Ibn Ali Lilia Tverdun Sharique Project Director SDC Program Officer Centre for Urban Studies Prof. Nazrul Islam Chairman Democracy Watch Taleya Rehman Wazed Feroj Executive Director Deputy Executive Director National Institute for Local Government Kabir M. Ashraf Alam Director General Data International Najmul Hossain A F. M. Azizur Rahman Managing Director Director UzPAB Faizur Rahman Fakir President BUPF Mahbubur Rahman Tulu President UNDP Shaila Khan Md Mozammel Haque Tofail Ahmed Md. Sydur Rahman Molla Assistant Country Director, Local Governance Project Manager, Upazila Governance Project National Technical Advisor Programme Analyst Ministry LGRD&C Mozammel Haq Ashok Madhab Rao Additional Secretary, LGSP II Additional Secretary, Municipalities Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 69 Organization Individuals interviewed Titles Shamima Nargis Mohammad Yah-ya￾Bhuian Ajit Kumar Paul Joint Secretary, Local Gov Div Deputy Secretary & Deputy Project Director, LGSP II Sr. Local Governance Audit Specialist, LGSP II Ministry of Liberation War Hon A. K. M. Mozammel Minister (former Executive Committee member, MAB) SDC Melina Papageorgios Trippolini Lilia Tverdun Sohel Ibne Ali Program Manager, Local Governance Portfolio Director, Local Governance Program Sharique Program Manager, Local Governance RTI/SGLG Azmal Hossain Former Program Officer, SGLG Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 70 Field interviews Organization Individuals interviewed Titles Kolaroa PS Saiful Islam PS Secretary Kolaroa PS Aktarul Islam Mayor Kolaroa UZP Aminul Islam Vice Chairman Kolaroa PS Rafiqul Islam Razu Jamil PS Council Members Shushila PNGO, Satkhira PS Mostafa Aktheruzzaman Mahabubul Alam Minto Sumon Chatterjee S.Abul Haranat Sushan Monelal Ahmed Sharif Moniruzzaman Golam Mostofa Deputy Director & Program Coordinator Project Officer Project Officer M&E Officer Program Assistant Program Officer Sr. Program Officer Program Manager Kerakata UP Rafiqul Islam Ismail Hossain Nurul Amin Mizanur Rahman CiG Male Members Keralkata UP Najma Momtaz CiG Female Members Keralkata UP AsgarAli Rabiul Islam Shohidul Islam Ashraf Ali UP Male Council Members Keralkata UP Nur Jahan Begum Sonia Lila UP Female Council Members Keralkata UP Morshed Ali UP Chairman Kolaroa Upazila Anup Kumar Talukdar Upazila Nirbahi Officer Kolaroa Upazila Nazrul Islam Upazila Chairman Kolaroa Upazila Aminul Islam Laltu Upazila Vice Chairman Kolaroa PS Aktarul Islam Mayor Kolaroa PS Saiful Islam Secretary Kolaroa PS Ishaque Ali Abdul Khaleque Sheikh Faruk Ahmed Ranajit Ghosh CiG Male Members Kolaroa PS Rashma Khatun Kalima Khatun CiG Female Members Kolaroa PS Zamil Hossain Rafiqul Islam Raju Male Council Members Jibannagar Golam Mortuza Sarhana Akter Rini UZP Chair UZP Vice Chair Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 71 Organization Individuals interviewed Titles Chuadanda PS Ramjan Ali Jahangir Alam Malik Shahidul Kador Joadar Md Sirajul Islam Monir Md Nazmux Salahin Liton Md Faraz Ali Sheikh PS Male Council Members Chuadanga PS Shakil Ahmen Md Abdul Kuddus Masud Hossain Jourdar CiG Male Members Chuadanga PS Sultana Ara Jahamra Begum Selina Yasmin (Shompa) CiG Female Members Chuadanga PS Saiful Arif Biswaw Acting Mayor Chuadanga PS Kazi Shoruful Islam PS Secretary Uthali UP Younus Ali Panel Chairman Uthali UP Liakat Ali Secretary Uthali UP Aminul Islam Abdul Kuddus Bazlur Rahman Tutu Mahbub Uddin Dalu Uddin Abdul Hannan UP Council Male Members Uthali UP Morium UP Council Female Member Uthali UP Beauty Shahina Akter CiG Female Members Uthali UP Abu Jafor Nurul Islam Bazlur Rahman Hafizur Rahman CiG Male Members Wave Foundation, PNGO SDLG Abdus Salam Akhtaruzzaman Assistant Project Coordinator Project Officer Democracy Watch PNGO, Jessor Dulal Hossain Md Mujibur Rahman Parimal Karmokar Layla Akter M&E Officer Program Officer Program Officer Program Officer Sahjadpur PS Golam Azam (Mawla) UZP Vice Chair & Acting Chair Sahjadpur Upazila Rasel Sabrim Upazila Nirbahi Officer Sahjadpur PS Najbul Islam Mayor Sahjadpur PS Shafiqul Islam - male Zabir Hussain – male Zakra Parvin - female CiG Members Sahjadpur PS Liaquat Hossain Pradip Kumar Poddar Md Yunus Ali PS Council Members Sahjadpur PS Beljur Rahman Khan PS Secretary Kayampur UP Zainul Alam Chairman Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 72 Organization Individuals interviewed Titles Kayampur UP Aklima Khatun UP Female Council Member Kayamput UP Abul Hasan Zahidul Islam Zakir Hossain Habibur Rahman UP Male Council Members Kayampur UP Sajeda Pervin CiG Female Member Kayampur UP Aktar Hossain Ramjan Ali CiG Male Member Manab Mukti Sangostha (MMS), PNGO Md Habibulla Bahar Gupal Chandra Shil\ Shamima Akhter Nusrat Jahan Md Fariduddin Md Abdul Halim Shahanar Begum Shuma Khatum Director Program Coordinator M&E Officer Program Assistant Finance & Administration Program Officer Program Officer Program Officer Garadaha UP Md Saiful Islam UP Chair Garadaha UP Humayun Kabir UP Secretary Garadaha UP Abdullah Md Azahar Ali Md Ayub Ali Mapul Rana UP Council Members Garadaha UP Ilias Hossain CiG Male Member Garadaha UP Champa Rani Dev Chama Rani Dev CiG Female Members Garaadaha UP Monerwara Khatun Zahera UP Female Council Members Pabna PS Jahurul Islam PS Council Member Pabna PS Kampul Hasan Mayor Pabna PS Ahmed Kabir Sajjad Hossain Nazmul Islam Alamgir Hossain CiG Male Members Pabna PS Farhad Joarden Md Razibul Hasan Md Shahidul Islam Ahraf Pramanik Ayub Ali Sardar PS Council Male Members Pabna PS Afroja Khatun Chhabi Toma Islam (Pushpa) Laily Begum Sardina Akter Shahana PS Council Female Members Pabna PS Md Sayedul Islam PS Secretary RDRS, PNGO Md Atiqur Rahman Program Officer Pabna PS Ray Hanna Islam Upazila Nirbahi Officer Barara UP, Pabna Abu Sayed Khan UP Chair Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 73 Organization Individuals interviewed Titles Barara UP Rejaul Karim Shofiqur Rahman Belal Hossain CiG Male Members Barara UP Champa Sajida Sanjida Akhter CiG Female Members Barara UP Rashida Khatun Jayda Khatun Ashiya Khatun Female Council Members Barara UP Shamsul Sardar Halimuzzaman Abul Kalam Abdus Sobhan Kamruzzaman Kuddus Male Council Members Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 74 ANNEX V: TABLES AND FIGURES Figure 1: SDLG Goals and Components Table 1 Field Visit Plan Date Division District Upazila Paurashava Union 2-3 Mar Khulna Satkhira Kolaroa* Kolaroa* Keralkata* 4 Mar Khulna Chuadanga Jibannagar* Jibannagar Chuadanga* Utholi* 9 Mar Rajshahi Sirajganj Sahjadpur* Sahjadpur* Kayampur* 10 Mar Rajshahi Sahjadpur Sahjadpur Sahjadpur Garadaha Rupati* 11 Mar Rajshahi Pabna Pabna Sadar* Pabna* Barara* *LGUs visited by the evaluation team Also interviewed four PNGOs Component 3 – Transparent and Effective Service Delivery by Local Governments Component 2 – Advocacy and Capacity Building of Local Government Associations Component 1- Roles and Authorities of Local Governments Strategic Objective: More effective and responsive democratic institutions and practices Component 4 – Citizen Participation in Local Decision Making Component 5- Windows of Opportunity (Crosscutting) Program Goal: Improve transparency and participatory public administration at the sub-national level and enhance legal and policy reform at the national level in order to promote and expand decentralization Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 75 Figure 2: SDLG Sample LGUs Visited Chuadanga (1 UZP, 1 PS, 1 UP) Satkhira (1 UZP, 1PS, 1 UP) SiSirajganj (1 UZP, 1 PS, 3 UPs) Pabna (1 UZP, 1 PS, 1 UP) Table 2: SDLG Evaluation Questions and Conclusions in Brief Question Sub-question Components 1 & 2 Components 3 & 4 1. Effectiveness 1a. Results Repetitive research Little institution building Mostly good PMP T/C results mixed 1b. Surprises Strong LGA pushback Enthusiasm level 1c. Added value US macro-level lobbying model. RevGen + CiG model 2. Efficiency Alternatives Long-term USAID strategy Longer LOP Confrontational advocacy 3. Management & administration– working with: 3a. SDLG partners Conflicts with LGAs PNGOs good but one-off 3b. Other donors Little linkage Little linkage 3d. LGU elected Little LGA work with its members Well done 3e. Citizens Little involvement CiG good Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 76 Question Sub-question Components 1 & 2 Components 3 & 4 3f. GOB macro￾level No sustained engagement Not part of project design 4. Sustainability 4a. Program activities MAB may survive BUPF & UZPAB less likely CIG enthusiasm but little institutionalization 4b. Possible measures Links to other LGAs Longer LOP 5. Relevance 5a. To present dev situation Little technical depth CiG an excellent model 5b. To future programming Needs revamped strategic vision CiG model should be spread 6. Cross-cutting 6a. Gender Some mention Legal & SDLG reqts met Small leadership programs 6b. Youth Little attention given Little attention given Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 77 Table 3: SDLG Spending on Support to LGAs March 2011 to December 2013 (in US$) LGA Core Costs Secretariat Program Costs - Direct Program Costs – Grants Total MAB 58,130 149,383 39,402 246,915 BUPF 44,355 281,798 16,275 342,428 UzPAB 20,556 82,180 - 102,736 Subtotal 123,041 513,361 55,677 692,079 Source: SDLG Table 4: Comparison of the Percent of Union Parishad Respondents who Report Being Aware of UP Efforts to Raise Revenue (PMP 4.2) Baseline Endline Treatment difference Control difference Group Treatment Control Treatment Control difference % % % % % % % Aware of efforts 4.1 4.5 11.6 4.1 7.5 -0.4 7.9 Unaware of efforts 31.0 49.6 29.6 33.6 Don’t know 64.9 45.9 58.8 62.3 Base (Unweighted) 921 359 750 300 Source: Sabet and Haque (2014). Table 5: Comparison of the Percent of Paurashava Respondents who Report Being Aware of PS Efforts to Raise Revenue (PMP 4.2) Baseline Endline Treatment difference Control difference Group Treatment Control Treatment Control difference % % % % % % % Aware of efforts 4.8 15.1 10.3 16.0 5.5 0.9 4.6 Unaware of efforts 31.1 31.1 28.0 23.8 Don’t know 64.2 53.8 61.8 60.2 Base (Unweighted) 479 81 288 120 Source: Sabet and Haque (2014) Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 78 Table 6: Comparison of the Percent of Union Parishad Respondents who Agree that the UP Manages their Funds with Transparency and Accountability (PMP 4.2) Baseline Endline Treatment difference Control difference Group Treatment Control Treatment Control difference % % % % % % % Yes 25.1 21.9 48.2 37.3 23.1 15.4 7.7 No 25.0 29.0 28.6 35.9 Don't know 50.0 49.1 23.2 26.8 Base (Unweighted) 898 349 553 228 Source: Sabet and Haque (2014). Table 7: Comparison of the Percent of Paurashava Respondents who Agree that the PS Manages its Funds with Transparency and Accountability (PMP 4.2) Baseline Endline Treatment difference Control difference Group Treatment Control Treatment Control difference % % % % % % % Yes 17.6 16.3 57.3 57.9 39.7 41.6 -1.9 No 23.0 18.6 21.7 19.1 Don't know 59.3 65.1 21.1 23.0 Base (Unweighted) 480 81 193 87 Source: Sabet and Haque (2014). Table 8: Own Source Revenue Expenditure Patterns in 2012-13 LGU Type Salaries Development Objectives Office et al. Unspent Total PSs 27% 31% 28% 14% 100% UPs 24% 50% 29% 6% 100% Source: SDLG. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 79 Table 9: Comparison of Citizens’ Evaluations of the Quality of Road Construction, Repair, and Maintenance in Union Parishads (PMP 3.5) Baseline Endline Treatment difference Control difference Group difference Treatment Control Treatment Control % % % % % % % Poor/Very poor 11.3 13.5 6.5 2.9 Moderate 35.9 38.9 36.7 43.3 Good/Excellent 52.8 47.7 56.9 53.1 4.1 5.4 -1.3 Base (Unweighted) 879 342 603 171 Total sample size 921 359 750 300 Source: Sabet and Haque (2014). Table 10: Comparison of Citizens’ Evaluations of the Quality of Maintaining Market Places for Citizens in Paurashavas (PMP 4.10) Baseline Endline Treatment difference Control difference Group difference Treatment Control Treatment Control % % % % % % % Poor/Very poor 6.1 5.7 5.5 2.9 Moderate 45.3 36.8 39.6 17.0 Good/Excellent 48.5 57.6 54.9 80.0 6.4 22.5 -16.1 Base (Unweighted) 352 57 80 44 Total sample size 480 81 288 120 Source: Sabet and Haque (2014). Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 80 ANNEX VI: PERFORMANCE MANAGEMENT PLAN FOR SDLG: TARGETS AND ACTUAL RESULTS, 2011-2013 SDLG Performance Indicators – Results of 2011 (Year 1), 2012 (Year 2) and 2013 (Year 3) Indicator Definition Treatment/ Control Baseline Target Yr. 1 Actual Yr. 1 Target Yr. 2 Actual Yr. 2 Target Yr. 3 Actual Yr. 3 LOP Target LOP Actual Component 1- Roles and Authorities of Local Governments 1.1 } Number of LGUs where information on local government services is readily available to community members (Citizen Charter) Treatment 0 0 0 500 394 500* 499 500 1.2 } Percentage of elected council officials and civil servants with increased understanding of local government issues and capabilities Treatment/ Control 4% of Sample 4% Baseline 4% 20% of Sample 22.48% 40% of Sample 51% 40% of Sample 1.3 } Number of policy dialogue roundtables or research conferences on local government held regionally or Treatment 0 1 1 7 12 4 4 12 Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 81 Indicator Definition Treatment/ Control Baseline Target Yr. 1 Actual Yr. 1 Target Yr. 2 Actual Yr. 2 Target Yr. 3 Actual Yr. 3 LOP Target LOP Actual nationally 1.4} Number of advocacy actions conducted by women elected representatives in local government Treatment 0 4 4 6 7 8 1 18 Component 2 – Advocacy and Capacity Building of Local Government Associations 2.1} Percentage increase of BUPF, MAB and Upazila Association’s own revenue** Treatment MAB Tk500,000 includes contributio ns 0% Baseline 0% Baseline 100% Tk1,000,000 + 270% Tk1,842,000 includes contributions (Tk404,000 in Dues) 150% Tk1,250,000 + 70.4% Tk852,000 in Dues MAB=150% Tk1,250,000 Treatment BUPF Tk50,000 includes contributio ns 0% Baseline 0% Baseline BUPF=3,000 % Tk1,550,000 + 1077% Tk588,500 in Dues BUPF=8000% Tk4,050,000 + 1968% Tk1,034,000 in Dues BUPF=8,000 % Tk4,050,000 Treatment Upz Assoc. Upz=Tk0 0% Baseline 0% Baseline 50% Dues Tk723,000 Tk155,550 in Dues 70% Dues Tk1,012,200 21.4% Dues Tk309,000 in Dues 70% Dues Tk1,012,200 2.2 } Total number of BUPF and MAB members** Treatment MAB= 200 BUPF= 3,000 Baseline Baseline MAB= 275 BUPF= 3,500 MAB-308 (67 dues paid) BUPF=593 MAB= 300 BUPF= 4200 MAB = 308 -136 dues paid BUPF = 1034 MAB 300 BUPF= 4,200 2.3 } Number of local and non￾governmental and public sector associations supported with USG assistance Treatment 0 2 2 3 (2* + 1) 3 3* 3 3 2.4 } Number of legal challenges to GOB limits on local gov’t authority Treatment 1 1 0 2 4 1 1 4 Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 82 Indicator Definition Treatment/ Control Baseline Target Yr. 1 Actual Yr. 1 Target Yr. 2 Actual Yr. 2 Target Yr. 3 Actual Yr. 3 LOP Target LOP Actual Component 3 – Transparent and Effective Service Delivery by Local Governments 3.1} Number of sub￾national govt entities receiving USG assistance to improve their performance Treatment 0 0 0 600 526 (including 26 Upazila) 600* 592 (including 92 Upazila) 600 3.2} Number of sub￾national govts receiving USG assistance to in￾crease their annual own-source revenues (OSR) Treatment 0 0 0 500 500 500* 500 500 3.3 } Number of individuals who received USG￾assisted training, including mgt skills and fiscal mgi, to strengthen local govt and/or decentralization Treatment 0 4,008 4,426 (revised) 24,610 23,207 25,900 (23,500* + 1,500) 35,058 30,118 3.4} % increase in tax revenue generated in targeted LGUs (own source revenue – OSR) Treatment / Control Avg. OSR Union – 220,000 / 220,000 Muni – 18,500,000 / 18,650,000 Baseline Baseline 30% above baseline Avg. OSR Union￾300,274/159,173 Mean: 21.7% incr. Median: 20.5% Incr. Muni￾11,072,225/1,049,2 78 Mean: 43.8% incr. Median: 58.3% incr. 50% above baseline Avg. OSR Union￾698,164/488,453 Treatment: +217.3% Control: +122.0% Muni￾40,078,165/15,017,6 21 Treatment: +116.6% Control: -19.4%. 50% above baseline 3.5} Percentage increase in citizen satisfaction with selected LGU Treatment % Good or Excellent Union-47% (local Baseline Baseline 15% above baseline % Good or Excellent Union-69.5% (local roads) 30% above baseline % Good or Excellent Union-56.7% (local roads) 30% above baseline Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 83 Indicator Definition Treatment/ Control Baseline Target Yr. 1 Actual Yr. 1 Target Yr. 2 Actual Yr. 2 Target Yr. 3 Actual Yr. 3 LOP Target LOP Actual services roads) Muni-37% (local markets) Result: 17.0% incr. Muni-67.8% (local markets) Result: 19.5% incr. Result: +10.0% Muni-55.1% (local markets) Result: +18.1% Component 4 – Citizen Participation in Local-Decision Making 4.1} Number of Citizens’ Forums established Treatment 0 0 0 500 499 500* 500 500 4.2} Percentage of citizens that are better informed about LGU revenue generation capacity and fiscal transparency. Treatment / Control % Yes Revenue Generation Union￾5%/4% Muni￾5%/18% Fiscal Transp. Union￾25%/19% Muni￾17%/25% Baseline Baseline 15% above baseline % Yes Revenue Generation Union-19.9%/9.7% Muni-25.3%/32.7% Increase in Union:15%/4.70% Muni:19.7%/15.4% Fiscal Transp. Union-41%/48% Muni-47%/36.7% Increase in Union:15.1%/25.2 % Muni:28.7%/19.5% 30% above baseline % Yes Revenue Generation Union-10.3%/4.0% Muni-10.0%/15.0% Increase in Union:5.3%/0.0% Muni:5%/ -3% Fiscal Transp. Union-48%/37% Muni-58%/55.2% Increase in Union:23.0%/18.0% Muni:41.0%/30.0% 30% above baseline 4.3} Participatory strategic planning and budgeting implemented in targeted LGUs Treatment 0 0 0 500 247 600 (500* +100) 498 (including 8 Upazila) 600 4.4} Number of local mechanisms supported with USG assistance for citizens to engage their sub-national government. Treatment 0 0 0 3 5 5 (3* + 2) 5 5 Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 84 Indicator Definition Treatment/ Control Baseline Target Yr. 1 Actual Yr. 1 Target Yr. 2 Actual Yr. 2 Target Yr. 3 Actual Yr. 3 LOP Target LOP Actual 4.5} Percentage of citizens that feel their input and feedback was considered in local government decision making process Treatment / Control % Somewhat or Very Much Union- 34% / 30% Muni- 29% / 26% Baseline Baseline 15% above baseline % Somewhat or Very Much Union-99.4% Muni- 99.4% *For increase, see the footnote on 4.5 below 30% above baseline % Somewhat or Very Much Union- 90% / 86% Muni- 91% / 89% Increase in Union:56.0%/56.0% Muni:62.0%/63.0% 30% above baseline **For MAB, annual dues are collected on a July-June fiscal year basis. Membership includes all municipalities but only some pay MAB dues. ** For BUPF and UzPAB, annual dues are collected on a calendar year basis. Membership is counted only for dues paying members. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 85 ANNEX VII: FOOTNOTES ON SURVEY BASED INDICATOR VALUES FOR 2012 (3.4, 3.5, 4.2, AND 4.5) The PMP indicators accumulate data from two sources: 1) implementing partners and 2) surveys (baseline, midline, and LGCI). There are four specific indicators (3.4, 3.5, 4.2, and 4.5) of which targets and actuals come from the baseline and midline surveys. To determine Year 1 values for the abovementioned indicators, the evaluation team used data from the baseline survey, and for Year 2 values, the team used midline survey data. However, after analysis of the midline findings, it impossible to keep the same values because the list of comparative LGUs for the baseline and midline changed due to technical error (see tables below). Thus, the value of paurashavas and union parishads in the baseline increased by three and ten, respectively due to this error. LGU type Baseline Midline Reason for changes Paurashava (PS) 20 17 Three PSs were removed as they were not recipients of the treatment as intended during the baseline. These three PSs are in fact part of the control group. Union Parishad (UP) 80 70 Ten Ups were removed as they were not recipients of the treatment as intended during the baseline. These ten UPs are in fact part of the control group. Total 100 87 The changing baseline values: Indicator 3.4 Type of LGUs Changing values Midline values Result Treatment UP 246,748 300,274 21.7% increase Control UP 193,582 159,173 -17.8% increase Treatment Muni. 7,699,576 11,072,225 43.8% increase Control Muni. 981,729 1,049,278 6.9% increase Indicator 3.5 Type of LGUs Changing values Midline values Result Treatment Union Good and Excellence 52.5% local roads 69.5% local roads 17.0% increase Treatment Muni. 48.3% local markets 67.8% local markets 19.5% increase Indicator 4.2 Indicator Changing values (Treatment/Control) Midline values (Treatment/Control) Result Revenue Generation Union-4.9%/5.0% Muni-5.6%/17.3% Union-19.9%/9.7% Muni-25.3%/32.7% Increase￾Union-15.05%/4.7% Muni-19.7%/15.4% Fiscal Transparency Union-26.5%/22.9% Muni-18.3%/17.3% Union-41.6%/48.1% Muni-47.0%/36.7% Increase￾Union- 15.1%/25.2% Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 86 Muni-28.7%/19.5% Indicator 4.5 Type of LGUs Changing values Midline values Result Treatment % of somewhat and very much Union-90.8% Muni- 91.6% Union- 99.4% Muni- 99.4% Increase￾Union-8.6% Muni-7.8% Control Union-89.2% Muni -75.0% Union-100% Muni-100% Increase￾Union-10.8% Muni-25% Special Note for Indicator 4.5 Indicator 4.5 was originally based on the question, “If there is no Open Budget or Participatory Planning Meeting, how well do you think your views and input are considered by the UP/paurashava during its decision making?” The target for this indicator was set at 15 percent increase in the percent of citizens who responded “very much” or “somewhat” rather than “not at all” among midline respondents. Unfortunately, a change in the instrument made comparison between the baseline and the midline impossible. Perhaps more importantly, however, this question would have offered very limited means to measure project impact. Individuals who had developed an appreciation for the SDLG-promoted open budget and planning meetings as a result of the intervention might have actually been less likely to state “very much” or “somewhat” rather than more likely.28 As a result of these two methodological problems, the evaluation team selected an alternative variable that offers a better measure of the intended concept of “voice.” Respondents who were aware of planning/ward were asked, “Do you think that citizens can voice their views and opinions in ward/planning meetings?” This was then repeated for those aware of open budget meetings. While these questions offer a better indicator, they still present a methodological challenge, as only a reduced percentage of respondents were aware of such meetings and therefore able to express an opinion of them. As such, to increase the sample size, the team combined respondent evaluations of both the planning/ward and open budget. This allows the team to be fairly confident in the resulting percentages among union parishad respondents and treatment paurashava respondents; however, the evaluation team still cannot be confident in the percentages in the paurashava control areas. As a result of the reduced sample sizes, in the tables that follow, we present both the percentages and the frequencies. 28 There is good reason to think that this would be the case. In the baseline, only 34 percent of respondents in the treatment group (and 30 percent of respondents in the control group) felt that their input was very much or somewhat considered in the absence of planning/ward and open budget meetings, a very poor evaluation compared to the generally positive evaluations provided by survey respondents of their public officials. In short, it seems likely that the question was inadvertently asking people to evaluate the new public participation mechanisms rather than the openness of public officials. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 87 The vast majority of union parishad respondents aware of either planning/ward meetings and open budget meetings felt that they could somewhat voice their views and opinions in these venues. Together, 99.4 percent of midline treatment respondents felt somewhat or very much that they were able to voice their views, compared with 90.8 percent of baseline respondents, a difference of 8.6 percentage points. Nonetheless, a similar change was observed in the control group, and as such, the team is not able to conclude with sufficient confidence that the intervention had an impact on perception of the ability to voice views. . Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 88 ANNEX VIII. EVALUATION DESIGN MATRIX – 26 FEBRUARY 2014 DATA COLLECTION STRATEGIES--In Response to Evaluation Questions Evaluation Question Assumptions/Com ments Field Visit Dhaka Interviews Document Review 1. Effective￾ness: To what extent has the SDLG program been successful in achieving its planned objectives? This is the core query of the evaluation. Our field visits will provide the main data to answer this ques-tion. Because we will visit UPs, PSs and UZPs, the sample set will be small and stretched, and the data gathered will neces-sarily be qualitative, but there will be enough detail from a broad spectrum of elected officials, govt officers, LG association personnel, and citizens to provide a good account. Dhaka interviews will give us a “big picture” here. USAID, SDLG, GOB, BUPF/MAB personnel will all be interviewed at some length. The survey data will be useful in gauging progess on SDLG components 3 and 4. SDLG’s periodic reports & other documents will provide details. Were there any unexpected outcomes? Their absence would be surprising. Local respondents will provide examples. Other donor experience should give insight on this. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 89 What added value has USAID brought to the local governance sector in Bangladesh through SDLG? The question asks what value has SDLG added to the art of strengthening local govern-ance, not just to governance in the LGUs supported. Local respondents will have plenty to say about this. The question also relates to sustainability (#4): will any added value endure beyond EOP? Institutional memories at this level (esp. USAID and GOB personnel) will probably be more reliable than those at local level, SDLG reports will show project outputs, enabling us to look for outcomes. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 90 Evaluation Question Assumptions/Comm ents Field Visit Dhaka Interviews Document Review 2. Efficiency: Is there evidence from the implementation of SDLG to suggest that alternative program approaches may have been more successful or was SDLG’s approach the most efficient? Bangladeshis love to speculate, so it will not be hard to elicit ideas here. Other donor LG programs encountered on our own field visits would provide ideas, but this would be serendipitous. Asking our own respondents “what could SDLG have done better” will elicit many responses. Other donors presently involved in LG programs (esp. World Bank, UNDP, SDC, JICA) have each used their own approaches, which we will compare with SDLG. If time permits, other donors with past experience could also be queried (e.g., DfID, DANIDA). GOB officials will also have ideas on this topic. We expect little attention to “paths not taken” in the documents we will review. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 91 Evaluation Question Assumptions/Comment s Field Visit Dhaka Interviews Document Review 3 Manage￾ment & Ad￾ministration: How effective and flexible has the SDLG management been in working with (a) implement-ing partners, Answers would come both from SDLG staff and the partners. Partners may show some reluctance in offering negative answers. Partners’ field staffs may be more forthcoming. Some partners will have their offices in the field, probably most will be headquartered in Dhaka. Extended interviews with all 9 partners will not be possible, but we should be able to meet those working in the field areas we will visit. Partners will have documentation required by USAID and GOB. Some may have conducted analyses on their own that will be useful. (b) other donors (e.g. World Bank LGSP) LGSP has been the heavyweight player in local govt programming. We will likely find some other donors active in the areas we want to visit, and will try to touch base with their field operations in those areas. Other donors: UNDP, SDC, DANIDA, JICA. World Bank should have much documentation, other donors as well. (c) and beneficiaries, such as local government units, UP/PS secretaries generally have the most useful information. We will seek out LGU secretaries, officers in sectors (TBD) supported by SDLG activities. (d) locally elected officials, LGUs engaged in UZP elections should be avoided (too much distraction) (e) citizens, CiG members will be interviewed (esp. women members). (f) and GOB? GOB officers in LGRDC ministry to be interviewed. LGRDC reports as available. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 92 Evaluation Question Assumptions/Comment s Field Visit Dhaka Interviews Document Review 4. Sustaina￾bility: How sustainable are SDLG program activities? Query: to what extent have USAID-supported LG programs enabled LGUs to avoid providing their own sustainability? What do local elected officials think is worth keeping after EOP? What are they willing to pay for? BUPF, MAB, UzPAB leaders – what do they think is worth keeping after EOP? What are MLGRD views on this? How prominent has sustainability been featured in USAID guidance to SDLG? How has it been framed? What measures could have been taken to enhance sustainability? How far could USAID and SDLG have pushed LGUs to increase local revenue mobilization? Given the low % local revenues have comprised of the LGU budget, even if fully collected, was it wise to emphasize this program component so much? LRM will be a major focus of our field interviews: (1) track record of LGUs; (2) views of elected officials, govt servants, citizens on LRM; (3) commitment at LGU level to emphasize LRM post-SDLG. SDLG surveys show detailed data on LRM, which will be useful in our field visits. Note: LRM = local revenue mobilization Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 93 Evaluation Question Assumptions/Comme nts Field Visit Dhaka Interviews Document Review 5. Rele￾vance: To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh, GOB policy on LG has always featured strong rhetorical support combined with implementation through patronage to benefit ruling party members while at the same time delivering some local-level development. This pattern cannot be expected to change under the present AL ministry. Even so, there is room for maneuver. BUPF and MAB have been trying to mobilize their members to engage in advocating policy change at the macro-level. We will ask how responsive are local member units of these bodies to these desires from above. We will also seek their views on SDLG efforts to build capacity among the member units. What are local views on citizen involvement in LGU activity? Has BUPF, MAB, UZPAF leaders in Dhaka will be asked for their views on training received from SDLG on advocacy and capacity building. In the past, BUPF and MAB documents have not been hugely helpful. After TA from SDLG, this may have changed. and will they provide sufficient guidance for appropriate programmatic and technical assistance decisions? Do respondents at local level believe it will continue to be worth while to pay annual dues to BUFP and MAB? Will citizen groups want to continue investing in the effort to have a voice in determining local public policy? What are BUPF/MAB/UZPAF plans for providing guidance to their members in the future? Can they be weaned from reliance on USAID for their basic support? Doubtful that documents will be helpful with this topic. Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 94 Evaluation Question Assumptions/Comme nts Field Visit Dhaka Interviews Document Review 6. Cross￾cutting Issues: To what extent were gender and youth effectively addressed by SDLG’s interventions in the targeted areas? Our impression based on previous Bangladesh experience, is that gender equality objectives have made some real (if imperfect) progress at local level… All field interviews, especially group interviews,will explore women’s inclusion in LGU activities. Additionally, women’s role in UPs, Ward councils, standing committees, and CiG groups will be explored. Did women benefit In ways different from men (e.g., inclusion in decision-making for the first time)? SDLG staff and NGOs implementers will be interviewed on this issue. SDLG reports … but youth have not been included in any meaningful role. Both impressions will be empirically tested in our field visits All field interviews, especially group interviews, will explore youth inclusion in LGU activities. Women’s role in UP, Ward councils, standing committees, and CiG groups will be explored. SDLG staff and NGOs implementers will be interviewed on this issue. SDLG reports Final Performance Evaluation of Strengthening Democratic Local Governance (SDLG) Project 95 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523