July 2015 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by William Cartier, AKM Saifullah, and Naim Mostofa for Social Impact, Inc. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period Cover Photo: Tax campaign organized by Citizens in Governance (CIG), Malanchi Union Parishad, Pabna Photo Credit: Rangpur Dinajpur Relief Service (RDRS), Bangladesh PERFORMANCE EVALUATION OF THE STRENGTHENING DEMOCRATIC LOCAL GOVERNANCE PROJECT EXTENSION PERIOD PROJECT EXTENSION PERIOD: MARCH 31, 2014 – FEBRUARY 28, 2015 July 2, 2015 USAID Contract AID-OAA-I-10-00003 Task Order AID-388-TO-12-00001 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period i CONTENTS Executive Summary ........................................................................................................................................................ ivv Introduction ................................................................................................................................................................ iv The Development Problem and USAID’s Response ........................................................................................ iv Evaluation Purpose and Evaluation Questions................................................................................................... iv Evaluation Methodology ........................................................................................................................................... v Findings and Conclusions......................................................................................................................................... v Priority Recommendations ................................................................................................................................... vii Introduction ........................................................................................................................................................................ 1 The Development Problem and USAID’s Response................................................................................................. 3 The Development Problem..................................................................................................................................... 3 USAID’s Response..................................................................................................................................................... 3 Evaluation Purpose ............................................................................................................................................................ 6 Evaluation Questions ................................................................................................................................................. 6 Evaluation Methodology................................................................................................................................................... 7 Data Collection Methods......................................................................................................................................... 7 Limitations.................................................................................................................................................................... 9 Findings and Conclusions............................................................................................................................................... 11 The Role of CIG Groups in Local Governance ............................................................................................... 11 The Role of Standing Committees in Service Improvement ......................................................................... 13 SDLG’s M&E Data on SC Service Monitoring................................................................................................... 18 Conclusions: Question 1........................................................................................................................................ 19 Linkages with USAID Projects/Sectors............................................................................................................... 20 Linkages with Other Actors.................................................................................................................................. 23 Conclusions: Question 2........................................................................................................................................ 25 Conclusions: Question 3........................................................................................................................................ 28 SDLG Activities with Women .............................................................................................................................. 29 SDLG Activities with Youth .................................................................................................................................. 30 Conclusions: Question 4........................................................................................................................................ 31 Recommendations........................................................................................................................................................... 32 Lessons Learned............................................................................................................................................................... 34 Annexes ............................................................................................................................................................................. 35 Annex I: Evaluation Scope of Work .................................................................................................................... 36 Annex II: Map of Evaluation Sites......................................................................................................................... 46 Annex III: Documents Reviewed.......................................................................................................................... 46 Annex IV: Persons Interviewed ............................................................................................................................ 49 Annex V: Evaluation Matrix ................................................................................................................................... 56 Annex VI: Data Collection Protocols.................................................................................................................. 59 Annex VII: Survey Protocol ................................................................................................................................... 66 Annex VIII: Select SDLG Project Performance Indicators............................................................................. 69 Annex IX: Disclosures of Conflict of Interest................................................................................................... 72 Tables Table 1: SDLG Extension Period Sub-IRs and Performance Indicators................................................................ 4 Table 2: SDLG KAP Survey Respondent Profile ........................................................................................................ 7 Table 3: Key Informants for Qualitative Data Collection........................................................................................ 8 Table 4: Demographic Profile of Survey Respondents............................................................................................. 9 Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period ii Table 5: Frequency of CIG Group Meetings in Previous Three Months........................................................... 12 Table 6: Activities of CIG Groups in Previous Three Months............................................................................. 12 Table 7: Frequency of Ward Meetings during Previous Three Months............................................................. 13 Table 8: Number of UPs Receiving SDLG Training for SCs ................................................................................. 14 Table 9: Monitoring Activities by UP Standing Committees................................................................................. 15 Table 10: Reported Frequency of SC Meetings in UPs.......................................................................................... 16 Table 11: Examples of Service Improvements as a Result of SC Monitoring.................................................... 17 Table 12: Perceived Improvement in Health Service Delivery (SDLG/Non-SDLG UPs) ............................... 17 Table 13: Perceived Improvement in Agricultural Service Delivery (SDLG/Non-SDLG UPs) ..................... 17 Table 14: USAID Project Overlap with SDLG ......................................................................................................... 20 Table 15: Status of MOUs with Sector Projects...................................................................................................... 21 Table 16: Differences between SDLG and non-SDLG UPs on KAP in Local Government.......................... 28 Table 17: Differences in Male and Female Participation and Knowledge........................................................... 30 Table 18: Differences in Youth and Adult Participation and Knowledge........................................................... 30 Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period iii ACRONYMS ACT Actions for Combatting Trafficking in Persons BDGPE Bangladesh Democracy and Governance Program Evaluations BNP Bangladesh Nationalist Party BUPF Bangladesh Union Parishad Forum CATI Computer-Aided Telephone Interviewing CEO Corporate Executive Officer CIG Citizens in Governance CREL Climate Resilient Environments and Livelihoods DRG Democracy, Human Rights, and Governance DLGP Democratic Local Governance Program FGD Focus Group Discussion GOB Government of Bangladesh ILLG Improving Local Level Governance IR Intermediate Result KAP Knowledge, Attitude, Practice KII Key Informant Interview LGRD Local Government and Rural Development LGSP Local Government Support Project LGU Local Government Unit M&E Monitoring and Evaluation MAB Municipal Association of Bangladesh MLGRD&C Ministry of Local Government, Rural Development, and Cooperatives MOU Memorandum of Understanding MP Member of Parliament NCSLG National Constituency for Strong Local Government NGO Non-Governmental Organization NHSDP NGO Health Service Delivery Project PHR Protecting Human Rights PMP Performance Management Plan PNGO Partner Non-Governmental Organization PS Paurashava (municipal government) RDRS Rangpur Dinajpur Relief Service SC Standing Committee SDLG Strengthening Democratic Local Governance SOW Scope of Work TO Task Order ToT Training of Trainers UDCC Union Development Coordination Committee UNO Upazila Nirbahi Officer UP Union Parishad USAID United States Agency for International Development USD United States Dollar USG United States Government UZP Upazila Parishad WB World Bank Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period iv EXECUTIVE SUMMARY INTRODUCTION The Strengthening Democratic Local Governance (SDLG) project was implemented by Tetra Tech ARD from December 2010 to March 2014. The project was extended for one year until February 2015, with a total overall budget of USD 14.7 million. During the extension year, coverage was reduced from 450 to 190 Union Parishads (UPs); from 50 to 10 paurashavas (PSs); and from 50 to 10 upazilas. This evaluation assesses SDLG’s performance during the extension year, focusing on its approach to improving UP and PS service delivery in some priority sectors for the U.S. Agency for International Development (USAID). THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE Service delivery in USAID/Bangladesh priority sectors is hindered by multiple governance challenges, the overall effect of which is to reduce the responsiveness and accountability of service providers at the local level. In response, the activities carried out by SDLG during the extension year included: • Component 1 – Roles and Authorities of Local Governments. SDLG supported field research on upazila policies and practices, and on best practices in local government unit (LGU) service delivery. Research results were disseminated through conferences and seminars. • Component 2 – Transparent and Effective Service Delivery by Local Governments. SDLG trained LGU officials and volunteer Citizens in Government (CIG) groups and LGU Standing Committees (SCs) on local governance and management topics, including monitoring and improvement of service delivery. • Component 3 – Citizen Participation in Local Decision-Making. SDLG provided training on LGU governance in the 200 selected UPs and PSs, supported “folk dramas” on local governance, and trained youth on video journalism in topics related to local governance and services. EVALUATION PURPOSE AND EVALUATION QUESTIONS The evaluation assesses SDLG’s results, particularly with respect to integration with other USAID priority sectors, and makes recommendations to USAID/Bangladesh concerning future programming in the local governance sector. The evaluation questions are: 1. How effectively has SDLG been able to integrate local governance issues to successfully address governance challenges in other sectors such as health, agriculture, and climate change? 2. How far has SDLG been able to ensure that the linkages it has created with the other USAID sectors are sustainable? What follow-up work needs to be done? 3. To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh?1 4. To what extent were gender and youth effectively addressed by SDLG’s interventions in the targeted areas? 1 The USAID Contracting Officer’s Representative clarified that this question was intended to query whether the SDLG project’s objectives provided sufficient guidance for appropriate programmatic and technical assistance decisions. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period v EVALUATION METHODOLOGY The evaluation team used a mixed methods approach: a review of literature and SDLG project documents, including SDLG’s monitoring and evaluation (M&E) plan2 and other quantitative data; key informant interviews (KIIs) and focus group discussions (FGDs) with 105 project stakeholders in seven UPs and two PSs in Rangpur, Jessore, Bogra, Satkira, and Rajabari districts; and a survey of 370 SDLG participants. In the presentation and discussion of findings, the report makes use of these data sources to triangulate the findings to support the conclusions and recommendations. One limitation faced by the evaluation team was recall bias, in this case, the tendency of respondents to meld their extension period activities with activities conducted during the SDLG Task Order (TO) base period. Another limitation was response bias in which participants tend to report only positive attributes of a project. The team countered this through strong facilitation of FGDs and through triangulation of responses from different informants. The evaluation team also faced challenges of selection bias. Due to political conflict in Bangladesh during the evaluation period, which limited travel, the team did not select at random the participants for the FGDs. The team countered this potential bias by triangulating KII and FGD findings with the survey results and review of SDLG project documents. FINDINGS AND CONCLUSIONS 1. How effectively has SDLG been able to integrate local governance issues to successfully address governance challenges in other sectors such as health, agriculture, and climate change? SDLG’s greatest success has been the mobilization of CIG group members to become formal members of SCs and to strengthen their capacity to carry out monitoring of service delivery in health, agriculture, women’s rights, and—to a lesser extent—climate change. Having worked for three years with the CIG groups enabled SDLG to activate the dormant SCs in the selected UPs in only a few months, ensure appointment of CIG group members, and engage them in monitoring of UP/upazila service providers. In this sense, the SDLG project’s extension year gave USAID/Bangladesh considerable value for money. 2. How far has SDLG been able to ensure that the linkages it has created with the other USAID sectors are sustainable? What follow-up work needs to be done? SDLG made some limited progress in establishing linkages with other USAID sector projects. One major issue that emerged was the absence of modifications to the TOs, budgets, and work plans of the other projects, which could have allowed them to deploy more resources toward SDLG activities. The short time and limited resources meant that the achievements in this respect were modest. Nevertheless, all of the project stakeholders agreed that the SDLG approach held promise. SDLG did not establish linkages with the upazila offices responsible for management of service delivery, which limited the impact of the SCs’ work in generating lasting solutions supported by the sector managers in the upazilas. It seemed that the SCs preferred to try to address the problems directly with the service providers, sometimes with the help of the UP chair. Moreover, SDLG did not have enough time to build these linkages during the extension year; the time constraint was compounded by the decision to begin close-out of sub-grants almost six months before the project end date, thus making it impossible to continue implementation in the field. 2 The SDLG M&E plan was approved in 2011 and was termed a Performance Management Plan (PMP). Following revised USAID usage, this report will refer to it as the M&E plan. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period vi 3. To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh? SDLG proposed a realistic, incremental strategy to reinforce the accountability of local government in a context in which there will probably not be major shifts in national government policies in the short term. This means that other institutional actors—Members of Parliament (MPs) and bureaucrats in the upazila offices—will continue to limit the autonomy of LGUs. Absent an attitudinal shift among national policymakers, donor programs for strengthening local government in Bangladesh have few strategic options except to focus on reinforcing citizen engagement as a way of holding local decision-makers accountable. As discussed under Evaluation Question 1, SDLG has been very successful in this regard. SDLG’s approach is an appropriate strategy in anticipation of more fundamental reforms of local government. Other countries’ experiences with decentralization reform offers plenty of cautionary tales. 4. To what extent were gender and youth effectively addressed by SDLG’s interventions in the targeted areas? SDLG was very effective in engaging women in project activities, particularly in promoting female councilors as SC presidents. The results of SDLG efforts in the high levels of activity of the respective SCs highlight the critical activist role of female UP councilors. The youth dimension of SDLG was attenuated, to say the least. While SDLG was able to recruit and retain youth leaders in the work with CIG groups and SCs, this did not translate into monitoring activities in function of UP/upazila services. PRIORITY RECOMMENDATIONS 3 1. USAID should clarify what it means exactly by sector linkages. For future local governance projects with a sector integration focus, the program design must take into account the difference between “sector linkages between projects” and “sector linkages of projects with local government.” The latter definition should be the main program objective. 2. USAID should fund a follow-on program to build upon SDLG’s achievements. The success of SDLG’s extension year in support of service monitoring and improvement through the mobilization of an active citizenry in the functioning of UP SCs is a notable achievement for USAID/Bangladesh. For this reason, the team recommends that USAID consider a follow-on program to build upon the success of SDLG’s extension year. 3. USAID should consider a bridge project. The team recommends that the SDLG process in LGUs be sustained. Understanding that the design and procurement of a follow-on program based on the results of the SDLG extension year might take more than one year to conclude, the evaluation team recommends that USAID consider a “bridge” project within the USAID Forward principles, contracted through a local non-governmental organization (NGO). 4. Follow-on projects should move quickly to establish SCs in partner LGUs. While SDLG’s three-year base period undoubtedly contributed to its ability to generate results in a relatively short extension period, this conclusion must be tempered with the acceptable performance of the UPs that were added to the project in the extension period. Any follow-on project should quickly incorporate new UPs and activate their SCs in a relatively short time. 5. USAID implementing partners in women’s rights should replicate SDLG’s work with SCs. The notable success of the Women and Children’s Welfare and the Resolution of 3 This list includes the evaluation team’s five priority recommendations, which have been slightly abbreviated for the Executive Summary. Please see the body of the report (pages 32-33) for the full list and complete text of the recommendations. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period vii Family Disputes SCs in the SDLG extension year should be analyzed by USAID’s other democracy, human rights, and governance (DRG) projects, specifically those in the fields of women’s rights and anti-trafficking. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 1 INTRODUCTION As in other developing countries, local governments in Bangladesh face increasing demands for better services while being subject to constitutional, legal, and regulatory norms that restrict their ability to meet these same demands. There are three levels of elected sub-national government. In rural areas, there are approximately 4,498 Union Parishads (UPs), with an average population of close to 30,000 people. Above the UPs, there are 508 Upazila Parishads (UZPs) with directly elected chairs and vice chairs. There is also a council composed of the chairs of all of the UPs within the jurisdiction of the UPZs, which have on average a population of about 300,000 people. Alongside these two levels of rural local governments, in urban areas there are 309 municipalities—or paurshavas (PSs)—and 11 city corporations, again with directly elected mayors and councils.4 The population of these urban governments ranges from as few as 25,000 people to as many as seven million people (for each of the two city corporations in Dhaka), all with largely the same service delivery functions. Despite this large number of local councils, Bangladesh continues to be one of the most centralized governments in the world, as measured by sub-national government expenditures as a proportion of total government spending.5 In Bangladesh this measure is 3.7 percent—2.3 percent going to the urban governments and only 1.4 percent going to rural governments. Most government spending in rural areas is done by a plethora of national government ministries, departments, institutes, and other agencies. This situation reflects the weak revenue base assigned to local governments; low tax effort on the part of all three types of local government units (LGUs); and a weakly-developed system of inter-governmental transfers, which allows for meager transfers (block grants) with significant discretion in transfer amounts on the part of the national government. Indeed, the current system of block grants and performance grants to UPs is part of a jointly-funded Government of Bangladesh (GOB)-World Bank (WB) project, and there is no legal guarantee that the UP grant facility will be continued once the project funding ends. In Bangladesh, local governments suffer from a variety of governance failures. While local elections are putatively non-partisan, candidates are usually chosen by political parties, which are organized through tens of thousands of party committees at all levels of local government (UP, UZP, PS). Members of Parliament (MPs) also play a key role in candidate selection. These dynamics undermine local government accountability, as performance issues take second place to partisan loyalties in elections. Local governments also are an integral piece of Bangladesh’s ingrained patron-client politics by organizing access to local-level opportunities for rent-seeking in coordination with the thousands of party committees at all levels of local government (UP, UZP, PS), and in consultation with MPs. Weak accountability and rampant corruption in local government disrupt service delivery in different ways, not least because of the involvement of civil servants in corruption. This also reinforces existing bureaucratic resistance to calls for greater access to information, which are seen to threaten rent-seeking. Over the last decade, there has been growing pressure for decentralization, increased fiscal resources, and more decision-making autonomy for local governments. Likewise, there have been numerous donor programs to strengthen local government and promote decentralization reform, including a series of programs funded by the U.S. Agency for International Development (USAID). Related initiatives funded by USAID/Bangladesh include the Democracy Partnership 1997-2002, National Constituency for Strong Local Government (NCSLG) 2001-2005, Improving Local Level Governance (ILLG) 2002-2011, and the Democratic Local Governance Program (DLGP) 2005-2008. 4 Data on local governments from the Ministry of Local Government, Rural Development, and Cooperatives (MLGRD&C). 5 See P. Smoke. 2014. “The Challenge of Local Government Financing in Developing Countries.” Barcelona: UN HABITAT, p. 6. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 2 The most recent USAID/Bangladesh local governance initiative—and the largest in terms of geographic coverage—is the Strengthening Democratic Local Government (SDLG) project. Implemented by Tetra Tech ARD, SDLG had a base performance period from December 21, 2010 to March 29, 2014, with a budget of USD 19.2 million. The project was extended for one year until February 28, 2015. However, due to funding limitations, the budget was reduced to USD 14.7 million. During the base performance period, SDLG worked with 450 UPs, 50 PSs, and 100 UZPs for a total of 600 LGUs. During the extension period, the project footprint was reduced to 190 UPs and 10 PSs in 50 upazilas. This performance evaluation looks at the SDLG project extension period with a view toward drawing lessons learned for the selection, design, and implementation of future projects—particularly with respect to the project’s success in integrating activities with other USAID priority sectors, in this case, health, agriculture, climate change, and other democracy, human rights, and governance (DRG) sector initiatives, specifically, protection of women’s rights. The evaluation purpose and methodology are discussed in detail in subsequent sections of this report. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 3 THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE THE DEVELOPMENT PROBLEM The SDLG base period Task Order (TO) modification does not give a general explanation of the development problems that it will address. Rather, there is a discussion of the development problems found in two of USAID’s priority sectors. First, in agriculture: Agricultural development, improved food security and poverty reduction depend on the effective delivery of public services to farmers and the rural poor. However, the efficient and regular delivery of public services to the poor in rural areas is hindered by far-reaching governance challenges. Reaching rural women is a particular challenge… One of the prominent governance challenges includes the leakage challenge that leads to the capture of support services…by better-off elites. Similarly, in the health sector, the SDLG extension period TO modification lays out several development problems: Health governance is extremely centralized, relationship of plans to budgets is blurred and local managers have little flexibility or capacity to interpret or adjust plans to fit changing circumstances. There is little culture of management for results and weak accountability to the community and beneficiaries at all levels. In addition, in urban areas the responsibility for public health services is shared among the Ministry of Health and Family Welfare and the Ministry of Local Government, Rural Development, and Cooperatives (MLGRD&C), which creates stagnancy in urban health governance leading to inequitable health progress among urban population, especially poor urban mothers and children. The SDLG TO modification does not discuss the development problems in the other service sectors that were included in SDLG’s extension year activities, i.e., climate change, and within the DRG sector itself, protection of women’s rights. USAID’S RESPONSE As discussed in the Introduction, USAID/Bangladesh’s successive local government programs have sought to address some of the major governance failures in LGUs. In the case of SDLG, during the base period, the project primarily focused on creating tools and practices that enable LGUs to become more democratic, effective, and responsive institutions of governance. The purpose according to the SDLG extension TO is to: “Improve rural people’s ability to demand better services and hold service providers accountable.” The development hypothesis set out in the SDLG extension period monitoring and evaluation (M&E) plan was that “USAID development activities in health, agriculture, and other sectors will be more effective and sustainable if integrated with a functioning local government system which possessed better processes of accountability and transparency including citizen oversight of service delivery.” In addition to supporting policy research and advocacy on decentralization and strengthening of LGU service delivery roles and responsibilities, the project worked on both the “supply” side to strengthen aspects of LGU management and the “demand” side to engage citizens in decision-making processes. The SDLG approach was to establish volunteer Citizens in Government (CIG) groups at the ward level of the UPs and PSs, represented by a CIG executive committee to liaise with the UP or PS leadership. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 4 The project extension period maintained this supply/demand approach, working with LGUs to promote improvementsin service delivery—even while continuing to work with members of CIG groups to support citizen monitoring of LGU services through participation in the legally-mandated Standing Committees (SCs). This provision was relatively new, having been introduced in a modification of the UP Law in 2012. The SDLG extension period introduced another innovation: use of the CIG groups and SCs to link the partner LGUs to other USAID sector projects in health, agriculture, climate change, and human rights, in order to improve the effectiveness and sustainability of those project interventions. SDLG extension period activities were organized in three components: • Component 1 – Roles and Authorities of Local Governments. SDLG supported research on upazila policies and practices, and on best practices in LGU service delivery, focusing on the role of the UP SCs. The research was disseminated through conferences and publications. • Component 2 – Transparent and Effective Service Delivery by Local Governments. SDLG used partner non-governmental organizations (PNGOs) to provide training to LGUs and CIG and SC members, including refresher training on LGU governance. SDLG introduced a training module on SCs and service delivery monitoring to increase SC members’ understanding of their roles and to give them practical tools—in the form of SC action plans and service checklists—for monitoring services. • Component 3 – Citizen Participation in Local Decision-Making. SDLG provided refresher training on LGU governance to all of the CIG members in the 200 selected LGUs, as well as trained and supported CIG members and LGU officials to carry out short “folk dramas” on LGU roles and governance processes. SDLG also trained youth on video journalism techniques. The selection of 200 LGUs for the extension period was based on overlap with the activities of other USAID sector projects in health, agriculture, climate change, and human rights. The SDLG work plan notes that “the objective of building linkages will be for SDLG to assist LGUs and local communities to better focus on, respond to, and manage important service delivery and local development issues that have also been identified by other USAID projects.” SDLG’s training helped LGUs and SCs to better understand service delivery roles and responsibilities while also involving the USAID sector projects, their local partners, and community counterparts to promote service improvements via linkages to the SCs. To this end, SDLG sought to establish a formal agreement with each of the USAID sector projects. Performance Indicators Table 1: SDLG Extension Period Sub-IRs and Performance Indicators Sub-IR 2.2: Improved Services in Health, Agriculture, Human Rights and Climate Change Percentage (%) of community served by clinics supported by groups of mobilized local influential stakeholders Number (#) of health clinics offering improved services in response to local SC oversight Number (#) of agriculture extension officer actions improving services in response to local SC oversight Number (#) of institutions with improved capacity to address climate change issues as a result of U.S. Government (USG) assistance Number (#) of citizens receiving information on climate change adaptation during local government outreach activities Number (#) of local SCs successful in preventing women’s rights violations Sub-IR 3.2: Increased Citizen Roles in Service Delivery Oversight Number (#) of citizens participating in local SC oversight SDLG M&E activities included data collection for a total of 13 performance indicators. In the SDLG M&E plan, the Results Framework for the project extension period incorporates two additional sub￾intermediate results (IRs) and respective performance indicators to reflect the greater emphasis on Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 5 work with the SCs and sector integration with other USAID projects. Those sub-IRs and performance indicators are shown in Table 1. In addition, SDLG conducted a Knowledge, Attitude, and Practice (KAP) survey of project participants, with a baseline and final measurement. The survey results were not used to populate the SDLG M&E indicators but rather were presented and discussed at length in a quarterly progress report to USAID.6 6 USAID/Bangladesh. SDLG Quarterly Report No 11 October 2014. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 6 EVALUATION PURPOSE As set out in the Evaluation Scope of Work (SOW) in Annex 1 of this report, the purpose of this performance evaluation is to measure the development outcomes of the SDLG project extension period with a view toward drawing lessons learned for the selection, design, and implementation of future projects. The performance evaluation also assesses the relevance and sustainability of the project outcomes. The evaluation: • Reviews SDLG’s overall performance during the extension period by assessing actual results against targeted results; and • Makes recommendations to USAID/Bangladesh concerning future programming in the local governance sector. During the one-year extension period, SDLG worked with local governments and citizens to initiate improvements in service delivery and an increase in citizen participation in service delivery. The extension was mandated to deepen and expand upon successful practices from the SDLG base performance period—especially in the area of SC oversight—to contribute to service delivery in the sectors prioritized by USAID. Thus, one of the main emphases in the SDLG extension period was to explore and create linkages with other non-DRG programs—specifically health, agriculture, and climate change; and with other DRG programs—specifically, the human rights and violence against women activities under the Protecting Human Rights (PHR) program. The audience for this evaluation is USAID/Bangladesh, USAID/Washington leaders of USAID Forward, and other USAID missions; Tetra Tech ARD; stakeholders such as Bangladesh Union Parishad Forum (BUPF), Municipality Association of Bangladesh (MAB), elected local government officials, community leaders, and community members; and other relevant USAID implementing partners and other bi-lateral and multi-lateral donors in Bangladesh. EVALUATION QUESTIONS The evaluation team was required to answer the following evaluation questions and, where applicable, identify opportunities and make recommendations for future programming in the local governance sector. In answering these questions, the evaluation team was asked to assess both the performance of USAID and that of the implementing partner. The specific evaluation questions in order of priority were: 1. How effectively has SDLG been able to integrate local governance issues to successfully address governance challenges in other sectors such as health, agriculture, and climate change? 2. How far has SDLG been able to ensure that the linkages it has created with the other USAID sectors are sustainable? What follow-up work needs to be done? 3. To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh?7 4. To what extent were gender and youth effectively addressed by SDLG’s interventions in the targeted areas? 7 The USAID Contracting Officer’s Representative clarified that this question was intended to query whether the SDLG project’s objectives provided sufficient guidance for appropriate programmatic and technical assistance decisions. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 7 EVALUATION METHODOLOGY DATA COLLECTION METHODS Following USAID guidance as set out in the January 2011 Evaluation Policy and related technical guidelines, the performance evaluation of SDLG extension period activities used a mixed (qualitative and quantitative) methods approach: a review of literature and SDLG project documents, including the M&E plan and other quantitative data produced by the project; key informant interviews (KIIs) and focus group discussions (FGDs) with a range of project stakeholders in selected SDLG project sites; and a survey of a representative sample of SDLG project participants from the 200 LGUs selected as extension project sites. See Annex V for an evaluation matrix showing how these different methods were used to address the evaluation questions. Document Review In that this evaluation seeks to compare actual results against targets, a primary methodology was the review and analysis of project documents. The evaluation team had access to all of the SDLG project documents, including quarterly progress reports, final report, M&E plan, and indicator data. The team also reviewed the policy analysis documents produced by SDLG during the extension period. The list of documents reviewed by the evaluation team is presented in Annex III. Table 2: SDLG KAP Survey Respondent Profile Respondent Categories Treatment (SDLG) Control (Non SDLG) No. Percentage No. Percentage UP Members/Chairs 165 34.8 57 26.9 Citizens – SCs 98 20.7 44 20.8 Citizens – CIG Groups 176 37.1 0 0 Citizens – No affiliation 35 7.4 110 52.0 Total 474 100 2128 100 Source: SDLG, KAP Survey 2014 In addition, SDLG conducted its own KAP survey of a sample of local government officials, members of SDLG’s CIG groups, members of local government SCs, and citizens (with no institutional affiliation) in the SDLG partner local governments and in a control group of non-SDLG local governments. The distribution of respondents is included in Table 2. The evaluation team attempted to triangulate some of the findings from other data sources with the results of the KAP survey. The team did not do its own data analysis of the KAP survey, but rather used frequency tables generated by SDLG. Qualitative Methods: Key Informant Interviews and Focus Group Discussions Qualitative data collection is an essential part of the mixed methods approach and allows the evaluation team to probe the knowledge, understanding, and opinions of project stakeholders about the activities, results, challenges, and achievements of the project, as well as their opinions about the performance of the implementing partner and the sub-contractors. Qualitative data collection methods consisted of both KIIs and FGDs with project participants. These activities were conducted in Dhaka and in selected SDLG project sites. See Annex II for a map of evaluation sites. As Table 3 shows, the evaluation team was able to meet with 105 key informants (64 8 The evaluation team found an error in the SDLG KAP survey tables, which calculate the number of “Control (Non SDLG)” respondents to be 212. Figures actually sum to 211. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 8 male, 41 female). A comprehensive list of key informants is found in Annex IV. The semi-structured protocols used for the KIIs and FGDs are found in Annex VI. Each of the protocols sets out a series of questions about the project objectives, activities, and outcomes. Data collection was conducted in two stages. The first stage took place between January 10 and February 1, 2015 and coincided with a violent blockade of roads by the opposition Bangladesh Nationalist Party (BNP). As a result of the blockades, the evaluation team was able to interview stakeholders from only four UPs and two PSs in Rangpur, Jessore, Bogra, and Satkira.9 It was not possible for the evaluation team to travel to any of the selected SDLG partner local governments. Instead, the team organized FGDs during daylight hours with CIG group members and local government officials in the district towns of Rangpur and Jessore, paying travel costs for the FGD participants. It was also possible to conduct some individual KIIs with the field staff of SDLG’s local implementing partners and with the implementing partners of other USAID sector projects who were based in the district towns. Upon analyzing the data collected in these four districts, the evaluation team determined that it was necessary to conduct further site visits to collect more information about the details of the integration between SDLG and the other USAID sector projects. Table 3: Key Informants for Qualitative Data Collection Division Key Informant Interviews Focus Group Discussions Subtotal Dhaka Males 9 - 9 Females 5 - 5 Rangpur Males 3 14 17 Females 2 10 12 Jessore Males 6 13 19 Females 1 9 10 Rajbari Males 7 - 7 Females 4 - 4 Bogra Males - 6 6 Females - 4 4 Satkhira Males - 6 6 Females - 6 6 Total 37 68 105 M: 64 F: 41 Thus, the team traveled to Rajbari district from March 15 to March 18, 2015, conducting individual KIIs with CIG group members, local government officials, staff of SDLG local implementing partners, and staff of other USAID sector projects. Owing to road blockades and the threat of political violence, the evaluation team could not travel to the three selected UPs in the district; but, it was possible to convene SDLG project stakeholders for meetings in Rajbari during daylight hours. Quantitative Methods: Survey To complement its document review, KIIs, and FGDs, the evaluation team contracted a survey of a random sample of SDLG project participants. This survey allowed the evaluation team to collect quantitative data across SDLG’s entire project area on some key aspects of project implementation. 9 In consultation with SDLG, the evaluation team had identified 12 LGUs (8 UPs and 4 PSs) to visit in Rangpur, Jessore, Satkira, and Bogra districts to conduct data collection. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 9 The random sample was taken from SDLG’s database of 5,400 participants in CIG groups and 4,000 members of UP SCs in the 190 partner UPs and 10 PSs. From this universe of 9,400 SDLG project participants in the partner local governments, a sample of 370 people was taken using a proportionate clustering method; of these respondents, 213 respondents (57.6 percent) were CIG group members and 157 (42.4 percent) were SC members.10 The data were collected by a team of eight enumerators using a computer-aided telephone interviewing (CATI) system, between February 3 and February 8, 2015.11 In that most of the participants’ phone numbers had been registered almost four years prior, it was necessary to make a large number of substitutions: in order to complete 370 questionnaires, 893 respondents were contacted. The most frequent reasons for substitution of respondents related to contact information in the SDLG database: telephone number no longer in service (251); respondent did not answer (138); or wrong phone number in the database (81). Only 53 substitutions were because of unwillingness to take the survey. Table 4: Demographic Profile of Survey Respondents Characteristic Percentage No. Sex Male 68.6 254 Female 31.4 116 Age Up to 34 years 27.3 101 >34 years 72.7 269 Education Below Secondary 39.2 141 Secondary and higher 60.8 219 No response 10 CIG Group Level Ward Committee 33.2 123 UP/PS Executive Committee 65.4 242 Other 1.4 5 Table 4 shows that approximately two-thirds of SDLG project participants surveyed from the partner LGUs—specifically in the CIG groups and the SCs—were male, more than 34 years old, and well￾educated (with secondary education or higher). This triangulates with the profile of participants in the FGDs, which were two-thirds men and often involved in business or contracting, or held other occupations like lawyers, teachers, rural doctors, or former government employees. Most of the survey respondents were members of the CIG groups’ executive committee at the UP or PS level. Along with three demographic variables (age, sex, and education), the survey protocol (See Annex VII) contained 18 “non-evaluative” questions about the respondents’ experience with SDLG’s activities; their knowledge and participation in local governance, specifically SCs and the CIG groups; and their knowledge about the focus and results of the service monitoring carried out by their SCs. LIMITATIONS 10 The sample size was smaller than SDLG’s KAP survey referenced above because the evaluation team’s survey did not use a control group. 11 In the CATI system, the questions are programmed and presented one by one within the software system installed on Android tablets, and the enumerators register the responses directly in the system, thus reducing data input requirements. The survey questions were piloted manually and revised prior to being programmed in the system. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 10 Recall bias is a common evaluation problem, which was compounded in this case by the evaluation’s focus on the SDLG project extension period. During the KIIs and FGDs with SDLG participants, respondents tended to meld their experience during the extension period with their experience during the previous 40 months of the base performance period. To counter this bias, the evaluation team took care to focus questions on the main extension period activities relating to CIG groups, SCs, and service monitoring. For instance, the survey of SDLG participants was focused on the topic of local government SCs and their effectiveness in service monitoring. Another methodological challenge is response bias. In Bangladesh, like in other contexts, it is frequent in evaluations to encounter “socially desirable response bias” in which participants tend to report only positive attributes of the project. Other drivers of response bias include the perception that negative responses will reduce the chance of a new project being approved by the donor. The evaluation team countered response bias through effective management of FGDs to encourage frank discussion of project successes and challenges, and through triangulation of responses from different target groups of key informants. Also, survey questions were non-evaluative, in that they focused on the respondents’ experience and knowledge rather than on their opinions about the SDLG project. In the team’s qualitative data collection, selection bias was also a challenge. Owing to the ongoing political conflict and the inability to travel to the project sites, it was not possible for the team to control the selection of all respondents. In some cases, the availability of respondents was owed to their location vis-à-vis that of the evaluation team; in other cases, selection bias might have reflected “self￾selection” in response to the incentive of paid transport to the district town. This might also explain the high proportion of male respondents. It may have also given the selection an “urban bias,” i.e., people living in or close to the district town could have been more likely to participate in data collection as opposed to people residing in more distant locations. While the evaluation team endeavored to select project participants at random, SDLG PNGO staff assisted with identifying replacements for people who could not travel during the road blockades. SDLG PNGO staff may have selected participants who would be favorable to the project. This challenge motivated the evaluation team to conduct a second stage of qualitative data collection in Rajbari district, to enhance generalizability of the findings. The survey had its own limitations, owing to the high number of replacements that were made as a result of changes in participants’ phone numbers. There may be an inherent bias with respect to changes in phone numbers; i.e., some categories of people may change phone numbers more or less frequently than others. For example, the sampling appears to have been biased toward women; while 31.4 percent of survey respondents were female, in the universe of SDLG participants, only 22.5 percent were female. The bias may reflect relatively lower proportions of women changing telephone numbers—or a higher proportion of women available to answer the call from the enumerator. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 11 FINDINGS AND CONCLUSIONS This section presents the main findings and conclusions of the performance evaluation, based on the methodologies discussed in the previous section. The presentation and discussion is organized following the sequence of the questions set out in the evaluation SOW. 1. How effectively has SDLG been able to integrate local governance issues to successfully address governance challenges in other sectors such as health, agriculture, and climate change? During the base period, SDLG created CIG groups in each ward in the partner UPs and PSs, along with an executive body in each local government representing all of the CIG ward-level groups. The CIG groups were described as a “helping hand” for the local government. According to the final performance evaluation of the SDLG base period, the CIG groups were the most innovative and effective of SDLG’s interventions, providing local governments with support in decision-making.12 In USAID/Bangladesh’s design of the SDLG extension SOW, the emphasis on citizen participation was maintained. In Component 2 of the project, SDLG introduced a new set of activities (training of SC members, facilitation of SC meetings, and organization of SC monitoring visits) to support SCs and service delivery monitoring in the areas of health, agriculture, climate change, and human rights, to increase SC members’ understanding of their roles and to give them practical monitoring tools such as SC action plans and service checklists. According to SDLG management, Component 2 was the principle focus of activities in the partner LGUs during the project extension year. It involved a large amount of training and detailed facilitation of the activities of the CIG groups and SCs and their thousands of members in 200 local governments. This task was aided by SDLG’s experience during the base period in 187 of the 200 partner LGUs. Only 13 of the LGUs selected for the extension period had not participated in the SDLG base period activities. Given the scale of the project activities, SDLG opted to continue the implementation approach used in the base period, which depended on PNGOs to identify and mobilize participants; facilitate the formation and strengthening of CIG groups and SCs; and train the members of CIG groups and SCs on structures and functions, roles and responsibilities of local governments, and in particular on the roles of the SCs with respect to monitoring and improvement of service delivery. As will be discussed in the findings on Evaluation Question 2, support to CIG groups and SCs involved other USAID project implementing partners in providing technical inputs into training materials and support to training. THE ROLE OF CIG GROUPS IN LOCAL GOVERNANCE According to SDLG management and staff, the CIG groups are intended to get citizens involved in the process of governance, especially development planning and improvement of services. It was expected that the CIG members on the one hand would raise their voices on behalf of the citizens for services, and on the other support the UPs in positively responding to citizen demands. The formation of CIG groups starts at the ward level where 10 citizens are chosen at a general meeting. Ten members from each of the wards in the UP or PS then met to form a 27-member executive committee. The process was facilitated by the field staff of SDLG’s PNGOs in their respective working areas. 12 H. Blair, W. Cartier, A. Hussain, and N. Mostafa. 2014. Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project: Macro level Advocacy and Local level Participation. Dhaka: USAID/Social Impact Inc. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 12 CIG groups supported by SDLG met quite frequently as shown in Table 5. Even after the main training activities were completed, only one-quarter of survey respondents reported that the CIG group had not met in the three months prior to the survey (although another one-quarter of respondents were unsure how many times it had met). Almost one-third of respondents said that their group had met twice or more. Table 5: Frequency of CIG Group Meetings in Previous Three Months Frequency Percentage No. Never 25.9 96 Once 21.4 79 Twice 13.8 51 Three times 12.2 45 More than 3 times 5.4 20 Don't know 21.4 79 Total 100.0 370 Source: BDGPE, SDLG Extension Evaluation Survey From respondents’ answers in Table 6 about the activities of their CIG groups during the three months before the survey, it appears that the main activities were assistance to the UP chair and councilors in identifying and solving problems. In the FGDs, this was often described as attending to social problems in the wards (crime, eave teasing, truancy, child marriage, dowry, domestic violence, human trafficking, etc.). In addition, survey respondents mentioned participating in door-to-door campaigns and rallies to improve UP tax collection efforts and open budget meetings in the wards, followed by monitoring of UP/PS projects, and monitoring of services, which is discussed in detail below. In this regard, CIG activities supported by SDLG during the extension year show considerable continuity with the activities conducted with CIG groups during the SDLG base period. Table 6: Activities of CIG Groups in Previous Three Months Activity of CIG Group Percent* No. Responses No activity 19 71 Worked with standing committees to monitor UP or Upazila services 16 58 Supported the UP tax collection 31 113 Held open budget meetings 28 104 Monitored implementation of UP projects 22 81 Helped the UP to identify and solve problems 38 137 Others 4 14 Don't know 21 76 Total - 654 Source: BDGPE, SDLG Extension Evaluation Survey *Multiple mentions, sums to more than 100% (out of 370 respondents) One activity that helps to explain the interest of UP chairs in the work of the CIG groups is ward meetings, in which residents are invited to contribute to annual development plans. According to the UP Act of 2009, the UPs are to organize ward meetings for making their development plans through generating priorities of the respective wards. Almost all FGD respondents attended open budget sessions in UPs—although they often mentioned there was not enough discussion on the priority projects undertaken by the UPs. However, the consensus in most of the FGDs was that it was better to start opening up decision-making in this way than have no mechanism at all, as it was before the reform. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 13 Holding ward meetings is also one of the pre-conditions for receiving the block grant from the central government through the Local Government Support Program (LGSP). In several FGDs, it was reported that the UP Service Center staff routinely photograph the ward meetings organized by SDLG, to document them for the report to LGSP in application for the block grant. The concerned UP councilor is responsible for organizing and presiding over the ward meetings; in this task, he or she receives the support of the respective CIG group in contacting and convening ward residents and organizing the meetings. Table 7 shows that 52 percent of survey respondents reported at least one ward meeting in the three months prior to the survey. Over half of respondents report that they regularly attend the meetings—either as participants, or as organizers or conveners. Table 7: Frequency of Ward Meetings during Previous Three Months Frequency Percent No. None 22 80 At least once 52 190 Don't know 26 95 Total 100 365* Source: BDGPE, SDLG Extension Evaluation Survey *There were only 365 valid responses to this question. THE ROLE OF STANDING COMMITTEES IN SERVICE IMPROVEMENT One of the differences between the SDLG base period activities and the extension year activities was the degree of emphasis on working with the SCs of the UPs (and to a lesser extent, PSs) in monitoring of services. During the base period SDLG had initiated some work with SCs, but this was incipient; the extension year SOW made it the central aspect of SDLG’s work. According to SDLG’s M&E reporting, by December 2014, a total of 1,718 people from the 190 partner UPs (excluding the 10 PSs) were involved in the service monitoring (“oversight”) activities carried out by the SCs targeted by SDLG. The decision to work with the SCs was in response to reforms to the UP Law in 2012. Prior to this reform, the SCs in UPs were composed almost exclusively of UP councilors, although the law allowed one non-voting member from the community. The change to the UP Law provided for up to six community members in each of the 13 statutory SCs of the UP, under the presidency of a UP councilor. SDLG took advantage of this reform and worked to reorganize the SCs in the partner UPs, recruiting the SC community members from the membership of the CIG groups that were functioning in the partner LGUs. Responding to the SOW requirements to focus on integration with other USAID projects, SDLG focused its activities on four of the committees: (i) Agriculture, Fisheries, and Livestock; (ii) Education, Health, and Family Planning; (iii) Women and Children’s Welfare, and Resolution of Family Disputes; and (iv) Audit and Accounts. SDLG dedicated significant time and resources to training members of the newly-constituted SCs in all of the project LGUs; indeed, this was the main focus of the project during the extension year, to prepare the SC members for their service monitoring activities. During 2014 there were many incidents of political conflict and violence at the local level that required SDLG to cancel training, notably in UPs in the Bagherhat, Satkira and Naogaon districts and to a lesser extent in Gopalganj, Pabna and Natore. As a result of these difficulties, as Table 8 shows, the project was successful in training the SC members in only 129 of the targeted 190 partner UPs. The training provided by SDLG to the SCs focused on 1) the use of monitoring tools in each of the service sectors; and 2) the drafting and presentation of reports to local authorities on the results of the monitoring. The training was at most two days—based on the evaluation team’s review of the progress reports of SDLG’s PNGOs, it appears that in some cases both training programs were imparted in a Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 14 single day. As discussed in detail below, in the design of the training materials and the formats used for monitoring, SDLG reached out to other USAID projects for expertise. Also, some of the other USAID projects provided staff for training. SDLG opted to use a Training of Trainers (ToT) approach to create a pool of qualified trainers among the field staff of the project’s PNGOs. Table 8: Number of UPs Receiving SDLG Training for SCs Districts Partner UPs Training on Monitoring Training on Reporting Sirajgonj 15 15 15 Bogra 16 16 16 Rangpur 12 9 9 Natore 12 8 8 Naogaon 6 1 1 Pabna 10 6 6 Jessore 14 14 14 Narail 6 6 6 Faridpur 6 6 6 Rajbari 5 5 5 Satkhira 21 0 0 Bagerhat 20 2 2 Gopalgonj 9 2 2 Kishoregonj 6 6 6 Chandpur 12 13 13 Lakshmipur 13 13 13 Noakhali 3 3 3 Feni 4 4 4 Total 190 129 129 Source: SDLG database SDLG’s attempt to activate the SCs to carry out monitoring of service provision in the different sectors appears to have been partially successful, at least as measured by the frequency of meetings and monitoring activities. Table 9 shows data from SDLG’s project monitoring database. In the 190 SDLG partner UPs during the extension year period, 807 SC meetings were held, and 843 service monitoring visits were carried out by the SCs. In that SDLG worked with four targeted SCs in each of the 190 partner UPs, there would be 760 active SCs. Thus, from the Table 9 results, it can be deduced that each SC conducted on average just over one meeting and carried out just over one service monitoring visit during the project extension period. The averages of SC meetings and monitoring visits in the partner UPs also seem to vary considerably between districts. Based on SDLG’s own data, districts such Naogon and Gopalganj, for example, show less than .5 meetings held per SC during the extension year period. These results might be due to the lower training effort in these and several other districts, owing to the ongoing political violence noted above. Another factor, of course, is the short period of SC “activation” supported by SDLG. Training of SCs took place in Quarters 2-3 of 2014; while the SDLG project extension work plan indicates that the facilitation of the SC meetings by PNGO field staff would continue through Quarter 4, it continued only to the end of October 2014, when the sub-contracts with the PNGOs ended. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 15 Table 9: Monitoring Activities by UP Standing Committees Districts Partner UPs SC Meetings Monitoring Visits Sirajgonj 15 31 75 Bogra 16 37 73 Rangpur 12 34 29 Natore 12 28 36 Naogaon 6 15 13 Pabna 10 41 44 Jessore 14 97 89 Narail 6 21 15 Faridpur 6 48 45 Rajbari 5 32 15 Satkhira 21 69 116 Bagerhat 20 43 98 Gopalgonj 9 18 41 Kishoregonj 6 59 29 Chandpur 12 89 54 Lakshmipur 13 93 41 Noakhali 3 24 14 Feni 4 28 16 Total 190 807 843 Source: SDLG database Based on KIIs and FGDs, the evaluation team did not find a difference in performance between the partner UPs from the base period and the new UPs. For example, as Table 9 shows, the UPs in Rajbari district met on average more frequently and conducted more monitoring exercises than other UPs. Fieldwork in Rajbari showed a similar degree of interest and commitment from project stakeholders as in other districts visited; the PNGO reported that there were no major issues owing to the later incorporation of these five UPs into the project, other than the need to provide the respective SCs with more training on UP governance that had been imparted to the SCs in base period UPs, who in any case received a refresher training at the start of the extension year activities. The evaluation team’s survey contained several questions about the activities of the SCs. Of the 370 respondents in the sample, 215 reported having participated in activities with SCs in the previous 12 months; 178 of the 215 respondents (82.8 percent) who answered positively also reported that they had been appointed as formal members of the respective SC. This confirms the finding above in the discussion of the role of CIG groups that while SDLG endeavored to have CIG members appointed to the SCs, some CIG members worked with the SCs on an informal basis, accompanying them on their service monitoring visits. The survey results support SDLG project data regarding the frequency of SC meetings. As Table 10 shows, almost three-quarters of respondents said that the SCs meet every three months or twice per year. Only 15.8 percent reported that their SCs meet monthly as stipulated by the UP Law. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 16 Table 10: Reported Frequency of SC Meetings in UPs Frequency of Meetings No. Percent Every few weeks 2 .9 Monthly 34 15.8 Every two months 2 .9 Once in 3 months 118 54.9 Twice a year 37 17.2 Once a year or less 6 2.8 Never 3 1.4 Don’t know 13 6.0 Total 215 100.0 Source: BDGPE, SDLG Extension Evaluation Survey When asked about the sector focus of the SCs’ activities, survey respondents gave a wide array of examples, reporting that the SCs looked at tax collection and assessment, health services, upazila clinics, school management, school repairs and furniture, agricultural services, basic sanitation, maintenance of tube wells, selection of beneficiaries of social safety net programs, family planning, prevention of child marriage, protection of women and children, and family dispute resolution. Of the 215 SC members surveyed, 95 respondents could give a concrete example of SC activities. Overall, the responses on this question correspond closely to the sector focus of SDLG’s work with the four targeted UP committees: (i) Agriculture, Fisheries, and Livestock; (ii) Education, Health, and Family Planning; (iii) Women and Children’s Welfare, and Resolution of Family Disputes; and (iv) Audit and Accounts. With respect to the organization of the SCs’ service monitoring activities, the evaluation team’s survey shows that most of the respondents (78.6 percent) said that the committee meetings were organized by the SC president, the UP chair, or another councilor. FGDs, however, suggest that in all of the SDLG project sites visited, monitoring was facilitated and accompanied by field staff of the SDLG PNGO. For the evaluation team, one of the most challenging aspects of fieldwork was to understand the eventual consequences of the monitoring conducted by the SCs. The results of the FGDs were not conclusive. Most FGD participants said that the monitoring reports were given to the UP chairs; a few people mentioned the Union Development Coordination Committee (UDCC) or said that the reports were given to the service agencies directly. However, the majority of FGD participants also said that they did not know what happened once a monitoring report had been prepared and submitted—i.e., what action was taken as a result. Only a few FGD participants were able to point to concrete actions, such as the purchase of furniture for schools, which had been identified as a need during a SC monitoring visit. The evaluation team’s survey findings about the results of the service monitoring conducted by the SCs are more helpful than the FGD findings. Survey respondents gave more than 300 examples. Some of the responses were too general to be useful: “better education” or “health sector development.” As Table 11 shows, there were many examples given of service improvements as a result of monitoring by SCs, including: purchase of furniture and sports equipment for schools; building or repairing school facilities; tube wells repaired; and improvement in selection of social safety net beneficiaries. These are not to be interpreted as 20 cases of “building or repairing school facilities”; rather they are mentions, and there could be some overlap, i.e., several people referring to the same case. Nevertheless, the survey clearly shows there were some concrete improvements in services as a result of SC monitoring. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 17 Table 11: Examples of Service Improvements as a Result of SC Monitoring Improvement in Services No. Percentage Control of child marriage 37 12.3 Building or repairing school facilities 20 6.7 Building or repairing basic sanitation services 16 5.3 Selection of beneficiaries of social safety net 15 5 Prevention of violence against women and children 14 4.7 Control of student truancy and teacher attendance 12 4 Collection of solid waste 12 4 Purchase of school furniture and equipment 11 3.7 Organization of tree plantation on public land 8 2.7 Water treatment plant repaired 7 2.3 No impact of monitoring on services 18 6.0 Other minor mentions 130 43.3 Source: BDGPE, SDLG Extension Evaluation Survey In that there was no baseline or control group for the evaluation team’s survey, it is impossible to know whether other UPs have experienced similar results from the activities of SCs. However, the KAP survey conducted by SDLG does shed some light on this question. Table 12: Perceived Improvement in Health Service Delivery (SDLG/Non-SDLG UPs) Have you and/or your family member/neighbor/relatives noticed any change in service delivery in the last six months? Non-SDLG (Percent) SDLG (Percent) Don't know 40.6 10 Others 8.2 15.3 Cleanliness of clinic improved 16.4 63.6 Doctor/FWV were available during office hour 14.5 37.0 Displayed Service charter 1.4 29.7 Enough medicine available 6.8 19.5 Separate sitting arrangement available for women 4.3 17.2 No improvement notice 29.0 8.2 Source: SDLG, KAP Survey 2014 The findings of the evaluation team’s survey and FGDs with respect to service improvements are confirmed by SDLG’s own KAP survey. Table 12 shows that compared to non-SDLG UPs (control), a higher proportion of respondents in SDLG partner UPs (treatment) perceived improvements in health service delivery. While some of this might be due to response bias (about 93 percent of the respondents in the SDLG partner UPs were direct project participants through the training to UPs, which could create a “halo effect”), overall, the differences are large. Table 13: Perceived Improvement in Agricultural Service Delivery (SDLG/Non-SDLG UPs) Have you and/or your family member/neighbor/relatives noticed any change in service delivery in the last six months? Non-SDLG (percent) SDLG (percent) Others 3.7 17.7 Availability of high quality seeds 44.4 62.8 Availability of appropriate fertilizer 85.2 59.6 Harvesting planning (harvest, storage, marketing) 14.8 34.8 Source: SDLG, KAP Survey 2014 Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 18 Similar results are found in the perception of improvements in agricultural extension—with 59 percent of respondents in SDLG partner UPs saying they saw an improvement, versus 13 percent in non-SDLG UPs. SDLG’S M&E DATA ON SC SERVICE MONITORING In discussing the effectiveness of SDLG’s attempt to link local governance interventions with governance in service delivery, it is useful to review briefly SDLG’s use of performance indicators and to comment on their relevance for answering the question about the effectiveness of SDLG’s interventions in support of the SCs. As discussed above, SDLG supported four SCs in each partner UP to carry out monitoring of local government services. The evaluation team reviewed the M&E methodology, indicator data sheet definitions, and data collection methods, along with the targets and end-of-project achievements for SDLG M&E indicators measuring the project’s results with respect to service delivery monitoring and improvement. The selected indicators and their values are presented in Annex VIII. Overall, these figures show that SDLG has met its targets for indicators measuring the effectiveness of the project’s work with other sectors. The different indicators show that the SCs in fact monitored the service providers and made recommendations for improvement. Indeed, they show that the project made significant progress in very short time in mobilizing the four SCs in 200 partner local governments to carry out service monitoring. The evaluation team, however, has concerns about the design and definition of two of the indicators: • 2.5. Percentage (%) of community served by clinics supported by groups of mobilized local influential stakeholders. The indicator value shows that government clinics in the SDLG partner UPs are attending to 36 percent of the estimated service area target population. Major methodological issues emerge from the evaluation team’s review of technical data (indicator sheets) and KIIs with staff of other sector programs, in that it is difficult to attribute any change to SDLG; and in the absence of a baseline and control group, it is hard to know how to interpret this figure. • 2.9. Number (#) of institutions with improved capacity to address climate change issues as a result of USG assistance. From the technical review of the data sheets and KIIs with SDLG and PNGOs, it is evident that what this indicator measures is limited to “training of SCs” rather than what happened after the training in terms of the SCs’ service monitoring or service improvement. SDLG did not do any measurement of the SCs’ subsequent capacity to address climate change. In this sense, the indicator is misleading. The other indicators in the M&E reports are reasonable measures of SDLG effectiveness, and they are attributable to SDLG’s own interventions. The indicator data show that there were improvements in clinic management in 189 UPs, and that there were improvements made in extension activities in 118 UPs. In the M&E reporting, SDLG did not provide details on the nature of these improvements. However, KAP survey respondents mentioned: “cleanliness of clinic improved”; “Doctor/FWV available during office hour”; “Displayed Service charter”; “Enough medicine available”; and “Separate sitting arrangement available for women” (see Table 12). Similarly, there were 173 UPs where SCs were able to prevent incidents of violence against women, early marriage, dowry, and trafficking in persons. The evaluation team found that the indicator measures were provided by the PNGOs, on the basis of reviewing the SC monitoring reports. In some cases, the SCs made numerous monitoring visits and reports. According to one of SDLG’s field research papers contracted to Unnayan Samannay, one of the SCs made recommendations on 15 Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 19 different services issues and resolved five of them in the timeframe of three months.13 It is important to note that the paper was finalized in October 2014, only six months after the start of SDLG extension year activities, and probably three to four months after the SC had received its training. The reported results are very positive overall. Unfortunately, SDLG did not conduct further monitoring and documentation of results to see whether the initial momentum had been sustained. Also, as the PNGOs ceased to provide support and facilitation to the SCs after October 2014, it was not possible to verify the degree of implementation of these hundreds of monitoring reports. It is possible that the results of the SCs’ monitoring were even better than what SDLG reported. In the case of women’s rights, the indicator is measuring the success of the SC itself in identifying and preventing violations of women’s rights. In fact, these SCs build on considerable efforts of SDLG in the base period, in which the project involved CIG groups and female UP councilors in conducting what might be seen as community social work in the wards, e.g., identifying cases of domestic violence and taking action to prevent new incidents, or discovering preparations for a child marriage and preventing its occurrence. In this respect, SDLG’s efforts have been particularly successful. Although it might be a misnomer to call it “service monitoring,” there is great potential for involving SCs in this sector. The Unnayan Samannay paper gives some specific examples of the role of SCs in this field. CONCLUSIONS: QUESTION 1 The findings from the FDGs and KIIs with SDLG, PNGOs, and other USAID sector program managers and staff coincide with the data collected through the evaluation team’s survey and SDLG’s KAP survey. Overall, the SDLG project successfully addressed governance issues in other sectors, particularly in health, agriculture, and women’s rights—and to a more limited extent climate change. SDLG’s greatest success has been the mobilization of the CIG group members to become formal members of UP SCs and to strengthen their capacity to carry out monitoring of service delivery. The performance evaluation of the SDLG base period identified the CIG groups as the most promising aspect of the project, concluding that they constituted “SDLG’s most outstanding achievement and a noteworthy innovation in the field of local governance in Bangladesh.”14 While the work with UP SCs was still incipient in the base period, having worked for more than three years with the CIG groups allowed SDLG to activate the dormant SCs in the selected UPs in only a few months and engage them in monitoring of UP/upazila service providers. In most of the four targeted SCs in the partner UPs, SDLG was able to ensure appointment of CIG group members who were instrumental in putting the SCs to work. In this sense, the SDLG project’s extension year gave USAID/Bangladesh considerable value for money, building on the previous investment in CIG members’ capacity. However, it should also be highlighted that SDLG was able to effectively organize SCs in UPs that had not participated in the base period activities, and these UPs have not been shown to be worse performers. The decision to suspend all SDLG project field activities in October 2014, by closing out the sub-grants six months before the project close-out, effectively forestalled further support to the CIG groups and made it difficult for SDLG to carry out monitoring and follow-up of the SCs’ recommendations. While the SDLG implementing partner explained that it needed six months to close out the project, the evaluation team believes that more attention to project M&E from October through December (for example, using short-term technical assistance to hire field monitors) would have been helpful in understanding the kinds of improvements in services that were made as a result of the SCs’ monitoring reports. Despite these difficulties, the evaluation team concludes from its various data sources that the 13 M. Kabir A.R. Khan, R.S. Guda, 2014. “Improving Services: The Role of Union Parishad Standing Committees”. Dhaka: SDLG/Unnayan Shamannay. 14 See H. Blair, W. Cartier, A. Hussain, and N. Mostofa. 2014. Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project: Macro level Advocacy and Local level Participation. Dhaka: USAID/Social Impact Inc., p. 32. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 20 SDLG project produced visible, immediate results in the improvement of local government services in other USAID programming sectors. In summary, during implementation of the extension year activities, SDLG faced two major challenges: 1) ensuring that there was effective follow-up on the SCs’ recommendations for service improvements; and 2) documenting those improvements. The short time available (six months) for project implementation in the field probably was a major factor in this respect. With a longer project performance period, SDLG would have been able to better address these challenges. 2. How far has SDLG been able to ensure that the linkages it has created with the other USAID sectors are sustainable? What follow-up work needs to be done? LINKAGES WITH USAID PROJECTS/SECTORS In response to the extension period SOW, SDLG designed a strategy to promote cooperation with other USAID projects working on health and family planning, agriculture extension, human rights, environment, and livelihoods. The first step was to identify where the original SDLG working areas— made up of 450 UPs and 50 PSs—coincided with the sector areas of the other USAID programs under consideration. The review conducted by SDLG at the start of the extension period showed that there were in principle 345 LGUs in which SDLG could promote integration of activities with other USAID projects, whether in other sectors or within the DRG sector itself. Table 14: USAID Project Overlap with SDLG Sector USAID Sector Projects Implementer Overlap with SDLG LGUs Health/Family Planning NGO Health Service Delivery Project (NHSDP) Pathfinder International 175 Family Planning Mayer Hashi II Engender Health 67 Agriculture Extension USAID Agriculture Extension Project Dhaka Ahsania Mission 53 Climate Change Climate Resilient Environments and Livelihoods (CREL) Winrock International 22 Human Rights Actions for Combating Trafficking in Persons (ACT) Winrock International 22 Human Rights Protecting Human Rights (PHR) Plan International 6 Source: SDLG Quarterly Progress Reports As Table 14 shows, among the other sector projects, the NGO Health Service Delivery Project (NHSDP), branded as “Smiling Sun,” overlapped with 175 of SDLG’s LGUs; followed by the family planning project, Mayer Hashi II; then USAID Agricultural Extension project; and Climate Resilient Environments and Livelihoods (CREL). There was also some overlap with two other DRG projects: Actions for Combating Trafficking in Persons (ACT), with 22 overlapping LGUs; and PHR with six LGUs. Following the extension SOW, SDLG selected 190 UPs and 10 PSs, with the principle selection criterion being the overlap with other ongoing non-DRG sector projects and DRG sector projects. The next step was to establish a formal mechanism for cooperation with the other USAID sector programs: signing a Memorandum of Understanding (MOU) between SDLG and the other sector project was seen by SDLG management as the point of departure for the proposed integration of activities. During the extension period, as shown in Table 15, SDLG signed MOUs with four USAID projects: two in the health sector, and two in the DRG sector. KIIs with SDLG and implementing partners of the other sector projects found that while it was not possible to reach an agreement on MOUs with the Agricultural Extension project and the CREL project, there was a degree of cooperation in the field with the former. KIIs with SDLG management and the management of other sector projects Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 21 led the team to conclude that the process of negotiation was too protracted for SDLG’s short performance period; at some point, SDLG opted instead to focus on trying to identify concrete integration activities in the field. Table 15: Status of MOUs with Sector Projects Sector Projects * MOU NHSDP MOU signed Mayer Hashi – II MOU signed Agricultural Extension None Planned CREL None Planned ACT MOU signed PHR MOU signed Source: SDLG Project Documents From the perspective of the other implementing partners, while they appreciated the objective of achieving greater sector integration (between DRG and other sectors, and within the DRG sector), they believed that any formal understanding had to be mutually beneficial for both projects. As the Mayer Hashi II project put it, “we were looking for some projects who maintained good relationships with local government bodies. When SDLG proposed for such collaboration we jumped in. We would have extended our collaboration even had there not been an instruction from our donor.” Smiling Sun project management also stated that it saw a clear benefit in SDLG’s proposed MOU. The evaluation team’s findings are mixed with respect to SDLG’s implementation of the MOUs. Early on, SDLG organized orientation and planning workshops with the management of the other implementing partners in Dhaka. These meetings produced recommendations on concrete activities for inter-sector (and intra-sector) integration. According to Mayer Hashi II, these workshops were useful in that they were reflected in field activities further on. The other implementing partners were less positive, stating that after a rapid start SDLG was unable to maintain the impetus, reflected in an inability to identify and implement specific activities in the field, beyond orientation of project staff on local governance. Several key informants mentioned that soon after the training, SDLG was already planning its project close-out given the short extension period, and could not follow on from the training. Orientation activities in the field mainly consisted of inviting the staff of the other USAID implementing partners to attend meetings with SDLG project counterparts (principally CIG groups and SCs) and on some occasions to join the training organized by SDLG’s own PNGOs for the CIG and SC members. Not all of the staff of the different sector projects understood that this was part of a more general MOU between their respective organizations and SDLG. KIIs with staff of the Smiling Sun project in Jessore, for example, showed that there was low awareness or understanding of the MOUs and the subsequent planning meetings with their head office. Most of the sector project field staff said that they had received instructions from Dhaka project offices to cooperate with SDLG, mainly by attending training and coordination meetings. The Smiling Sun staff reported to the evaluation team that they understood the instructions to meet with SDLG as part of their overarching “development commitment.” Nevertheless, they also stated that as Smiling Sun was planning to extend services from its existing urban network to rural areas, it was essential to establish working relations with local governments. For this reason, they considered the cooperation with SDLG at the field level as an opportunity to build long-term relationships with UP chairs and councilors. The managers and staff of other USAID sector projects told the evaluation team that agreeing to the proposed cooperation with SDLG did not require any adjustments to their respective TOs, work plans, or budget allocations. Thus, from the perspective of project management, the process of integration was seen as hurried and somewhat ad hoc; from the perspective of field staff, the activities were seen as an Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 22 “add on” to their existing work. The field staff of the sector projects mentioned that instructions were sometimes difficult to follow when they were busy with implementing their work plan commitments. The project managers agreed, citing examples of when they were unable to spare staff to attend to SDLG collaboration. Both managers and staff said that inter-sector collaboration should start at very early stages when project budgets are being finalized; alternatively, collaboration should be handled through a more formal process of TO and budget modification. Despite some difficulties communicating the terms of the MOUs to field staff and reorganizing project activities in function of the proposed collaboration with SDLG, KIIs with SDLG and other USAID project staff reflected numerous, specific instances of cooperation in field-level activities: • Integration with NHSDP. This project has a mandate to deliver free or low-cost health services to the poorest strata in its areas. NHSDP staff told the evaluation team that through the cooperation with SDLG they received training on local governance and made contacts with UPs. They were able to receive the lists of poor households drawn up by the UPs. This cooperation allowed them to reduce time and costs in targeting beneficiaries. Also, after joint meetings with NHSDP field staff and the CIG groups in the respective UPs, CIG members referred poor families to Smiling Sun clinics. Staff noted that the cooperation was still incipient at the time SDLG began close out. Smiling Sun had offered to provide SDLG with training materials and formats for SC monitoring of services, but this had not been completed at the time of the evaluation data collection. • Integration with Mayer Hashi II. This project is aimed at disseminating family planning methods in Bangladesh, working at the upazila level with the Ministry of Health and Family Planning and the Directorate of Family Planning in particular. Project staff told the evaluation team that they welcomed the initiative by SDLG. They provided technical support for the design of the SDLG training curriculum and the drama performed by CIG members, as well as designed sections of the monitoring formats developed for SCs. Mayer Hashi II staff also assisted SDLG during the ToT courses for SDLG’s PNGO staff. When asked about the benefits of collaboration with SDLG, Mayer Hashi II mentioned: accountability at the local level, judicious use of resources by reducing overlap, improved performance, and institutionalization of family planning services. While the project did not collect statistics, Mayer Hashi II contended that collaboration with SDLG increased elected representatives’ support for family planning clinics and, thus, increased patient inflow. • Integration with USAID Agricultural Extension Project. Implemented by Dhaka Ahsania Mission and Care as technical partner, this project did not sign an MOU with SDLG. Yet, the two projects engaged in collaborative activities that were arranged at the field level between program coordinators, who kept each other informed about ongoing training and other field activities that offered possibilities for participation of CIG and SC members. For example, the Agricultural Extension field staff reported having attended training in Rajbari. Another activity highlighted by the project management was the role of CIG groups supported by SDLG in promoting the formation of farmers’ groups. The project works together with the government agriculture extension services in the upazila to ensure services for the farmers, through coordination with the Upazila Agriculture Extension Officer and Sub-assistant Agriculture Officer based in the UP. The project required formation of a farmers’ group in each UP to distribute agricultural inputs, as well as share information on farming technology and markets. In principle, this might also create a role for the SC on Agriculture, Fisheries, and Livestock; however, the Agricultural Extension staff observed that the SCs were subject to capture by rural elites and were not always an appropriate partner for supporting small farmers, in that the elite groups tried to monopolize access to subsidized agricultural inputs. However, they were positive about the new relations with local stakeholders cultivated through the CIG groups. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 23 • Integration with CREL. Although CREL had not signed an MOU with SDLG, the projects did work together to conduct field activities in a limited number of UPs. The coordination occurred through interactions between field staff in the respective districts, who looked for opportunities to involve the CIG and SC members in CREL activities. Their joint work can be described as an active learning experience where CREL learned from SDLG about local government strengthening, as an input for scoping and designing future CREL activities. CREL staff told the evaluation team that they learned from SDLG by visiting project sites, participating in meetings with CIG groups and SCs, and attending training. CREL formed groups called Village Conservation Forest; following SDLG’s lead, the CREL group members have attended the ward meetings organized by the CIG groups and convinced residents to work with them on a project to prevent river erosion. • Integration with other DRG projects. The PHR and ACT projects signed MOUs with SDLG to promote integration of activities. Staff from both projects told the evaluation team that they attended joint meetings in Dhaka and contributed materials and staff during the ToT courses for the SDLG PNGOs on how to support SCs in monitoring of services. SDLG quarterly progress reports also mention that issues like domestic violence and trafficking in persons were covered during the training sessions. All of the sector project staff interviewed told the evaluation team that they appreciated SDLG’s approach and thought that, in the long term, it would be beneficial for their sector projects to incorporate local governance mechanisms into their implementation approaches. At the same time, they admitted that SDLG’s activities were still incipient at the time of project close-out and that it would have required several more years of effort to understand the full potential of sector integration with a local governance project. One of the program directors also observed that it was not necessary to partner with SDLG in order to achieve integration—by understanding the opportunities and with appropriate technical inputs into program design, any of the sector projects could build local governance into their respective activities, perhaps with fewer transaction costs. The evaluation team came to the conclusion that before SDLG approached the other USAID projects, they had been unaware of the recent legislative reform of UPs that had opened up avenues for citizen activism on services through the SCs. Managers of other projects admitted that, prior to the coordination with SDLG, they were skeptical about involving UPs in their activities. LINKAGES WITH OTHER ACTORS Another dimension of sustainability relates to links to the various institutional actors in the local government ambits (UPs, PSs, and upazilas). SDLG was effective in building relationships with the UPs, but less so with respect to the GOB ministries and departments in the upazilas. The evaluation team found that SDLG did not have the time (and, perhaps, the budget resources) to drill down into the recommendations and to work with the service providers to explore options for implementing and sustaining them. In the SDLG extension year work plan, there is mention of “demand-driven facilitation” as Activity 2 under “Task 2 – Enhance LGU-sector Collaboration for Service Delivery.” It is described as follows: “SDLG anticipates supporting a number of cases (minimum of 10) where SCs and public officials directly work to improve service delivery.”15 The last mention of this in the quarterly progress reports states: “Current Status: Completed. This activity provides guidance and support for LGUs as they act on and implement the processes taught in the Activity 1 trainings in the previous quarter.” 15 USAID/Bangladesh. SDLG Quarterly Report No 11 October 2014. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 24 When asked about the exact meaning of this task description in the work plan and about the specific cases, SDLG management told the evaluation team that “all of the facilitation to CIG groups and SCs was demand driven.” The evaluation team considers that this response misses the intent of this particular task—understood as drilling down into specific cases and exploring how recommendations can be effectively implemented and sustained by the service providers, who are managed from the offices of GOB ministries and departments in the upazilas. Nevertheless, on a more positive note, during the extension year, SDLG contracted two field-based research studies through its partner research institute Unnayan Samannay. The results of these studies are relevant to the discussion about the impact and sustainability of SDLG interventions. • Improving Services: The Role of Union Parishad Standing Committees case studies show that 1) SCs interact directly with service providers to persuade them to be more responsive and to take action on specific problems, and 2) SCs have much less interaction with the upazila officers and, as a result, are often unable to take service delivery issues to a higher level for resolution. Under the UP Law, it is the chair who is legally mandated to receive the SC reports and take them to the upazila for action. • Upazila Parishad Governance to Improve Service Delivery Policy, Practice and Model Upazila Functioning - Bhairab Upazila, Kishoreganj shows that bringing the recommendations of SC monitoring to the upazila council is a long process, because the respective UP chair must first present the report to one of the upazila’s internal committees (made up of upazila officers), which meet every three to four months, and then place the report on the agenda of the upazila council which often meets every two months. Even if the recommendations are not delayed in the committees, the entire process can take upwards of six months. • Not surprisingly, therefore, SCs supported by SDLG have tended to prefer the “short route” to accountability, talking directly with service providers. This is more expeditious than the “long route” through upazila committees and upazila councils. The research showed that many of the internal upazila committees functioned poorly, with little commitment from the senior officers. The issues discussed in the upazila councils are often not followed up by internal committees for action in the respective departments. SDLG told the evaluation team that it did not have enough time to apply these research outputs via systematic interventions in the upazilas. Rather, the results were presented and discussed in conferences and workshops with national policymakers and local government officials, accompanied by a series of policy recommendations. No doubt some of the Unnayan Samannay research findings have implications for policy, i.e., reforms to the UP and upazila laws, or the detailed regulatory circulars emitted by Local Government and Rural Development (LGRD), but they also are a practical starting point for intra￾upazila governance and management innovations within the frame of the existing UP and upazila laws. Unfortunately, this was not possible because SDLG was in close-out mode by October 2014; the contracts with PNGOs were ending, and there was no field capacity. With the support of some of the other sector projects, SDLG designed simple monitoring tools to enable the SCs to collect and organize data about services, identify specific problems, and generate specific recommendations. These tools did not incorporate any information routinely collected by GOB offices about the services, which reflects the absence of links to the respective GOB offices in the upazilas. While this could have made the monitoring process more complex, this might have been remedied had there been a more active link to the upazila and frequent interaction between SCs and upazila managers. SDLG staff reiterated to the evaluation team a major constraint: the short time available for implementing the strategy. The evaluation team concurs with SDLG’s view that it would have been very difficult to build these links in only one year. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 25 Finally, having said that SDLG did not link with the GOB upazila offices, it is also true that the SCs supported by SDLG opted to engage directly with the government officers directly responsible for service provision. This occurred despite the legal requirement that directs SCs to submit reports to the SC president, who then directs the report to the UP, UDCC, or upazila committee—to be eventually taken up by the upazila council, if necessary. Indeed, according to SDLG, only two SC reports were ever sent to upazila councils. FGDs with project participants clearly found that the SCs preferred to discuss the reports with relevant officers, who they described as open and willing to listen. The evaluation team’s survey supports this finding, with only 2 of 370 respondents saying that the government officers in the upazila or UP were uncooperative with the monitoring conducted by the SCs. CONCLUSIONS: QUESTION 2 Based on KIIs with the managers and staff of the other USAID sector projects, the evaluation team concludes that SDLG made some limited progress in establishing linkages. Overall, the response from the other projects was adequate, considering the short time available for implementation during the SDLG project extension period. One major issue that emerged from the findings was the absence of formal modifications to the TOs, budgets, and work plans of the other sector projects, which could have allowed them to deploy more resources toward activities conducted in collaboration with SDLG. The abbreviated time and limited resources meant that the achievements in this respect were modest. Nevertheless, all of the project stakeholders agreed that the approach held promise. Regarding other actors, the evaluation team concludes that SDLG did not establish linkages to the upazila offices responsible for management of service delivery, which limited the impact of the SCs’ work. These linkages were impossible for the SCs to develop on their own, in that there are legal restrictions on their interactions with the upazila—their monitoring reports need to take the “long route” to the UP chair, then to either the internal upazila committee and/or the UDCC, and then to the upazila council itself. Upazila senior officers resist direct SC engagement. Not surprisingly, therefore, SDLG only reported two cases of SC recommendations being resolved in the upazila council. It seemed that the SCs preferred to try to address problems directly with the service providers. The evaluation team mentions these two different kinds of linkages because they point to a problem and a possible solution. The SDLG extension year activities were characterized by weak engagement with the GOB—this issue was also flagged in the performance evaluation of the SDLG base period, which noted that there was no uptake of SDLG interventions into policy and recommended that, in subsequent projects, USAID explore how to work more closely with GOB agencies.16 In principle, this recommendation applies, too, to the extension year activities, which could have benefitted from deeper engagement with the upazila offices in the targeted sectors. In this task, it would have been possible to bring in the expertise of the other USAID sector projects, to assist SDLG in designing and implementing interventions to focus the attention of the upazila offices on the service improvement process that was initiated from the SC monitoring. As for follow-up work, as noted in the response to Evaluation Question 1, the evaluation team considers that the work with the SCs in initiating service improvements was an important success for USAID/Bangladesh DRG programming and ought to be considered as a key component in the design of subsequent programs. Such a program would also look at how to build a linkage with the GOB through the upazila offices, involving senior officers and the Upazila Nirbahi Officer (UNO). Understanding that design and procurement of such a program might take more than one year, USAID might consider a 16 See H. Blair, W. Cartier, A. Hussain, and N. Mostafa. 2014. Performance Evaluation of the Strengthening Democratic Local Governance (SDLG) Project: Macro level Advocacy and Local level Participation. Dhaka: USAID/Social Impact Inc., p. 34. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 26 “bridge” project through a local NGO, to focus on maintaining the work of the SCs in the 190 partner UPs and documenting in detail the successes and challenges in sustaining service improvements. If USAID does not fund a follow-on local governance program, the evaluation team recommends that the other current USAID sector projects incorporate some elements of the SDLG approach and, in particular, continue to support the work of the SCs in the 190 UPs that participated in the SDLG extension year activities. The existence of an “installed capacity” in SDLG’s PNGOs would facilitate this task considerably. The evaluation team concludes that the NGO staff continue to work in the field (in other donor projects). However, the team understands that this option would require TO modifications in the respective sector projects, which may be an insurmountable obstacle. 3. To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh?17 The current system of local government in Bangladesh consists of three different levels of elected councils as summarized below: • As in many other developing countries, directly-elected municipal councils (PSs in Bangladesh) in urban areas have responsibility for provision of a wide range of urban services but suffer from a weak local tax base only partially offset by inter-governmental transfers and low administrative and technical capacity. PSs have their own characteristics and governance challenges, which will not be discussed further here; rather the discussion will focus on rural local governments. • In rural areas, local government has a dual structure. The lowest level is made up of 4,550 directly-elected UPs with almost no significant service functions, miniscule fiscal resources, and scarce administrative capacity except for a UP secretary appointed by the central government— although the UP houses some government officers from the higher level of local government, the UZP. The UPs have a long history dating back to the Colonial era and enjoy strong public support as the lowest level of elected government. • The UPs are within the jurisdiction of the 488 UZPs, which include a directly-elected chair and vice chair, who preside over a council composed of the chairs of the various UPs in the respective upazila. While in principle, the elected UZPs have a critical governance function in overseeing the delivery of a broad range of services, in practice this function is incipient at best. The elected council faces hard legal constraints on its ability to control the decision-making of the upazila administration, which is in fact a collection of disparate offices of numerous government ministries and departments, staffed by civil servants appointed from different national civil service cadres and supervised by a UNO, who is appointed by the central government to act as a kind of Chief Executive Officer (CEO). • UZPs do not plan, budget, or manage the services provided by the upazila/UP offices, whose managers and staff work within policy objectives, priorities, and budgets established by the national levels of their respective ministries and departments. Moreover, the upazila councils do not have effective control of their human resource functions—neither planning, recruitment, appointment, capacity development, promotion, nor dismissal. Thus, the bureaucrats who staff and manage the upazila offices are not in the last instance accountable to elected councils but to the bureaucratic hierarchy within their respective ministries and departments. 17 The USAID Contracting Officer’s Representative clarified that this question was intended to query whether the SDLG project’s objectives provided sufficient guidance for appropriate programmatic and technical assistance decisions. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 27 There are numerous other governance failures in local government in Bangladesh: interference of MPs in decision-making on all aspects of local administration; high levels of corruption in all aspects of local government management; and weak technical capacity in management. However, it is the high degree of autonomy of the upazila offices vis-à-vis the elected upazila and UP councils that calls into question some of the main theoretical arguments in favor of decentralization—that greater accountability leads to increased responsiveness and higher efficiency in service provision. Absent this accountability, local governments in Bangladesh experience numerous governance and management failures: inattention of upazila planners to local infrastructure needs (e.g., repairing a bridge or a damn); unwillingness of upazila staff to visit the field to monitor and prevent recurring, common social problems (e.g., child marriage, dowry, violence against women); weak upazila management oversight and discipline of direct service providers (e.g., schools, clinics, health posts), which gives rise to widespread problems of absenteeism and theft of supplies; and recourse to bribery as a means of accessing services that are (or ought to be) free of charge; among other problems.18 During the project extension year, SDLG’s activation and training of the UP SCs was intended to circumvent the high degree of bureaucratic autonomy of the upazila/UP service providers with respect to the elected councils and, in this way, increase local government accountability and responsiveness. As discussed above, the SCs conducted monitoring of services, submitted reports to the UP chair, talked directly to the service providers in the upazila and UP, and in some cases channeled the monitoring reports and recommendations through the upazila bureaucracy (internal committees) and ultimately to the upazila council. It bears repeating that the means for activating the SCs was training of CIG members on different aspects of local governance over more than four years (including the SDLG base period); convincing the partner UPs to appoint CIG members to the SCs; training SCs on how to carry out service monitoring and reporting; and facilitating the initial service monitoring exercises by SCs (the last three interventions during the extension year). Thus, the capacity of SCs came down to the knowledge, capacities, and will of a large group of volunteers working with their local governments to improve services. The evaluation team found that many presidents of the CIG groups were close political allies of the UP chair or municipal mayor, always belonging to the same political party. In one case, the CIG group president was the elder brother of the UP chair; in another case, the CIG president was the UP party committee president, and the UP chair was the general secretary of the upazila committee of the same political party. In other cases, the team found that the CIG group members were related to the CIG group president, the UP chair, or the municipal mayor. FGD participants recognized that politics played a role in the selection process for CIG groups and SCs; the SDLG PNGOs had to navigate the political currents in each partner UP/PS. FGD participants mentioned the role of the UP chairs and councilors in selecting the original CIG group members. SDLG management is aware of the problem but did not propose a solution in response to the evaluation team’s query. The team’s KIIs with SDLG sub￾contractors reveal that the PNGOs have to negotiate their activities with local political elites, who frequently insist that their political supporters are included in project activities.19 It would be surprising if partisan politics did not play a role in SDLG activities in the UPs and PSs. Local government in Bangladesh, despite its avowedly non-party electoral process, is as partisan as national 18 See the various chapters in a recent, comprehensive study of local government in Bangladesh. Abul Barkat et al. 2015. Local Governance and Decentralization in Bangladesh. Dhaka: Pathak Samabesh, Swiss Agency for Development Cooperation. 19 The evaluation team notes that this challenge has been found in other BDGPE evaluations of DRG projects in Bangladesh. It was a salient feature in the selection of some of the participants in the Leadership Development Program (LDP). See W. J. Cartier, A.K.M Saifullah, K. Islam, S. Majumder, and N. Mostafa. 2015. Performance Evaluation of USAID/Bangladesh Leadership Development Program (LDP). Social Impact/USAID. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 28 politics. All of the major political parties have party committees in the different levels of local government. Nevertheless, whatever their political leanings, CIG group and SC members are also citizens, many of them concerned enough about local issues to dedicate significant time and energy to a volunteer activity that does not give them tangible benefits. This is seen clearly in SDLG’s KAP survey results, which suggest that the project was very successful in strengthening respondents’ knowledge of local governance processes; changing attitudes toward local government; and fomenting citizen participation in local decision-making. In Table 16, there is an evident difference between respondents from SDLG partner UPs and non-SDLG UPs regarding: knowledge of the Citizen Charter explaining upazila service standards; the use of village courts to resolve problems; knowledge of open budget provisions; own participation in UP planning; and understanding of the importance of UP and upazilas having their own tax base (own revenues) to be able to attend to the needs of poor (“mass”) people. Whatever their political affiliations, SDLG participants have clearly excelled in the local “school of government.” Table 16: Differences between SDLG and non-SDLG UPs on KAP in Local Government Knowledge/Attitude/Practice (KAP) Topics Non-SDLG (Percent) SDLG (Percent) Knowledge of Citizen Charter 25.0 92.7 Use of village court to resolve problems 22.6 40.3 Participate in ward shavas 39.7 73.5 Knowledge of provision for open budget process 28.5 88.5 Own role in assisting the UP in identifying projects (schemes) 29.4 67.9 Understand importance of own revenues for fulfilling needs of “mass people” 30.4 62.7 Source: SDLG Project Documents Finally, SDLG’s implementation approach relied upon communications strategies taken from traditional folk drama. This approach had been used in previous local governance programs in Bangladesh, as well as in numerous communications and behavior change strategies of other sector projects in areas such as reproductive health and human trafficking. SDLG reports positively on the results of this strategy—for example, stating that it has reached 52,779 people in its folk dramas on climate change (277 people in each performance). This is only 1.8 percent of the adult population of the 190 partner UPs. CONCLUSIONS: QUESTION 3 Over the past 30 years, the main political actors at the national level in Bangladesh have resisted pressure for decentralization. Successive donor programs in support of advocacy initiatives have not been all that successful in convincing national political actors to adopt more far-reaching reforms of local governments. Introducing the important concept of subsidiarity in service provision is still a long way away. MPs, in particular, are loath to give up control over local investment decisions.20 In fact, the two major reforms of rural local governments since Independence occurred under non-elected governments (the Ershad military government in 1984 and the military-backed Caretaker government in 2007). In this sense, the evaluation team considers that SDLG proposed a realistic, incremental strategy to reinforce the accountability of local government in a context in which there will probably not be major shifts in national government policies in the short term. MPs, in particular, continue to resist calls for greater autonomy of local bodies. The officers from national civil service cadres in the upazilas continue 20 See H. Zillur Rahman. 2015. “Strategy on Local government Strengthening: Background paper for 7th Five Year Plan”. Dhaka: Power and Participation Research Centre (PPRC). Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 29 to resist encroachment on their power, opposing proposals to give greater autonomy to local governments. In this context, absent an attitudinal shift among national policymakers, donor programs for strengthening local government in Bangladesh have few strategic options except to focus on reinforcing citizen engagement as a way of holding local decision-makers accountable. As discussed in the response to Evaluation Question 1, SDLG has been very successful in this regard. Despite the limits to decentralization reform in the short term, there will eventually be reforms— managing local services from the center in a country of 160 million people is not a viable institutional arrangement. Thus, SDLG’s approach is an appropriate strategy in anticipation of more fundamental reforms of local government. Other countries’ experiences with decentralization reform offer plenty of cautionary tales with respect to issues like local elite capture, increased corruption, inattention to service provision in favor of infrastructure investment, etc. In other words, decentralization has sometimes accentuated existing governance failures. By reinforcing citizen knowledge, understanding, attitudes, and practices in favor of accountable local government and responsive service provision, SDLG’s approach can help to mitigate some of the dangers of future decentralization reform initiatives. While it might seem like a minor point, the data on SDLG’s use of folk drama as a communications medium suggest that—without denying the aesthetic attraction of the folk drama—it may not represent good value for money for USAID. With limited project resources for activities spread over a relatively large number of partner LGUs, there may be more cost-effective ways of reaching stakeholders and the wider community. 4. To what extent were gender and youth effectively addressed by SDLG’s interventions in the targeted areas? SDLG ACTIVITIES WITH WOMEN While female UP council members have made great strides in recent years in taking up leadership roles in local government, women still face considerable barriers to participation in politics and civil society, particularly in rural areas.21 Despite these obstacles, overall, the evaluation team found that SDLG made an effort to involve women in the project activities, achieving a relatively high rate of female participation. Of a total of 4,269 people trained during the extension year, 3,309 (77.5 percent) were male and 960 (22.5 percent) were female. Looking at SDLG’s efforts to promote the appointment of CIG group members to SCs, the data are more promising: of 1,718 people participating in SCs during the extension year period, 1,086 (63.2 percent) were male and 601 (36.8 percent) were female. Beyond the proportion of women involved in training, there are other data that show effective involvement of women in activities. Table 17 presents answers of male and female respondents to two questions in the evaluation team’s survey of SDLG participants. There is only a small difference between men and women to the question of whether they had participated in the most recent ward shava— indeed, a slightly larger proportion of men reported that they had not participated. To the question as to whether respondents knew if there had been an improvement in services as a result of the SC monitoring, 28.6 percent of women said that they did not know compared to 25.6 percent of men. These results are consistent with the results of the FGDs with project participants in the various SDLG project sites visited by the team, which highlighted the role of women in the CIG groups and in the activation and monitoring activities of the SCs. These results suggest that SDLG was equally successful not just in training women for participation in CIG groups and SCs, but also in promoting their 21 See Asian Development Bank (ADB). 2004. Gender and Governance Issues in Local Government: Regional Report of Technical Assistance in Bangladesh, Nepal and Pakistan, Manila ADB. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 30 involvement in the activities of the CIG groups (which worked through ward shavas) and their understanding of the results of SC monitoring of services. Table 17: Differences in Male and Female Participation and Knowledge Male (Percent) Female (Percent) Didn’t participate in last ward shava 44.3 40.7 Don’t know if services improved 25.6 28.4 Source: BDGPE Survey, 2014 FGD participants offered numerous comments about the important role of female UP councilors in SDLG project activities, and about how involvement in the project reinforced women’s standing in politics. The evaluation team heard that during the SC activation stage, SDLG promoted female UP councilors as SC presidents in their respective unions. Generally, there are only three female councilors (i.e., those elected to the reserved seats); given that the UP Law prohibits a councilor from being president of more than one SC, the female councilors can occupy the presidency of up to three of the 13 legally-mandated SCs in the union. SDLG only worked with four SCs in each UP; thus, in principle the female councilors in the SDLG partner UPs would be able to occupy the presidency of three out of four of these targeted SCs. According to the SDLG quarterly progress reports, 78 percent of the targeted SCs in fact had female presidents.22 In other words, SDLG’s efforts resulted in the appointment of every female council member in the partner local governments to the presidency of one of the targeted SCs. This is a notable success. In the responses to Evaluation Question 1, the team noted that the Women and Children’s Welfare, and Resolution of Family Disputes SCs were particularly effective, not so much in service monitoring, but rather in directly providing a service to the community. Through outreach and communication at the ward level, and supported by the CIG groups, these SCs worked to identify and prevent incidents of domestic violence, child marriage, and dowry. In the FGDs, there were many mentions of the successes of this particular committee. While these results undoubtedly reflect SDLG’s work in this sector during the project base period, the evaluation team considers that the results achieved with the SCs during the extension were exceptional. SDLG ACTIVITIES WITH YOUTH The evaluation team’s survey, based on a random sample of SDLG participants (members of CIG groups and SCs), shows that 27.4 percent of respondents who reported being a member of a CIG group were under 35 years old—i.e., just more than one-quarter of the universe of SDLG project participants were in the GOB “youth” category. Of these youth participants of the CIG groups created during the SDLG base period, 88.2 percent went on to become members of SCs, as compared to 81.1 percent of the adult CIG group members over 35 years old. In this sense, SDLG appears to be successful in retaining youth participants for the extension year project activities. Table 18: Differences in Youth and Adult Participation and Knowledge Adult (Percent) Youth (Percent) Didn’t participate in last ward shava 41.6 47.4 22 The 78 percent figure (above the maximum consistent with the UP Law) might be due to non-compliance of some UPs with the provisions of the Law by appointing female Councilors to the presidency of more than one SC, or it may be a problem of reporting. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 31 Adult (Percent) Youth (Percent) Don’t know if services improved 24.5 31.7 Source: BDGPE Survey, 2014. Compared to adult SDLG participants, a somewhat lower proportion of youth CIG members reported having participated in the most recent ward shava meeting. Similarly, compared to adults, a larger proportion of youth participants in the SCs reported that they do not know whether there were service improvements after the SC monitoring was completed. However, in neither question are there very large differences in the responses that might indicate that the youth participants faced higher barriers to participation. However, unlike the activities that focused on issues affecting women, the SDLG extension year work plan did not emphasize youth issues, and the SCs supported by SDLG did not appear to target, for example, issues relating to culture, sports, or recreation. Nor did the SDLG quarterly progress reports disaggregate the M&E data by age. CONCLUSIONS: QUESTION 4 Based upon its findings, the evaluation team concludes that SDLG was very effective in engaging women in project activities, in particular promoting female councilors as presidents of SCs. The results of SDLG efforts in the high levels of activity of the respective SCs were especially notable. These results highlight the critical activist role of many female UP councilors—often in stark contrast to the passivity of the male councilors. These results also underline the need to build in a strong role for female councilors in subsequent local governance programs—not only in relation to gender issues, but also in relation to promoting effective oversight by SCs in all of the service areas of the UP and the upazila. To the extent that other USAID sector projects consider working with UPs and their SCs, these results should be communicated to the projects as a critical success factor in any UP/upazila service monitoring activity. The youth dimension of SDLG was attenuated, to say the least. While SDLG was able to recruit and retain youth leaders in its work with CIG groups and SCs, this did not translate into monitoring activities in function of UP/upazila services. Of course, this could reflect the low priority of youth issues within the UP/upazila administration and within each of the GOB central ministries and departments represented there. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 32 RECOMMENDATIONS 1. USAID should clarify what it means exactly by sector linkages. For future local governance projects with a sector integration focus, the detailed program design must take into account the difference between “sector linkages between projects” and “sector linkages of projects with local government.” SDLG extension activities combined both of these elements, and at times they were not clearly differentiated. This evaluation suggests that the latter definition of sector linkages should be the main program objective; the former, through collaboration between different USAID projects, is just one means of getting to that objective. 2. USAID should fund a follow-on program to build upon SDLG’s achievements. The success of SDLG’s extension year in support of service monitoring and improvement through the mobilization of an active citizenry in the functioning of UP SCs is a notable achievement for USAID/Bangladesh. For this reason, the team recommends that USAID consider a follow-on project to build upon the success of SDLG’s extension year. Such a program should have an exclusive focus on the rural local governments, not in PSs, in part because the Municipal Law does not allow for citizen participation in the PSs’ statutory SCs. Perhaps at a later date, should there be a legislative reform, there may be opportunities to work with PSs. In any case, as mentioned in Recommendation 7 below, including more levels of local government will spread scarce project resources more thinly among the partner local governments. 3. USAID should consider a bridge project. The team recommends that the SDLG process in LGUs be sustained. Understanding that the design and procurement of a follow-on program based on the results of the SDLG extension year might take more than one year to conclude, the evaluation team recommends that USAID consider a “bridge” project within the USAID Forward principles, contracted through a local NGO. The bridge project would focus on maintaining the work of the SCs in the 190 partner UPs and document in detail the successes and challenges of promoting, implementing, and sustaining the service improvements. 4. Follow-on projects should move quickly to establish SCs in partner LGUs. While SDLG’s three-year base period undoubtedly contributed to its ability to generate results in a relatively short extension period, this conclusion must be tempered with the acceptable performance of the UPs that were added to the project at the outset of the extension period. The evaluation team recommends that any follow-on project quickly incorporate new UPs and activate their SCs in a relatively short time (at least, less than the three-year SDLG base period). 5. USAID implementing partners in women’s rights should replicate SDLG’s work with SCs. The evaluation team recommends that the notable success of the Women and Children’s Welfare, and Resolution of Family Disputes SCs in the SDLG extension year be analyzed in detail by USAID’s other DRG projects, specifically those in the fields of women’s rights and anti-trafficking. SDLG’s work with these SCs demonstrates clearly that other projects should immediately build upon the results in any partner UPs that overlap with the SDLG extension year partner UPs, and also try to replicate the results in other (non-SDLG) partner UPs. The existence of significant capacity in SDLG’s former PNGOs could facilitate this effort as a key training resource. 6. Future local governance projects should ensure GOB buy-in and involvement. To ensure sustainability—and promote replicability—the evaluation team recommends that any future free-standing local governance project along the lines of the SDLG extension year have strong GOB buy-in. Such a project could be marketed as a pilot to improve upazila service delivery, anchored in LGRD or in the Prime Minister’s Office, with its implementing Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 33 arrangements aimed at achieving effective collaboration with the UNOs and targeted offices of sector ministries and departments in the upazilas. This would enable the project to work with senior upazila officers to make the service improvement process more expeditious, involving them in designing and overseeing the responses to the SC recommendations. This would give greater replicability and sustainability to the project intervention and increase the possibilities of promoting policy change (even in the form of circulars). 7. USAID should not spread project resources across too many LGUs. One of the success factors in the SDLG extension year was the focus on only 200 LGUs (190 UPs) and on only four of 13 legally-mandated SCs. The evaluation team recommends that future projects resist the temptation to create a much larger “footprint.” Given the large number of LGUs, it is recommended that future projects’ impact model build on the links to the GOB policy process mentioned in Recommendation 6, whether legal reforms (the respective UP and Upazila Laws) or regulatory reforms (circulars issued by GOB ministries and departments). Projects should also resist the temptation to expand to all 13 SCs. 8. Cross-sector integration activities should be formalized by USAID. The evaluation team recommends that cooperation with other USAID sector projects proceed via a formal TO modification or work plan amendment, in order for the other sector projects to be able to contribute human resources. Without such arrangements, cooperation with other projects will always be somewhat ad hoc and intermittent, despite the interest and enthusiasm of respective field staff. Formalization means that USAID needs more lead time in planning sector integration to be able to incorporate activities within ongoing projects and in SOWs for new projects. 9. Project M&E plans should have appropriate measures of service improvements. The evaluation team recommends that any follow-on local governance project working in other service sectors have a rigorous M&E plan and related indicators to measure outcomes in service improvement. The SDLG M&E plan measures mainly outputs, and the outcome indicators relating to service improvement reflect in many cases verbal commitments of the UP and upazila service providers to make changes, which might not yet be reflected in improvements. The argument in favor of sector integration with local governance programs (e.g., in health) is weakened if the service improvements cannot be clearly measured. 10. Where possible, project M&E plans should use data generated by partner GOB offices. The evaluation team recommends that any future project have an M&E plan that uses program data generated by the service monitoring process led by the SCs and with the cooperation of the upazila offices. This is a missing element in SDLG that might bring together the demand and supply elements of a follow-on project (i.e., the demand from SCs for better services, and supply of service improvement from local officials in the UPs and upazilas). The SC monitoring supported by SDLG is based on formats that collect information in situ; the SCs do not access and use other information collected by GOB agencies about the quantity and quality of services provided by the UP or upazila. 11. Future projects should try to use data to compare and benchmark service delivery performance of LGUs. The evaluation team recommends that any future project use information collected by the SCs and by the upazila offices to measure, benchmark, and compare services across UPs/upazilas. Collecting data and reporting on performance in all partner local governments will create awareness and raise expectations about service provision, as well as provide an objective focus for discussions about the performance of individual service units for review by upazila offices and the upazila council. Performance monitoring would also be an important input into UP/upazila investment prioritization and planning. Finally, it would provide inputs into the broader policy discussion about local government reform. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 34 LESSONS LEARNED 1. SDLG’s recent experience with CIG groups suggests that while local governance projects can mobilize large numbers of citizens in volunteer activities to press for more responsive local government, this demand-side strategy has a greater potential when it works through legally￾mandated bodies like the UP SCs. The formal, legal basis for the SCs constitutes a considerable “power resource” when dealing with other institutional actors in local government. The ad hoc, volunteer groups created by donor projects lack that necessary force of law. 2. The experience of SDLG suggests that there are two different strategies for sector integration: one aimed at incorporating “sector content” into the implementation approach of local governance programs; the other aimed at incorporating “local governance content” into the implementation approach of sector projects. For example, a local government program like SDLG working with UPs (and SCs) can focus on how to exercise oversight of clinic performance. Or a health project that supports local clinics might decide to explore how to work with UPs and SCs to this same end. Either seems to hold promise for improving services, by linking project activities to local government actors to generate accountability for service improvement. 3. The results of SDLG’s support for the work of the SCs shows the promise of the demand-side approach in addressing issues of governance in different service sectors. This has immediate implications for other USAID sector projects, whose management and field staff may not even be aware of the 2012 legal reform allowing citizens to participate in the SCs, mainly because the SCs continue to be largely dormant in most non-SDLG UPs throughout Bangladesh. The SCs may have not been on their radar; the sector projects should pay more attention to the changing local institutional context. 4. The results of SDLG’s work with the Women and Children’s Welfare, and Resolution of Family Disputes SCs reinforce the previous lesson that a legally-mandated body has the capacity to compel individuals and families to abide by the law and to respect legal rights of women and children. While SDLG could have documented this achievement in greater detail, the available data suggests that it holds great potential for protecting and promoting human rights in rural communities. 5. The approach developed by SDLG during the extension year period may be very cost effective, as the project worked with 200 LGUs at a total direct cost of approximately USD 4,500 each. This calculation is based on the contracts with the PNGOs that carried out the fieldwork, including many activities outside the support to the SCs whose cost might be less than half the total activity cost, that is, no more than USD 2,000 in each partner local government. For an existing sector project, this could be a low-cost activity to reinforce and multiply project results. 6. Female elected officials in local government are an untapped resource for strengthening local governance in Bangladesh. SDLG’s extension year results show the deep engagement of female UP councilors in their respective wards and broader communities through the CIG groups and SCs—especially, but not exclusively, in those policy and service sectors relating to women and children. This activism on the part of female UP councilors in the SCs offers opportunities for modeling and promoting women’s leadership in local governance more broadly. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 35 ANNEXES Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 36 ANNEX I: EVALUATION SCOPE OF WORK Scope of Work for the Strengthening Democratic Local Governance (SDLG) External Performance Evaluation USAID/Bangladesh Office of Democracy and Governance Program Identification Data Program Title: Strengthening Democratic Local Governance (SDLG) Program Number: EPP-I-00-04-00035-00, Task Order No: AID-388-TO-11-00001 Program Dates: Start Date: December 21, 2010 - End Date: March 29, 2015 Program Funding: $14,728,074 Implementing Organization: Tetra Tech ARD Contracting Officer Representative (COR): Sherina Tabassum I. Background Since independence in 1971 there have been several attempts by successive government to strengthen local government in Bangladesh and deliver services more effectively. Each and every successive government mentioned the importance of developing a sound democracy and increasing people’s participation in the political process, decision-making, and development of the country. Decentralization was viewed as a strategy that would allow democratic governance and encourage people’s participation. But unfortunately there is major disconnection between what is expected of the system and how the system is organized. Every successive government of Bangladesh has used the local government bodies to strengthen its own political base in the rural areas, ignoring the principles and importance of decentralization of power to the local level. In 2009, Bangladesh took some major steps to strengthen the local government structure per the requirement stipulated in the constitution of Bangladesh. Some legal reforms have taken place since 2009 to establish a strong locally-elected government at each administrative level (i.e. Union Parishad, Upazilla Parishad, Municipalities, etc.). With these developments, challenges remain in making local governments effective and active so that they can play a critical role in development. While legal reforms empower local governments to improve service delivery and address local development needs, in practice, often the local governments suffer from lack of adequate resources and have to deal with multiple power positions, including local Member of Parliament and the bureaucracy. More authority, resource flow, autonomy and exercise of balanced power are key areas that need to be addressed for effective and active local governments. USAID, in its endeavors to strengthen local government, focuses in the areas of Policy Reform, Local Government Strengthening, Local Government Association Building & Strengthening, and Participatory Strategic Plan. The previous program in local governance was the National Constituency for Strong Local Government (2001-2005). This was followed by a two-pronged effort to support local governance starting in 2005 through the Democratic Local Governance Program (September 2005 to September 2008) and the Improving Local Level Governance (August 2002 to March 2011). Respectively, the two programs worked on strengthening Union Parishad and creating citizens’ awareness. The Strengthening Democratic Local Governance (SDLG) program builds and expands upon USAID’s local governance work since 2001. The program aims to promote and expand decentralization, and initiate improvements in service delivery by 1) ensuring local government and citizen oversight 2) Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 37 increasing transparent and participatory public administration at the sub-national level; and 3) enhancing citizen and local government participation in ensuring service delivery. During its initial period of implementation, SDLG program primarily focused on creating the tools and practices that enable local governments to become more democratic, gender sensitive, effective, and responsive institutions of governance; during the extension period, the activity’s focus was revised. In the extension period (March 30, 2014-February 28, 2015), the SDLG activity works with local governments and citizens to initiate visible improvements in service delivery and an increase in citizen participation in service delivery in line with local government laws. The program components are: Component 1 – Roles and Authorities of Local Governments, Component 2 – Transparent and Effective Service Delivery by Local Governments, and Component 3 – Citizen Participation in Local Decision-Making The earlier component on Advocacy and Capacity Building of Local Government Associations was not included as a focus during the extension period due to lack of adequate funding. One of the focuses in the extension year was to explore and create linkages with other technical areas such as health, agriculture and climate change. A. COMPONENT 1 – Roles and Authorities of Local Governments The objective of this component is to plan and carry out specific advocacy activities to encourage the national government to adopt legal and policy reforms to expand roles, authorities and resource allocation for local governments so that they can provide better service delivery. For the purposes of this component, the supply side officials should be interpreted to include both locally elected officials and public servants deputed at the local level. Each activity of the program should work simultaneously with its Local Government Unit (LGU) counterparts at the national and local levels, in coordination with citizens. This component will assist LGU officials, public officials and citizens as appropriate, to operate more efficiently and effectively. This component will also include support to citizens and civil society groups that have emerged as champions in identifying and addressing the needs of LGUs and communities to promote decentralization. This component will continue focusing on linkages between and among LGUs to develop partnerships to promote best practices, and working with citizens as customers of the LGUs. B. COMPONENT 2 – Transparent and Effective Service Delivery by Local Governments The objective of building the capacity of the targeted LGUs is to enable them to be effective and transparent service providers at the sub-national level. The principal emphasis of this component will continue to focus on achieving an immediate impact on local government management capacity through on-the-job training and technical assistance. The Contractor should also help identify complementary areas of training and longer-term reforms or institutionalizing best practices that would bring further improvements in service delivery. Emphasis should be given on strengthening institutionalized and legally mandated local government mechanisms (such as UP standing committees) to create an enabling environment for better service delivery. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 38 C. COMPONENT 3 – Citizen Participation in Local Decision-Making The objective of increasing citizen participation in local decision-making is to ensure transparency and accountability in public management that will lead to improved service delivery, as well as citizen participation ensuring the services delivered reflect priorities of the community. Transparency in public management and in the use of public funds is fundamental to citizens’ trust and confidence in their local governments. Transparency can be enhanced by promoting and requiring the use of participatory planning by local governments and an active involvement of the citizenry in the affairs of the elected councils. USAID/Bangladesh local government programs have successfully introduced open budget hearings in all the local governments in which they worked, and SDLG has established Citizens in Governance (CIG) groups in selected local governments that have consistently engaged in local decision-making. The CIGs, a group of local citizens engaged with the local government, works closely with elected officials of local governments and related Standing Committees to ensure better and more equitable service delivery. This has created a high level of transparency and accountability in public management by these local governments. Throughout the SDLG activity, interventions focused on enhancing women local government officials’ leadership skills and their capacities to address issues that affect community development. During the extension period, SDLG worked with especially those standing committees that were led by women. The majority of these committees were on Health, Law and Order and Domestic Violence. There were also a few on Financial Oversight and Management at the local level. II. Objectives of the Evaluation The objective of the performance evaluation is to measure the development outcomes of the extension program with a view to drawing lessons learned for the selection, design, and implementation of future projects. The performance evaluation will also assess the relevance and sustainability of the program outcomes. The evaluation will: • Review SDLG extension program’s overall performance by assessing actual results against targeted results; • Make recommendations to USAID/Bangladesh concerning future programming in the local governance sector. The audience for this evaluation is USAID/Bangladesh, USAID/Washington leaders of USAID Forward, other USAID missions, Tetra Tech ARD, relevant stakeholders such as Bangladesh Union Parishad Forum (BUPF), Municipality Association of Bangladesh (MAB), Elected Local Government, community leaders, community members and other relevant USAID implementing partners and other bi-lateral and multi-lateral donors in Bangladesh. III. Evaluation Questions The evaluation should review, analyze, and evaluate the SDLG extension program by answering the following evaluation questions, and where applicable, identify opportunities and make recommendations for future programming in the local governance sector. In answering these questions, the Evaluation Team should assess both the performance of USAID and that of the implementing partner(s). The evaluation questions, in order of priority are: Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 39 1. How effectively has SDLG been able to integrate local governance issues to successfully address governance challenges in other sectors, such as, health, agriculture and climate change? 2. To what extent has SDLG been able to ensure that the linkages it has created with the other USAID sectors are sustainable? What follow-up work needs to be done? 3. To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh? 4. To what extent were gender and youth effectively addressed by SDLG’s interventions in the targeted areas? IV. Proposed Evaluation Methodology The detailed methodology of this evaluation will be described by the evaluation team in the Work Plan; this will include presentation of an evaluation matrix that will explicitly link evaluation questions and sub￾questions to particular data collection approaches and data sources. In general, the evaluation will apply a mixed-methods approach, with an emphasis on comparative field￾based case studies of Local Government Units. Some quantitative analyses (including non-experimental design) may be featured, for example, to assess SDLG’s overall performance or in the analysis of the program’s efficiency. The qualitative side of the evaluation will be incorporated to address several questions (regarding program relevance, and sustainability, for example). In addition, the field data collection will involve intensive case study visits, organized around a set of semi-structured individual interviews and focus group discussions (FGDs). Individual key-informant interviewees will include: the SDLG contractor, SDLG COR, members of Local Government bodies, staff from Local Government ministries, staff of Donor organizations working with Local Government, local opinion leaders, community members and beneficiaries, etc. The team will welcome suggestions from USAID as well as Tetra Tech ARD and other evaluation stakeholders, for additional data sources at the community level. FGDs will include balanced numbers of men and women; in addition, as appropriate to local circumstances sex- or age-segregated discussion groups will be used to promote free discussion by women, men, and youth. The evaluation team will analyze the information collected to establish credible answers to the questions and provide major trends and issues. USAID requires that evaluations explore issues of gender; thus, the evaluation should examine gender issues within the context of the evaluation of SDLG activities. The evaluation must collect and include gender disaggregated data in the analysis of findings and conclusions and in making recommendations. Methodological limitations and challenges for this evaluation are expected to include: • Ensuring adequate representation of interview and rapid appraisal sources vis-à-vis the full scope of SDLG activities and outcomes; and • Taking systematic actions to counter any biases in (a) reporting by data collection sources and (b) interpretations of collected data by the evaluation team. The methodology narrative should discuss the merits and limitations of the final evaluation methodology. The evaluation team will design appropriate tools for collecting data from various units of analysis. The tools will be shared with USAID during the evaluation and as part of the evaluation report. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 40 The evaluation team will be required to perform evaluation tasks in Dhaka, Bangladesh and also will travel to activity sites within the country, preferably in the Khulna Division. V. Existing Sources of Information USAID/Bangladesh DG Office will provide documents for the desk review. The list of available documents is as follows: 1. SDLG SOW and any relevant modifications 2. SDLG PMP 3. SDLG annual performance reports 4. SDLG quarterly reports 5. ILLG Evaluation report 2012 6. Bangladesh DG Assessment 2009 7. Bangladesh Evaluation of Local Government Activity 2008 8. Bangladesh CDCS 9. Documents from other donors on local governance The list is not exhaustive and the Evaluation Team will be responsible for identifying and reviewing additional materials relevant to the evaluation. The USAID/DG office will also help the Evaluation Team with contact information for relevant interviewees. VI. Deliverables All deliverables are internal to USAID and the Evaluation Team unless otherwise instructed by USAID. Evaluation deliverables include: Evaluation Team Planning Meeting(s) – essential in organizing the team’s efforts. During the meeting(s), the team should review and discuss the SOW in its entirety, clarify team members’ roles and responsibilities, work plan, develop data collection methods and instruments, review and clarify any logistical and administrative procedures for the assignment, and prepare for the in-brief with USAID/Bangladesh. Work Plan – Detailed draft work plan (including task timeline, methodology outlining approach to be used in answering each evaluation question, team responsibilities, and data analysis plan): Within 5 working days after commencement of the evaluation. In-brief Meeting – In-brief with USAID/Bangladesh: Within 2 working days of international team members’ arrival in Bangladesh. Evaluation Design Matrix – A table that lists each evaluation question and the corresponding information sought, information sources, data collection sources, data analysis methods, and limitations. The matrix should be finalized and shared with USAID/Bangladesh before evaluation fieldwork starts. It should also be included as an annex in the evaluation report. Data Collection Instruments – Development and submission of data collection instruments to USAID/Bangladesh during the design phase prior to the commencement of the evaluation fieldwork. The completed evaluation should also include the data collection tools, instruments, and list of people interviewed as an annex in the evaluation report. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 41 Weekly Updates – The Evaluation Team Leader (or his/her delegate) will brief the Bangladesh Democracy and Governance Program Evaluations (BDGPE) COR on progress with the evaluation on a weekly basis, in person or by electronic communication. Any delays or complications must be quickly communicated to USAID/Bangladesh as early as possible to allow quick resolution and to minimize any disruptions to the evaluation. Emerging opportunities for the evaluation should also be discussed with USAID/Bangladesh. Debriefing with USAID – PowerPoint presentation of initial findings, conclusions, and preliminary recommendations to USAID/Bangladesh before the international team members depart from Bangladesh. Debriefing with Partners – The team will present the major findings from the evaluation to USAID partners (as appropriate and as defined by USAID) through a PowerPoint presentation prior to the team’s departure from the country. The debriefing will include a discussion of achievements and activities only, with no recommendations for possible modifications to project approaches, results, or activities. The team will consider partner comments and incorporate them appropriately in drafting the evaluation report. Draft Evaluation Report – The Evaluation Team will analyze all data collected during the evaluation to prepare a draft Performance Evaluation Report and submit the report within 15 working days after the departure of international team members from Bangladesh. The draft report must be of a high quality with well-constructed sentences, and no grammatical errors or typos. The report should answer ALL the evaluation questions and the structure of the report should make it clear how the evaluation questions were answered. The draft report must meet the criteria set forth under the final report section below. USAID will provide comments on the draft report within 10 working days of submission. The Evaluation Team will in turn revise the draft report into a final Performance Evaluation Report, fully reflecting USAID comments and suggestions. Final Report: The Evaluation Team will submit a final Performance Evaluation Report that incorporates Mission comments and suggestions no later than 10 working days after USAID/Bangladesh provides written comments on the draft Performance Evaluation Report. The format of the final report is provided below. The final report must meet the following criteria to ensure the quality of the report:  The evaluation report must represent a thoughtful, well-researched, and well organized effort to objectively evaluate what worked in the project, what did not, and why.  Evaluation reports shall address all evaluation questions included in the scope of work.  The evaluation report should include the scope of work as an annex. All modifications to the scope of work—whether in technical requirements, evaluation questions, Evaluation Team composition, methodology, or timeline—need to be agreed upon in writing by the technical officer.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included as annexes in the final report.  Evaluation should include tables, graphs, and maps as necessary.  Evaluation findings will assess outcomes and impact on males and females.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 42  Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. The format of the final performance evaluation report should strike a balance between depth and length. The report will include a table of contents, table of figures (as appropriate), acronyms, executive summary, introduction, purpose of the evaluation, research design and methodology, findings, conclusions, lessons learned, and recommendations. Where appropriate, the evaluation should utilize tables and graphs to link with data and other relevant information. The report should include, in the annex, any “Statement of Differences” by any team member or by USAID on any of the findings or recommendations. The report should not exceed 30 pages, excluding annexes. The report will be submitted in English, electronically. The report will be disseminated within USAID. Upon instruction from USAID, Social Impact (SI) will submit (also electronically, in English) this report excluding any potentially procurement-sensitive information to Development Experience Clearinghouse (DEC) for dissemination among implementing partners, stakeholders, and the general public. The DEC submission must be within three months of USAID’s approval of the final report. All quantitative data, if gathered, must be (1) provided in an electronic file in easily readable format; (2) organized and fully documented for use by those not fully familiar with the project or the evaluation; (3) owned by USAID and made available to the public barring rare exceptions. A CD with all the data could be provided to the COR. The final report will be edited/formatted by Social Impact and provided to USAID/Bangladesh 10 working days after the Mission has reviewed the content and approved the final revised version of the report. VII. Team Composition/ Technical Qualifications and Experience Requirements for the Evaluation Team The evaluation team will include and balance several types of knowledge and experience related to program evaluation. Individual team members should have the technical qualifications as described below: 1. Team Leader: An international Senior Evaluation Specialist with experience in evaluating Local Government programs in developing countries. The Team leader will provide leadership for the Team, finalize the evaluation design, coordinate activities, arrange meetings, consolidate individual input from Team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to USAID/Bangladesh. At least ten (10) years of experience in evaluation management is required. Experience in conducting assessments and designing strategic responses to Local Government in developing countries is required. Ability to produce highly quality evaluation report in English is essential. 2. National Team Member: The proposed team composition will include one team from BDGPE’s long term technical assistance staff for this Evaluation. The national team member will conduct FGDs, KIIs and other tasks related to data collection. He/She will also analyze collected data and will help the team leader in report writing. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 43 All positions will be considered key staff and will require USAID approval. Overall the team will need expertise in USAID practices and expectations in program evaluation; program design and analysis; quantitative and qualitative data collection and analysis; survey design and analysis; program issues, issues related to local governance; and USAID practices and requirements in program performance measurement. VIII. Conflict of Interest All evaluation team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the project being evaluated. USAID/Bangladesh will provide the conflict of interest forms. IX. Scheduling and Logistics Work is to be carried out over a period beginning from December 2014, with field work completed in January 2015 and final report and close out concluding o/a March 2015. Funding and Logistical Support The proposed evaluation will be funded and implemented through the BDGPE project. Social Impact will be responsible for all off-shore and in-country administrative and logistical support, including identification and fielding appropriate consultants. Social Impact support includes arranging and scheduling meetings, translation services, international and local travel, hotel bookings, working/office spaces, computers, printing, photocopying, arranging field visits, local travel, hotel, and appointments with stakeholders. The evaluation team will make all logistic arrangements including the vehicle arrangements for travel within and outside Dhaka and should not expect any logistic support from the Mission. The team will make their own arrangement on office space for team meetings and equipment support for producing the report. Scheduling Task/ Deliverable Proposed Dates Team Leader Review background documents & preparation work (offshore) 12/15/2014- 1/7/2015 3 Travel to Bangladesh by expat team member 1/08-1/09/2015 2 Team Planning Meeting hosted by BDGPE 1/11 1 In-brief with USAID/Bangladesh 1/12 .5 Meet with SDLG/Tetra Tech ARD staff 1/12 .5 Final work plan due to USAID (OOB Dhaka time) 1/14 Data collection 1/13-1/29 15 Analysis and product drafting in-country Evaluation Team submits annotated report outline and draft presentation for USAID/Bangladesh DG Team review; data collection continues after submission 1/28 - Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 44 X. Reporting Requirements The total report, excluding references and annexes, should not be more than 30 pages. The following content (and suggested length) should be included in the report: Table of Contents Acronyms Executive Summary – concisely state the project purpose and background, key evaluation questions, methods, most salient findings and recommendations (2-3 pp.); 1. Introduction – country context, including a summary of any relevant history, demography, socio￾economic status, etc. (1 pp.); 2. The Development Problem and USAID’s Response – brief overview of the development problem and USAID’s strategic response, including design and implementation of the SDLG program and any previous USAID activities implemented in response to the problem, (2-3 pp.); 3. Purpose of the Evaluation – purpose, audience, and synopsis of task (1 pp.); 4. Evaluation Methodology – describe evaluation methods, including strengths, constraints, and gaps (1 pp.); 5. Findings and Conclusions – describe and analyze findings for each evaluation question using graphs, figures, and tables, as applicable, and also include data quality and reporting system that should present verification of spot checks, issues, and outcomes. Conclusions should be credible and should be supported by the findings (12-15 pp.); 6. Recommendations – prioritized for each evaluation question; should be separate from conclusions and be supported by clearly defined set of findings and conclusions. Include recommendations for future project implementation or relevant program designs and synergies with other USAID projects and other donor interventions as appropriate (3-4 pp). 7. Lessons Learned – provide a brief of key technical and/or administrative lessons on what has worked, not worked, and why for future project or relevant program designs (2-3 pp.); 8. Annexes – to include statement of work, documents reviewed, bibliographical documentation, evaluation methods, data generated from the evaluation, tools used, interview lists, meetings, FGDs, surveys, and tables. The Evaluation Design Matrix must be presented as an annex to the report. Annexes should be succinct, pertinent, and readable. Should also include if necessary, a statement of differences regarding significant unresolved difference of opinion by funders, implementers, or members of the Evaluation Team on any of the findings or recommendations. USAID provides comments (as needed) on report outline and draft presentation 1/29 - Presentation and debrief with DG Team and USAID/Bangladesh 2/1 .5 Debrief meetings with key stakeholders, including GOB if necessary 2/1 .5 Expat Team members depart Bangladesh 2/2-2/3 2 Produce draft report, delivers to USAID on 2/25 2/4-2/25 6 USAID and partners review draft and provide comments 2/26-3/11 - Team revises draft report and submits to BDGPE 3/12-3/25 4 SI delivers final report 3/25 TOTAL 35 Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 45 The report format should be restricted to Microsoft products and 11-point type font should be used throughout the body of the report, with page margins one inch top/bottom and left/right. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 46 ANNEX II: MAP OF EVALUATION SITES ANNEX III: DOCUMENTS REVIEWED SDLG M&E Plan and Work Plan: RANGPUR (Sadar Upazila) Chandanpath UP Saddyapuskurini UP BOGRA Kahalu Municipality RAJBARI (Sadar Upazila) Mizanpur UP Chnadani UP JESSORE (Sadar Upazila) Arabpur UP Basundia UP SATKHIRA Satkhira Municipality Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 47 1. USAID/Bangladesh. SDLG Extension Period Performance Indicators. (PMP Chart). 2014 2. USAID/Bangladesh. SDLG Yearly SC Action Plan (Climate). 3. USAID/Bangladesh. SDLG Yearly Standing Committee Action Plan (Ag, Health, W&C). 2014. 4. USAID/Bangladesh. Strengthening Democratic Local Governance (SDLG) Performance Monitoring Plan- Extension Period 2014-2015. September 2014, Tetra Tech ARD. 5. USAID/Bangladesh. Strengthening Democratic Local Governance (SDLG) in Bangladesh. Work Plan-2014-15. September 2014, Tetra Tech ARD. SDLG Quarterly Reports: 6. SDLG Health Collaboration Success (Small Note). 7. USAID/Bangladesh. SDLG Quarterly Report No 10 July 2014 8. USAID/Bangladesh. SDLG Quarterly Report No 11 October 2014 SDLG Project Documents: 9. SDLG. SDLG-LGSP-SDC Comparison. Powerpoint Presentation. 10. The Daily Star. Upazila Parishad: Victim of Power Politics. January 2014. 11. Unitrend Limited. SDLG Folk Drama: A Friendly Football Match (Script). 12. USAID/Bangladesh. KAP survey frequency tables (PS & UP). 2014. 13. USAID/Bangladesh. Revised Budgets for MMS, RDRS, Democracy Watch, Shushilan and BITA. 2014. 14. USAID/Bangladesh. SDLG Project - Sector Integration 2014 (PowerPoint Presentation). 15. USAID/Bangladesh. SDLG Training Module on Monitoring and Reporting for Standing Committees. (In Bangla). 16. USAID/Bangladesh. Terms and Condition for SDLG Participants (Eye Reports Youth Selection Criteria). 17. USAID/Bangladesh. We are the Government: Strategic Communication For Citizen Participation In Local Governance In Bangladesh. (Booklet). March 2014. SDLG Contracts: 18. USAID/Bangladesh. Addendum Grant SDLG004 Negotiation Memo- Grant Extension Activities (MMS) 19. USAID/Bangladesh. Addendum Grant SDLG005 Negotiation Memo- Grant Extension Activities (RDRS) 20. USAID/Bangladesh. Addendum Grant SDLG007 Negotiation Memo- Grant Extension Activities (Democracy Watch) 21. USAID/Bangladesh. Addendum Grant SDLG008 Negotiation Memo- Grant Extension Activities (Shushilan) 22. USAID/Bangladesh. Addendum Grant SDLG011 Negotiation Memo- Grant Extension Activities (Bangladesh Institute of Theatre Arts-BITA) 23. USAID/Bangladesh. MoU Between SDLG, Tetra Tech ARD and NGO Health Service Delivery Project, Pathfinder International. Period: February 1 to December 31, 2014. 24. USAID/Bangladesh. SDLG Task Order No. AID-388-TO-11-00001. Modification No. 10. Page 1 of 16 SDLG Advocacy and Research: 25. Kabir, Dr. Mahfuz, Ashfaq R. Khan and Robert S. Guda. Improving Services: The Role of Union Parishad Standing Committees. October, 2014. Unnayan Shamannay. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 48 26. Kabir, Dr. Mahfuz. Upazila Parishad Governance to Improve Service Delivery Policy, Practice and Model Upazila Functioning - Bhairab Upazila, Kishoreganj. N.p.: n.p., Unnayan Shamannay. 27. Unnayan Shamannay. National Conference on Improving Services: The Role of Union Parishad Standing Committees, (Draft Event Report) N.p.: n.p., 26 October 2014. CIRDAP Auditorium, Dhaka. 28. Unnayan Shamannay. National Conference on Upazila Parishad Governance to Improve Service Delivery, (PowerPoint Presentation) N.p.: n.p., 26 June 2014 CIRDAP Auditorium, Dhaka. 29. Unnayan Shamannay. National Conference on Upazila Parishad Governance to Improve Service Delivery, (Event Report) N.p.: n.p., 26 June 2014. CIRDAP Auditorium, Dhaka. 30. Unnayan Shamannay. Upazila Parishad Governance to Improve Service Delivery Policy, Practice and Model Upazila Functioning - Bhairab Upazila, Kishoreganj. N.p.: n.p., June 2014. USAID Documents: 31. Blair, Harry, Michael Calavan, Md. Azizur Rahman Siddique, and Naim Mostafa. Evaluation of the Improving Local Level Governance Project in Bangladesh: Combining Traditional Folk Arts with Democratic Local Governance Dhaka: Social Impact, November. 2012. 32. Cartier, William, A.K.M Saifullah, and Naim. Mostafa. Performance Evaluation of USAID/ Bangladesh Leadership Development Program (LDP). Dhaka: Social Impact. May, 2015 (forthcoming). 33. Keshishian, Mike and Taft-Morales, Maureen. Evaluation Of the USAID/Bangladesh Local Government Activity. October 7, 2008. 34. Khan, Dr. Mohammad Mohabbat. Functioning of Local Government (Union Parishad): Legal and Practical Constraints. N.p.: n.p., Democratic Local Governance Program/USAID 35. Khan, Dr. Mohammad Mohabbat. Tax Payment Behavior of UP Citizens. N.p.: n.p., Democratic Local Governance Program/USAID 36. USAID/Bangladesh. Bangladesh Country Development Cooperation Strategy FY 2011 – FY 2016. N.p.: n.p., 2011. Print. 37. USAID/Bangladesh. Bangladesh Democracy and Governance Assessment. 2009. 38. USAID/Bangladesh. Democratic Local Governance Project (Final Report). 2008. Secondary Research and Documents: 39. Ahmed, Bashir and Islam, Mohammad Tarikul. The Role of Union Parishad in Rural Dispute Resolution in Bangladesh: an Evaluation in the Light of People’s Perception. N.p.: n.p., Jahangirnagar University, Dhaka, Bangladesh. 40. Ahmed, Dr. Mohsin Uddin. Evaluation of Local Government in Undivided Bengal: Special Reference on the urban local government of west Bengal. N.p.: n.p., Khulna University. 41. Ahmed, Dr. Mohsin Uddin. Urban Local Government Services in Bangladesh: Study on the regulatory problem. N.p.: n.p., Khulna University. 42. As‐Saber, Sharif N. and Rabbi, Md Fazle. Democratisation of the Upazila Parishad and Its Impact on Responsiveness and Accountability: Myths versus Realities. 2009. JOAAG, Vol. 4. No. 2 43. Bangladesh. Country Reports on Local Government Systems. N.p.: n.p., Print. 44. Bangladesh. National Urban Sector Policy. N.p.: n.p., 2011. 45. Barkat, Abdul, et al. Local Governance and Decentralization in Bangladesh.2015. Dhaka: Pathak Samabesh/Swiss Agency for Development Cooperation. 46. CARE Bangladesh. Poor and Extremely Poor Women’s engagement in Local Government Development Initiatives: A good practice documentation. November 2012. Local Governance (EC-NSA) Project. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 49 47. Government of Bangladesh. How to work more effectively together to deliver real development outcomes. June 2010, Aid effectiveness Unit, ERD, Ministry of Finance. 48. Hossain, Dr. Monzur and Majumder, Dr. Badiul Alam. Baseline Survey Report on Upazila Governance in Bangladesh. June 2013. Bangladesh Institute of Development Studies (BIDS) 49. Hussain, Akther. Local Governance in Bangladesh: The Emerging Role of The Development Partners. N.p.: n.p., October-December, 2003 , CDRB Publication: Asian Affairs, Vol. 25, No. 4 :5-22 50. Ivy, Dr. Salina Hayat. Inclusive City Government: Reform Issues Urban Local Government System in Bangladesh. N.p.: n.p., Narayanganj City Corporation, Bangladesh. 51. LGRD ministry and Police Head Quarter. Administrative Structure of Bangladesh. Power point presentation, N.p.: n.p., 52. Odhikar. Union Parishad Elections 2011. 53. Panday, Pranab Kumar. Local Government System in Bangladesh: How Far is it Decentralized? July 2011. Lex Localis-Journal of Local Self-Government, VOL. 9, NO. 3, PP. 205 – 230 54. Paul, Sohini And Goel, Pooja Ramavat. Decentralisation in Bangladesh. September 7, 2010, NCAER New-Delhi. 55. Rahman, Atiur. Kabir, Mahfuz and Abdur Razzaque. Civic Participation in Sub-National Budgeting in Bangladesh. October 2004. World Bank Institute, Washington, D.C. 56. Rahman, Muhammad Sayadur. Upazila Parishad in Bangladesh: Roles and Functions of Elected Representatives and Bureaucrats. December 2012. Commonwealth Journal of Local Governance, Issue 11: http://epress.lib.uts.edu.au/ojs/index.php/cjlg 57. Research and Evaluation Division, BRAC. Gender and good governance issues in local government of Bangladesh: A baseline report. N.p.: n.p., January 2004. 58. The Asia Foundation. Union Parishad Elections in Bangladesh (Final Observation), 2011. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 50 ANNEX IV: PERSONS INTERVIEWED Dhaka Name Sex Position Affiliation Jerome Sayre Male Chief of Party Tetra Tech ARD Rosy Hossain Female Team Leader, Local Government Service Delivery Tetra Tech ARD Abu Md. Mohsin Male Team Leader, Citizen Participation Tetra Tech ARD Jeremy Davis Male Deputy Chief of Party USAID Ag Extension Project A. N. M. Kaiser Zillany Male National Technical Coordinator USAID Ag Extension Project, CARE Apurba Deb Roy Male Community Development Specialist USAID Ag Extension Project, CARE Shaikh M. Jobayed Hossain Male Manager- Advocacy and Communications Plan International Bangladesh Halida H. Akhter Female Chief of Party NHSDP and Country Representative, Pathfinder ARMM Kamal Male Community Mobilization Advisor NHSDP, Pathfinder K. M. Azizur Rahman Male Urban Specialist NHSDP, Pathfinder SherinaTabassum Female AoR/Governance Advisor USAID/B. Niaz Chowdhury Male Project Management Specialist Office of Population, Health, Nutrition & Education USAID / Dhaka Habiba Akter Female Human Rights and Rule of Law Advisor Office of Democracy and Governance, USAID / Dhaka Abu JamilFaisel Male Country Representative Project Director, Mayer Hashi II Azmal Hossain Male Program Specialist Mayer Hashi II Rangpur Name Sex Position Affiliation Manju Shree Saha Female Head of Programe Coordination RDRS Dipak Chandra Nath Male Assistant Project Coordinator SDLG, RDRS Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 51 Rangpur Name Sex Position Affiliation Md. Kabir Miah Male Project Officer SDLG, RDRS Md. Rabiul Islam Male CIG Member Chandanpat UP, Rangpur Tariqul Islam Male SC Member Chandanpat UP, Rangpur Mostafizar Rahman Male CIG Member Chandanpat UP, Rangpur Alim Uddin Male CIG Member Chandanpat UP, Rangpur Ismail Hossain Male SC Member Chandanpat UP, Rangpur Jahangir Alom Male UP Councilor Chandanpat UP, Rangpur Khoirat Hossain Male SC Member Chandanpat UP, Rangpur Anjuara Begum Female UP Councilor Chandanpat UP, Rangpur Nazma Begum Female CIG Member Chandanpat UP, Rangpur Rezina Begum Female CIG Member Chandanpat UP, Rangpur Golapi Begum Female SC Member Chandanpat UP, Rangpur Ivy Begum Female CIG Member Chandanpat UP, Rangpur Siddika Begum Female UP Councilor Saddapushkurini UP, Rangpur Nasima Begum Female SC Member Saddapushkurini UP, Rangpur Jenifa Begum Female SC Member Saddapushkurini UP, Rangpur Abu Motaleb Male UP Councilor Saddapushkurini UP, Rangpur Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 52 Rangpur Name Sex Position Affiliation Shajahan Male SC Member Saddapushkurini UP, Rangpur Sekender Ali Male SC Member Saddapushkurini UP, Rangpur Habjer Rahman Male CIG Member Saddapushkurini UP, Rangpur Shahinur Rahman Male CIG Member Saddapushkurini UP, Rangpur Amanat Shah Male CIG Member Saddapushkurini UP, Rangpur Korban Ali Male CIG Member Saddapushkurini UP, Rangpur Rezena Begum Female CIG Member Saddapushkurini UP, Rangpur Rina Begum Female CIG Member Saddapushkurini UP, Rangpur Sufia Begum Female FWV Saddapushkurini UP, Rangpur AbdurRouf Male UP Chair Saddapushkurini UP, Rangpur Bogra Name Sex Position Affiliation Momtaz Hossain Male CIG Chair Kahalu Municipality, Bogra. Toyez Uddin Male SC Chair, Urban Plan Kahalu Municipality, Bogra. Mofazzal Hossain Male SC Chair, Audit and Accounts Kahalu Municipality, Bogra. Golam Rabbani Male SC Chair, Tax Assessment Kahalu Municipality, Bogra. Anower Hossain Male CIG Member Kahalu Municipality, Bogra. Shodidul Islam Male CIG Secretary-finance Kahalu Municipality, Bogra. SheuliKhatun Female CIG, Vice Chair Kahalu Municipality, Bogra. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 53 Bogra Name Sex Position Affiliation Smrity Begum Female CIG Member Kahalu Municipality, Bogra. KhaledaAktar Female Councilor Kahalu Municipality, Bogra. Sheuli Female CIG Member Kahalu Municipality, Bogra. Jessore Name Sex Position Affiliation Zakir Hossain Male CIG Member Arabpur UP, Jessore Mizanur Rahman Mukul Male SC Member Arabpur UP, Jessore Noorjahan Begum Female UP Councilor Arabpur UP, Jessore Shefali Begum Female CIG Member Arabpur UP, Jessore Rizia Begum Female UP Councilor Arabpur UP, Jessore MerinaKhatun Female UP Councilor Arabpur UP, Jessore Ahammad Male CIG Member Arabpur UP, Jessore Nowsher Ali Male SC Member Arabpur UP, Jessore Atiyar Rahman Male CIG Member Arabpur UP, Jessore Rafiqul Islam Male CIG Member Arabpur UP, Jessore Altaf Hossain Male UP Councilor Arabpur UP, Jessore Asaduddoula Male Training Coordinator NHSDP Clinic, Jessore MollikZaman Male Regional Training Coordinator DAM-USAID Agri Extension Project RazuAhmmed Male CHCP Community Clinic, Arabpur UP, Jessore Tariqul Islam Male CIG Member Bosundia UP, Jessore Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 54 Jessore Name Sex Position Affiliation Nizam Uddin Male CIG Member Bosundia UP, Jessore Asura Begum Female SC &CIG Member Bosundia UP, Jessore KhadizaKhatun Female SC Member Bosundia UP, Jessore Mira Khatun Female CIG Member Bosundia UP, Jessore Farida Yesmin Female UP Councilor Bosundia UP, Jessore ShahnazParvin Female UP Councilor Bosundia UP, Jessore Makibur Rahman Male UP Councilor Bosundia UP, Jessore Sheikh GoffarRahaman Male CIG Member Bosundia UP, Jessore SelimBishwash Male SC Member Bosundia UP, Jessore Rafiqul Islam Male CIG/SC Member Bosundia UP, Jessore A Salam Male CIG/SC Member Bosundia UP, Jessore LatifaParvin Female CHCP Bosundia UP, Jessore Ripon Adhikari Male Union Facilitator DAM-USAID Agri Extension Project SayedMeeyez Hossain Male Clinic Manager SHE, NHSDP, Baradipara. Satkhira Name Sex Position Affiliation SkAsad Ahmad Anju Male Councilor Satkhira Municipality Md. Ali Siddique Male President CIG Satkhira Municipality Syed Haider Ali Tota Male CIG Secretary Satkhira Municipality MasumBillah Male Councilor, PS Satkhira Municipality Farida Akhter Beauty Female Councilor, PS Satkhira Municipality Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 55 Satkhira Name Sex Position Affiliation Farhadita Khan Sathi Female Councilor, PS Satkhira Municipality Khurshid Jahan Shila Female CIG, Vice Chair Satkhira Municipality Abdul Hakim Male CIG Member Satkhira Municipality NasimaPervin Female CIG Member Satkhira Municipality JosnaParvin Female CIG Member Satkhira Municipality JosnaAra Female Councilor, PS Satkhira Municipality Sk. SafiqUdDowla Male Panel Mayor Satkhira Municipality Rajbari Name Sex Position Affiliation Atiar Rahman Male Chairman Mizanpur UP, Rajbari Asia Begum Female Councilor (Women seat); and President SC on Audit and Accounts Mizanpur UP, Rajbari Abul Hossain Male President, CIG Mizanpur UP, Rajbari Sabur Ahmed Male Community Health Care Provider (CHCP) Suryanagar Community Clinic, Mizanpur UP, Rajbari Nasrin Akter Female Volunteer, Damien Foundation Suryanagar Community Clinic, Mizanpur UP, Rajbari Kohinur Begum Female Councilor (Women Seat); President SC on Audit and Accounts Chandani UP, Rajbari Anwar Sikder Male President, CIG Chandani UP, Rajbari Abu Raihan Male District Manager USAID Agricultural Extension Proejct, Rajbari BadrunnesaBakul Female Councilor (Women seat); President SC on Education, Health and Family Planning Khankhanapur UP, Rajbari Abdul Aziz Shikder Male President, CIG Khankhanapur UP, Rajbari Md Rakibuddin Male Upazila Agriculture Officer RajbariSadar, Rajbari Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 56 ANNEX V: EVALUATION MATRIX Evaluation Questions Sub-Questions Primary Data sources How effectively has SDLG been able to integrate local governance issues to successfully address governance challenges in other sectors, such as, health, agriculture and climate change? • What are the critical factors that are contributing to any progress towards this overarching program goal in 2014? • How much of an increase did you see in coordination with other USAID Sector programs in the past year? • In 2014, how well did the Councils, SCs & CIGs integrate with sectors, such as health, education and agriculture (formal or informal)? Do they have a good working relationship and discuss issues with regular meetings. • How much input does SCs/CIGs have in delivering services in different sectors? • Do the Councils consult citizens regarding sector planning (budget, resource mobilization, project allocation, etc) if so, how? • Are there official channels open to citizens (Re: Ward meetings) to make their concerns and issues be considered - in sector development. Have these channels increased in the past year? • The Participatory Planning Process is one of the most important activities at LGUs. Are citizens included in this process from the beginning and do they have a say in prioritizing integrated sector projects in their community? How? • Are their ways that can improve citizen participation with sector development? How? SDLG Docs, SDLG Staff, PNGOs FGDs, KIIs USAID Sector Partners, SCs and CIGs Mayors, UP Chair, Councilors. Sector Field staff To what extent has SDLG been able to ensure that the linkages it has created with the other USAID sectors are sustainable? What follow-up work needs to be done? • Are there sufficient linkages and guidelines created for Councils to measure progress in various sector development and with other USAID sector development? Have they increased in the past year? How? • Is there a specific calendar when reporting problems, issues and progress? • Were any of SDLG’s activities coordinated with activities of other USIAD Sector initiatives? Why or why not? • Did the choice of project sites allow SDLG to complement the work of other USAID Sector programs? • Are there additional potential areas for collaboration between SDLG and other organizations (especially USAID-funded)? • Does the SLDG include appropriate indicators for measuring outputs and impacts in sector development? • Is data being collected systematically and in a format easily aggregated and analyzed? Is SDLG tracking the kinds of cases, kinds of interventions, kinds of impact, and associated costs? SDLG Docs, SDLG Staff, PNGOs FGDs, KIIs USAID Sector Partners, SCs and CIGs Mayors, UP Chair, Councilors. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 57 • What kinds of challenges has SDLG faced with other USAID Programs, PNGOs, GoB, CIGs and/or other partners? • How have these problems and challenges been resolved? • Are there any examples of a lack of flexibility or effective response to challenges from SGLG management to create linkages in the past year with other USAID Sector Programs? • What challenges persist and what recommendations could be made for addressing those challenges? • What are the competences and role of LG authorities regarding sector integration? • How well do the Councils and the sectors work together, such as health education and agriculture (formal or informal)? Do they have a good working relationship and discuss issues with regular meetings. Sector Field staff To what extent are the project’s objectives still relevant to the current development circumstances in Bangladesh? • How ready are the councils to coordinate sectoral project implementation to the UP, such as regular meetings of the council, accounting, budgeting, keeping of records and the attendance of elected councillors? • How well do the Councils and Administration work together. (formal or informal, especially for special circumstances, urgent issues, etc)? Do they have a good working relationship and discuss issues with regular meetings? • Does SDLG adapt well to changing circumstances? Has it changed any partners and/or activities in the past year? If so, which ones and how? What other adaptations are recommended? • How does SDLG address any problems? How might SDLG address problems if it is not doing so already? • How well do the Councils and the sectors work together, such as health education and agriculture (formal or informal)? Do they have a good working relationship and discuss issues with regular meetings. • Do Warding Meetings provide citizens with ways citizens can provide their concerns, opinions and inputs? How? • Are their ways that can improve citizen participation? How? SDLG Docs, SDLG Staff, PNGOs FGDs, KIIs USAID Sector Partners, SCs and CIGs Mayors, UP Chair, Councilors. Sector Field staff To what extent were gender and youth effectively addressed by SDLG’s interventions in the targeted areas? • What levels of active participation in decision-making do women and youth have in the political and planning processes (women's Development Forum)? • What important roles do women and youth contribute to the community, and are there any women’s Community Based Organizations. Please explain their functions and activities and how they work with the Local Government. • In your opinion, what areas are required to improve the roles, responsibilities and empowerment of women and SDLG Docs, SDLG Staff, PNGOs FGDs, KIIs USAID Sector Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 58 youth at the local government levels? • In 2014, how many women were represented in SCs and CIGs. How active were the women in these groups? Partners, SCs and CIGs Mayors, UP Chair, Councilors. Sector Field staff Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 59 ANNEX VI: DATA COLLECTION PROTOCOLS SDLG Extension Evaluation Interview Checklist for UP representatives Name: Age: Sex: M/F Education: Occupation: UP: Upazila: District: Date: 1. How did you know about SDLG project? 2. What really did you do with SDLG project? 3. How often they used to meet you or visit your UP? 4. How did you help formation of Citizens in Governance (CIG) team? 5. What assistance you have so far received or asked from the CIG? 6. How important is to have a citizen group like CIGs in your UP? 7. What are the areas you think CIGs could have contributed more? 8. How many SCs were formed in your UP? How many SCs SDLG had been dealing with? 9. What changes did you observe in the SCs due to SDLG intervention? 10. How often SCs submit report based on their monitoring work on services delivered to the people? How many you have received so far? 11. What do you usually do once you receive a report from any of SC of your UP? 12. Do you think SC got a role in improving delivery of services? 13. Other than the government services what other service providers are providing services to people in your UP (especially on Health, Agriculture or rights issues)? 14. What roles do you think the CIGs can play to improve the quality of service delivery by government and other providers? 15. What support the CIGs can offer to improve the overall performance of the UP? 16. Are you aware of any action plan developed by SCs in your UP? 17. What support CIGs extended to convene Ward Meeting in your UP? 18. Did CIG support your revenue collection drive in any way? If yes, how? 19. How did you find the role of UP SC on Audits and Accounts in your UP? 20. How often the SCs in your UP meet? 21. What support should be given to make the SCs more active and functional? 22. What support you think should be provided to CIG to improve the service delivery in your UP? Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 60 SDLG Extension Evaluation Checklist for Interviewing CIG Leaders Name: Age: Sex: M/F Education: Occupation: UP: Upazila: District: Date: 1. How did you know about SDLG project? 2. What really did you do with SDLG project? 3. What is really a Citizen in Governance (CIG) Group? 4. Why CIGs ere formed? 5. How did the CIG formed? Can you please describe the process? 6. What kind of help did you receive from SDLG or DW/MMS/Shushilan/RDRS/BITA to form the CIG? 7. Did you receive any assistance from UP? How did they help you? 8. What are the key functions of CIG? Can you please make a list: a) b) c) 9. How important is to have a citizen group like CIGs in your UP? 10. How do you coordinate with the Standing Committees of UP? 11. How often do you have meeting with SCs? 12. Do SCs invite you to join them during monitoring visits to service delivery agencies? 13. How important, according to you, are monitoring the service delivery at UP level? What sense they make to monitor service delivery as SCs do not have any executive authority? 14. How many standing committees were you are working with or established relationship in your UP? 15. How important was SDLG or DW/MMS/Shushilan/RDRS/BITA support to form CIG in your UP? Do you think CIGs could be formed without SDLG or DW/ MMS/ Shushilan/ RDRS/BITA support? 16. Do SCs care to listen to what you say or ask them to do? 17. Did you see any changes in service delivery after SC monitoring visits paid to any of the service delivery agencies? Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 61 18. In your UP, what are the services being monitored by the SCs? 19. Are you aware of the services provided by different agencies? Can you name non-government service providers in your UP? 20. What support the CIG have extended so far to the UP? Can you make a list of those supports? 21. What support the CIG extended to convene Ward Meeting in your UP? Were you appreciated for the support by UP? 22. Did your CIG support UPs revenue collection drive? If yes, how? 23. Are you aware of any action plan developed by SCs in your UP? 24. Did you ever meet with the SC on Audits and Accounts in your UP? What was your impression about the work of that committee? 25. What roles do you think the CIGs can play to improve the quality of service delivery by government and other providers? 26. What support should be given to make the SCs more active and functional? 27. What support you think should be provided to CIG to improve the service delivery in your UP? Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 62 SDLG Extension Evaluation FGD for SC members UP: Upazila: District: Date: INTRODUCTION, GROUND RULES, CONFIDENTIALITY AND INFORMED CONSENT The moderator should first introduce herself or himself, then welcome and thank all of the FGD participants. After that, the moderator should explain the purpose of the FGD, how the participants were selected, any potential benefits or risks to participating in the FGD, how long it will take, and whether there will be any compensation for participating. After discussing these logistical issues, the moderator also should address expectations, or ground rules, for the FGD. The ground rules will vary depending on the FGD, but in general they will include: • Everyone is encouraged to share their ideas, and the FGD is strengthened if everyone participates. • There are no wrong answers, and everyone’s perspective is equally valued. • The ideas shared during the FGD should not be shared outside the FGD with non-participants in order to respect participants’ privacy. • Disagreements about ideas can be valuable and productive, but personal attacks will not be tolerated. After establishing these ground rules, the moderator should ask if there are any questions or concerns participants have, and these issues should be addressed and consensus reached as a group before moving on. The question of confidentiality is also important to address. The moderator should clearly describe how the data collected will be used, including with whom it will be shared, and crucially, whether names or other personal or identifying information will be included with the data. Many times, the experiences and opinions shared during a focus group will include sensitive information, and participants may not feel comfortable sharing openly if they feel it could have negative consequences for them in the future. The moderator must be honest about how the data will be used, but should also reassure the participants that the data will be treated sensitively and that their privacy will be respected to the greatest degree possible given the needs and purposes of the evaluation. After providing this information, it is important to describe what will or will not happen if they choose not to participate. To ensure the data collected are reliable, participation in an FGD should be entirely voluntary and there should be no consequences for declining to participate. After informing participants of all of this information, the moderator must ask each member to confirm that they consent to participate in the FGD. Often, to be consistent across FGDs, the language communicating these points is written in the moderator’s guide. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 63 Guiding Questions Question Probing Instruction 1 • What type of project you are engaged in with SDLG or partners (Democracywatch/ MMS/ Shushilan/ BITA/ RDRS)? Try to verify if the participants understand the objectives 2 • Who contacted you first to join this type of activities? And how did you join? Verify the role of political actors, SDLG PNGOs and others 3 • How frequently and/or actively did you interact with SDLG or DW/MMS/Shushilan/RDRS/BITA staff? a. Very actively participated b. Some participation c. No participation What were roles of SDLG and the PNGOs in the activities with CIGs and SCs? 3 • How did you hear about SDLG? a. From UP chairman b. From a senior person in the community c. NGO person told me d. A meeting was organized and I leaned it there e. Cannot remember How were the participants chosen? 4 • Overall, what was your impression of SDLG or DW/MMS/ Shsushilan/ RDRS/BITA? [Note to moderator: May be asked at the end of the session.] Prompts: • Was the staff knowledgeable? • Did they have a respectful approach? • Did they meet their commitments? • Did they ask your input and opinions about activities? • How often the project staff meet you? Get the impression of the participation (a) Mostly positive. (b) Mixed. (c) Mostly negative. 5 • What did you do since you got involved with the CIG group? Ask for specific examples from the participants 6 • How did you get selected to participate in the training programs organized by (DW/MMS/ Shsushilan/ RDRS/BITA)?? a. I was nominated by community people b. NGO staff included my name in the list c. UP chairman/member pushed my name in the list d. Cannot remember 7 • Were you present in the selection meeting in the UP? Who attends? What was discussed there other than the selection process? 8 • What was the first activities that you have done after joining SDLG (DW/MMS/ Shsushilan/ RDRS/BITA)?? a. b. Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 64 c. • How many training programs you have attended so far with (DW/MMS/ Shsushilan/ RDRS/BITA)?? a. Attended three training programs b. Attended two training programs c. Attended one training program d. Cannot remember • Can you please name some of the training you have participated during last two years or so? a. b. c. • What was the most exciting training program, according to your understanding? (Do not prompt, let the participants tell) 9 • After getting involved with SDLG, how many monitoring visits you have completed so far? Try to understand what is meant by a monitoring visit. What does it entail? • How many times SDLG (DW/MMS/ Shsushilan/ RDRS/BITA) staff accompanied you during the monitoring visits? • What did they (NGO staff) do when a monitoring visit came to an end? • Did you prepare an action plan for your SC? When did you do that and why? • Who helped you to prepare the action plan? Who monitors whether you are following the action plan or not? • Were you able to act according to the action plan? Why and why Not? • If not, what support you need to follow the action plan? • How CIG helped you in monitoring services in your UP? • Were they supportive enough? Or just making illogical demands to do things? • What is your overall impression of CIG and its role? 10 • How confident are you in doing a service delivery monitoring by your own? a. Confident b. Not enough confident c. No way confident • How confident are you in preparing a monitoring report by your own? a. Confident b. Not enough confident c. No way confident • What did you do once you finished a monitoring visit to service providing entity? • Do you know what you should do during a monitoring visit? Can Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 65 you please tell us some of those? 11 Do you know what you should not/ cannot do during monitoring visits? a. b. c. d. • Can you differentiate between a monitoring visit and an inspection? Please elaborate? • Did you ever called by the UP Development Committee meeting to justify your report? Please elaborate? • What are the services available for the people in your UP from government and other sources? Who provide what services? Service Providing Agency a) b) c) • Did you have meeting with any of the agencies other than the government one? If so, which agency or organization did you met? 12 • To conduct a better monitoring what more support you need from NGOs/ Donors? a) b) c) 13 • Do you think you need financial support to keep the service monitoring on going by SCs? Why and Why Not? Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 66 ANNEX VII: SURVEY PROTOCOL Union Parishad __________________ Upazila ________________________ I am calling to ask you some questions about the Strengthening Democratic Local Governance project that was funded by USAID and implemented in your Union Parishad. We have been asked by USAID to interview some of the SDLG project participants to ask them about their experience with the project. Your answers will help USAID to improve future projects with local governments. Your answers are confidential and will not be revealed to anyone. 1. In the last 12 months, have you been a member of the Citizens in Government group formed by the SDLG project in your Union Parishad or paurashava? a) Ward level b) Throughout the UP or paurashava c) No 2. If answered “No” to question 1, skip to question 6. How often has your Citizens in Government group met in the last three months (November, December and January)? Choose one. a) Never b) At least once Number of times_______________ c) Don’t know 3. What specific activities has your Citizens in Government group carried out in the last three months? Multiple responses allowed. a) No activity b) Worked with standing committees to monitor UP or Upazila services c) Supported the UP tax collection d) Held open budget meetings e) Monitored implementation of UP projects f) Helped the UP to identify and solve problems g) Other ________________________________________ h) Don’t know 4. How many ward shavas have there been in your ward in the last three months? a) None b) Number __________ c) Don’t know 5. If answered “None” in question 4, skip to question 6. Did you participate in the last ward shava in your ward? How did you participate? a) Didn’t participate b) Attended the meeting c) Invited other people in the ward to participate d) Helped to organize or facilitate the meeting 6. What are the five most important functions or services of the Union Parishad? a) _______________ b) _______________ c) _______________ Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 67 d) _______________ e) _______________ 7. In the last 12 months have you participated in any activities with one of the Standing Committees in the Union Parishad or paurashva? a) Yes b) No Which committee? _________________ 8. If answered “No” to question 7, skip to question 15. Were you appointed as a formal member of the standing committee? a) Yes b) No c) Don’t know 9. How often has the standing committee held meetings during the last twelve months? Choose one. a) Every few weeks b) Monthly c) Every two months d) Twice a year e) Once a year or less f) Never g) Don’t know 10. Who has usually been responsible for organizing the meetings of the standing committee in the last twelve months? Choose one. a) The SDLG project staff b) The President of the Committee c) One of the members of the Committee d) The UP Secretary e) The UP Chair or one of the UP Council members f) Don’t know 11. Has the Union Parishad or paurashava assigned budget funds to support the work of the standing committee? a) Yes b) No c) Don’t know 12. How often has your standing committee met in the last three months (November, December and January)? Choose one. a) Never b) At least once Number of times_______________ c) Don’t know 13. Has the standing committee monitored the services provided by the government offices in the UP, paurashava or Upazila? What services? Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 68 a) Yes Name of service ______________ b) No c) Don’t know 14. How did the standing committee monitor the services? Please give examples of the monitoring activities carried out by the committee. Open Question. _______________________________________________________________________ _______________________________________________________________________ _______________________________________________________________________ ___________________________________________ 15. Can you tell me the names of the different standing committees in your Union Parishad or paurashava? a) _______________ b) _______________ c) _______________ d) _______________ e) _______________ f) _______________ g) _______________ h) _______________ i) _______________ j) _______________ k) _______________ l) _______________ m) _______________ 16. After carrying out the monitoring and reporting on the results, did the standing committee verify if there were any improvements in the services provided by the government offices in the Upazila or UP? a) Yes b) No c) Don’t know 17. Do you know of any improvement in the services that occurred because of the monitoring by the standing committee? If yes, please describe briefly. Open question. _______________________________________________________________________ _______________________________________________________________________ _______________________________________________________________________ ___________________________________________ 18. Does your standing committee face any obstacles in trying to monitor and improve services? Multiple responses allowed. a) The UP Chair and Secretary don’t support the standing committee b) Other standing committee members don’t support the work of the committee c) The government officers in the UP or Upazila don’t cooperate with the monitoring d) The committee members don’t participate sufficiently e) The committee members don’t have sufficient knowledge to be able to monitor the services f) The committee doesn’t have budget to be able to monitor effectively Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 69 g) The committee made a report but the UP or Upazila didn’t follow the recommendations made by the standing committee h) Other obstacle_____________________ i) No obstacles DEMOGRAPHIC QUESTIONS We want to ask you a few more questions about your age, occupation, and employment. Again, we want to remind you that your responses are confidential and won’t be revealed to anyone. Knowing this information is useful for designing future programs. EDUCATION a. None b. Class 1-5 c. Class 6-8 d. Secondary School e. Higher Secondary School f. Post Secondary GENDER a. Male b. Female AGE Years______________ Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 70 ANNEX VIII: SELECT SDLG PROJECT PERFORMANCE INDICATORS Indicator Indicator Definition Target End of Project 2.4} Number (#) of health clinics offering improved services in response to local standing committee oversight Improved service refers clinic management, including timely opening and staff availability; service delivery includes installing service, proper medicine distribution, monthly patient flow, counseling patient and number of referral cases. Standing committee oversight visit refers to SC members visiting the community clinic to monitor service quality. If issues are identified, these will be raised with clinic management for resolution. If this is not possible, the issue is recorded in the SC monitoring report and submitted to the UP, Union Development Coordination Committee (UDCC) or Upazila Parishad for action. 200 189 2.5} Percentage (%) of community served by clinics supported by groups of mobilized local influential stakeholders Community served by clinic refers to 6000 citizens within the periphery of community clinic. Groups of mobilized local influential stakeholders refers to UP health SCs. SC members (at least three members of committee) visit the community clinic to monitor service quality. If issues are identified, these will be raised with clinic management for resolution. If this is not possible, the issue is recorded in the SC monitoring report and submitted to the UP, Union Development Coordination Committee (UDCC), or Upazila Parishad for action. 25% 36% 2.6} Number (#) of agriculture extension officer actions improving services in response to local standing committee oversight Actions improving services refers to block farming training and counseling for the farmer/citizen of the wards of the LGUs. SC oversight visit refers to SC members of UP visiting plot demonstration and counseling sessions to monitor service quality. If issues are identified, these will be raised with the officer for resolution. If this is not possible, the issue is recorded in the SC monitoring report and submitted to the UP, Union Development Coordination Committee (UDCC), or Upazila Parishad for action. 95 118 2.9} Number (#) of institutions with improved capacity to address climate change issues as a result of USG assistance Institutions with improved capacity refers to agriculture standing committees under union councils that received training relevant to climate change impacts on local agricultural production. 190 187 Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 71 2.10} Number of local standing committees successful in preventing women’s rights violations Local standing committees successful in preventing women’s rights violation refers to standing committees on women and child welfare in SDLG unions with documented cases preventing early marriage, dowry, trafficking in persons, and other violations of women’s rights. 200 173 3.2} Number of citizens participating in local standing committee oversight SC oversight of local services and reporting on anomalies through elected councils at the union and upazila levels is part of the transparency and accountability system under existing law to improve service delivery. Counting the number of citizens involved in oversight activities is a measurement of strengthened implementation of this system in SDLG project sites. 2200 1718 Male – 1086 Female – 601 Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 72 ANNEX IX: DISCLOSURES OF CONFLICT OF INTEREST Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 73 Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 74 Performance Evaluation of the Strengthening Democratic Local Governance Project Extension Period 75 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523