February 23, 2022 This report was produced for the United States for International Development (USAID) by International Business & Technical Consultants, Inc. (IBTCI), under the Monitoring and Evaluation in Tunisia and Libya (METAL) Activity (AID￾280-TO-17-00001). EVALUATION MIDTERM EVALUATION OF THE LOCAL GOVERNANCE AND CIVIL SOCIETY (LGCS) ACTIVITY ii MIDTERM EVALUATION OF THE LOCAL GOVERNANCE AND CIVIL SOCIETY (LGCS) ACTIVITY February 23, 2022 International Business & Technical Consultants, Inc. (IBTCI) In the U.S.: 8618 Westwood Center Drive Suite 400 Vienna, VA 22182 USA Tel: +1.703.749.0100 In Tunisia: 1er Étage, Blue Bloc Vital Cube Building 4 L’Ile de Tasmanie Les Berges du Lac 2 1053 Tunis, Tunisia AID-280-TO-17-00001 Cover photo: A resident of the Alfwayhat District takes part in a volunteer community cleanup activity. Credit: LGCS – authorized use. DISCLAIMER This report is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of IBTCI and do not necessarily reflect the views of USAID or the United States Government. iii ABSTRACT The Local Governance and Civil Society (LGCS) Activity is a $39.4 million four-year initiative to strengthen subnational governance in Libya, focusing on 30 of the country’s 100 municipalities. This evaluation was requested by USAID/Libya and recommendations will be used to inform the implementation of existing and future local governance activities. LGCS focuses most of its efforts on municipal governments and registered civil society organizations (CSOs) with the aim of improving municipal performance and citizen engagement in responsive governance. It also provides some support to the Ministry of Local Governance (MOLG), the central agency involved with decentralization. The Activity has begun to build the capacity of municipalities to deliver improved services that respond to their constituents’ interests, strengthen the ability of CSOs to advocate for the community, and provide some support to the MOLG. CSO and municipality informants appreciated LGCS’ efforts, especially its capacity development inputs, but reported seeing little improvement in municipality performance as a result. MOLG officials said LGCS/USAID inputs were not consistent enough and should be better informed by local conditions. Additional assistance should be provided to MOLG to acquire the capacity to do the long-term institutional development the municipalities require and to help sustain the work of LGCS. iv CONTENTS ABSTRACT.................................................................................................................................................iii ACRONYMS...............................................................................................................................................vi 1. EXECUTIVE SUMMARY ..................................................................................................................... 1 2. INTRODUCTION................................................................................................................................ 8 3. EVALUATION PURPOSE AND EVALUATION QUESTIONS ...............................................12 4. EVALUATION DESIGN, METHODS, AND LIMITATIONS....................................................15 5. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ................................................22 EQ 1: To what extent has the Activity improved citizen participation in and oversight of municipal decision-making and service delivery processes?............................................................................................... 22 EQ 2: To what extent has the Activity enhanced the responsiveness of local government institutions?.................................................................................................................................................................. 24 EQ 2a. How has the Activity contributed to improved municipal service delivery performance? ....... 26 EQ 2b. To what extent have the Activity’s interventions affected public perceptions of local government performance and legitimacy?................................................................................................................................... 28 EQ 3: How has the Activity contributed to Libya’s decentralization agenda?........................................... 29 EQ3a: To what extent has the Activity enabled two-way national–local government coordination?.. 31 EQ4: Has the LGCS Activity mitigated potential negative effects of decentralization?............................ 32 AOI 1: How appropriate and relevant is LGCS’ design?.................................................................................. 32 AOI 2: How has LGCS supported the Ministry of Local Governance (MOLG)? ...................................... 36 AOI 3: How are other donors supporting subnational governance in Libya? ............................................ 37 Annex 1: Evaluation Scope of Work ...................................................................................................39 Annex 2: LGCS Indicator Progress Update 2021.10.27 .................................................................48 Annex 3: Data Collection Instrument.................................................................................................50 Annex 4: Raw Data Samples .................................................................................................................59 Annex 5: Sources of Information .........................................................................................................66 Annex 6: Evaluation Team Members’ Qualifications.......................................................................69 Annex 7: Bibliography.............................................................................................................................70 LIST OF FIGURES Figure 1: LGCS summary infographics....................................................................................................................................... 6 Figure 2. Libya Map and LCGS Partner Municipalities (FY 2021 Quarter 2).................................................................11 Figure 3. Altai Research Findings on Municipality Responsiveness to Citizens’ Needs.............................................25 v LIST OF TABLES Table 1. LGCS Causal Model .....................................................................................................................................................10 Table 2. Comparing LGCS Objectives and Evaluation Questions....................................................................................14 Table 3. Municipalities Selected for Evaluation......................................................................................................................15 Table 4. Key Informant Groups and Locations.....................................................................................................................16 Table 5. Team Member Roles and Responsibilities..............................................................................................................16 Table 6. LGCS Objective 2 CSO Oversight Indicators and Cumulative Results as of Q4 FY 2021 ......................................22 vi ACRONYMS AOI Additional Areas of Inquiry C4C Champions for Change COP Chief of Party CSO Civil Society Organization DAI Development Alternatives Incorporated DG Democracy and Governance DOs Development Objectives EQ Evaluation Question ET Evaluation Team EU European Union FGD Focus Group Discussion GIS Geographic Information System GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit GNU Government of National Unity IBTCI International Business & Technical Consultants, Inc. IOM International Organization for Migration KII Key Informant Interview LGCS Local Governance and Civil Society Program METAL Monitoring and Evaluation for Tunisia and Libya Activity MOLG Ministry of Local Governance NGO Non-governmental Organization PWD Person with Disabilities SOW Statement of Work TPM Third Party Monitoring UK United Kingdom UN United Nations UNDP United Nations Development Programme USAID United States Agency for International Development USG United States Government VIS Visible Improvement Service WB World Bank 1 1. EXECUTIVE SUMMARY Background On May 13, 2021, USAID Libya submitted to the Monitoring and Evaluation for Tunisia and Libya Activity (METAL) a statement of work (SOW) to conduct a midterm evaluation of the Local Governance and Civil Society Activity (LGCS), locally known as “Taqarib”, a $39.4 million, four-year initiative implemented by Development Alternatives Incorporated (DAI). Libyan context This Activity is taking place as Libya undergoes a turbulent transition from the former regime’s centralized dictatorship to a decentralized constitutional democracy. Several institutional changes have been put in place by the new government, among them the Ministry of Local Governance (MOLG) Law 59 of 2012, which aims to distribute authorities and the country’s oil wealth to the municipalities so they can better serve the public. LGCS is designed to support this transition and strengthen municipal governance. Evaluation Purpose The statement of work (SOW) for this evaluation (Annex 1) that USAID/Libya prepared and issued to METAL describes USAID/Libya’s need for a midterm evaluation of LGCS to inform the USAID Democracy and Governance (DG) team on how the United States Government (USG) can best support Libya’s democratic political transition during a domestic conflict and growing external influence. Recommendations will be used to inform the implementation of the existing Activity and inform future local governance activities, ensuring that USAID’s efforts are as effective as possible given the challenging operational environment. Evaluation Questions (EQs) The core evaluation questions specified in the SOW are as follows: 1. To what extent has the Activity improved citizen participation in and oversight of municipal decision￾making and service delivery processes? 2. To what extent has the Activity enhanced the responsiveness of local government institutions? a. How has the Activity contributed to improved municipal service delivery performance? b. To what extent have the Activity’s interventions affected public perceptions of local government performance and legitimacy? 3. How has the Activity contributed to Libya’s decentralization agenda? a. To what extent has the Activity enabled two-way national–local government coordination? 4. Has this Activity mitigated potential negative effects of decentralization (e.g., fostering further fragmentation along political/geographic/ethnic lines, further weakening, or impeding the development of Libya’s central state institutions)? Additional Evaluation Areas of Inquiry (AOI) During the evaluation team’s (ET) initial kickoff meeting with USAID, it was agreed to assess the role of LGCS in supporting the MOLG to implement the decentralization process and strengthen municipal government capacity, and the coordination of donor inputs1 to subnational governments. While these are not stated in the questions above, they are relevant to USAID’s overall objectives in Libya. Additional questions related to the Activity’s effectiveness addressed the relevance and appropriateness of the initiative’s overall design. These additional areas of interest (AOI) are as follows: 1: How appropriate and relevant is LGCS’ design? 2: How has LGCS supported the Ministry of Local Governance (MOLG)? 1Other major donors active in strengthening central and subnational governance in Libya include Deutsche Gesellschaft für Internationale Zusammenarbeit GmbH (GIZ), the European Union (EU), the World Bank (WB), the United Kingdom (UK), International Organization for Migration (IOM), and the various agencies of the United Nations (UN) system. 2 3: How are other donors supporting subnational governance in Libya? Evaluation Design and Limitations The ET gathered data and information from a desk review of Activity documents and other pertinent sources (see Bibliography, Annex 7), and collected data from 40 key informants, including Activity staff and representatives of civil society organizations (CSOs), municipalities, national government entities, and other donors and stakeholders. The evaluation relied on quantitative data on citizen perceptions of government from secondary sources but was largely qualitative based on interviews conducted remotely, which, due to COVID-19 and security concerns, significantly compounded the limitations normally found in this type of evaluation. Findings and Conclusions The findings and conclusions to the four main evaluation questions in the SOW are summarized in this section, while the findings, conclusions, and recommendations of all evaluation questions and the additional AOIs are found in the main body of the report. EQ 1: To what extent has the Activity improved citizen participation in and oversight of municipal decision-making and service delivery processes? Findings  An overall finding is that the Activity’s documentation, as well as informants’ answers to this question, seem to assume that “citizen participation and oversight of municipal decision-making and service delivery” is directly attributable to the quality of collaboration between organized CSOs and municipal authorities, which most municipalities and CSOs said was good. While this collaboration is important, the public interacts with municipal authorities individually and in informal groups as well as through CSOs. LGCS’ Objective 2 (above) mentions “citizens, CSOs, private sector actors, and other formal and informal stakeholders.” LGCS is exploring ways of broadening its approach to supporting citizen participation and oversight beyond working only with registered CSOs.  LGCS’ Champions for Change (C4C) program is intended to bring together members of the community to design and implement projects benefitting the community. The feedback from participants of C4C training, as reported in METAL’s site visit reports, all showed that participants learned how to engage with others in the community to develop projects and apply for C4C grants.  LGCS has established 56 citizen charters that are signed by the mayor, CSO representatives, and the LGCS leadership. These charters articulate LGCS’ objectives and assistance planned for the municipality and serve as a commitment among the signatories to strengthen municipalities’ management capabilities, improve citizen participation and transparency in municipal deliberations, and ultimately result in the delivery of more and better services. Prior to developing a Charter, LGCS consults with municipalities and CSOs to assess and determine their needs and interests.  Most of the municipal informants said they had good relationships with CSOs, and about half of the CSOs echoed this view.  CSO informants said LGCS was providing support for linkages with municipalities, which helped them implement projects that supported municipalities. However, these relationships were relatively new in the country’s political experience, so results thus far were limited but were expected to improve. CSOs were collaborating with municipalities on issues such as children’s rights, youth empowerment, COVID-19 response, improving services for persons with disabilities (PWDs), and addressing environmental issues. A few CSOs also fostered citizen participation in municipal governance and reported that municipalities felt they needed their support to strengthen their legitimacy and public image. 3  Informants also said there were problems with how LGCS provides support for CSOs and municipalities, such as a slow bureaucratic approval processes, which limits the Activity’s effectiveness and credibility in this area. Conclusions  Informants’ responses were mixed on the question of whether LGCS improved citizen engagement in municipal decision-making, indicating quite different conditions in the six communities in our sample, a diversity that is likely to be similar across the other communities in which LGCS works.  While there seem to be several cases of good collaborative working relationships between municipalities and CSOs, this was not universal and does not necessarily extend to strengthening “citizen oversight of municipal decision-making and service delivery processes.” That would require CSOs to do more than just collaborate with municipalities, such as attend public hearings, demand improvements in services and infrastructure, and comment on municipal budgets and procurements. CSOs’ relatively recent experience with participatory democratic governance and accountability processes may limit their ability to engage in a more meaningful way that would influence municipal authorities’ behavior. LGCS’ continued efforts to strengthen collaborative linkages among CSOs and municipalities and train CSOs in applying citizen engagement and oversight are likely to improve citizen participation in municipal decision-making and service delivery processes. EQ 2: To what extent has the Activity enhanced the responsiveness of local government institutions? a. How has the Activity contributed to improved municipal service delivery performance? b. To what extent have the Activity’s interventions affected public perceptions of local government performance and legitimacy? Findings  This question is similar to EQ 1 and, therefore, its findings and conclusions are similar. While the focus of LGCS Objective 2 was on supporting citizen engagement and advocacy related to holding municipalities accountable for service delivery, information from its FY 2021 Quarter 2 report shows it was early in the process of setting in place measures to achieve desired results. It was difficult to get a clear picture from CSO and municipal informants of the effectiveness of LGCS’ impacts on the responsiveness or performance of local government institutions. One CSO informant attributed noticeable changes in the municipality to Taqarib’s support; however, the informant believed more could be done. More critical CSO informants had quite different views. One reported that councilors did not respond to their efforts, and another said, “I don’t think Taqarib had any effect on the work of the local institutions.” CSOs also said LGCS should visit communities to better understand their needs. CSOs saw a need for improved communication and collaboration with municipalities and believed that LGCS should help bring CSOs and municipalities together to help formalize their relationship.  While municipal informants expressed appreciation for LGCS’ intentions and found its COVID￾related support helpful, more than half of the informants reported its assistance as being largely ineffective. It was described as “a couple of workshops and some training – simple stuff,” and informants wanted more in-depth and in-person (not remote) training and organizational development support, as was starting to be provided by other donors. These comments on training quality contravene the opinions of nearly all participants who provided feedback on their training experience to third party monitors (TPM) following each LGCS training event. As one municipal council informant commented, “I benefited from this workshop in two ways: for me as a member who works in the municipal council and as an activist in civil society organizations. I 4 took advantage of the experiences of the youth and from the presentation given by the trainers, as that was very useful information.”  They also said LGCS needed to better manage expectations, and to follow through and complete projects before starting others.  Citizens’ perceptions of municipal government in LGCS communities were only marginally better than those of non-LGCS communities, based on an LGCS public perception survey of 8,000 informants conducted by their subcontractor, Altai (Figure 3). This result was attributed in part to the public’s steadily increasing levels of dissatisfaction with government and the civil war, factors outside LGCS’ ability to control. Most municipal informants felt that the public had yet to see any visible impact of LGCS’ interventions to improve municipal government performance. As one informant commented, “Taqarib is working with us; however, they have not provided such an extent of support to the point that the population has noticed.” The one exception seemed to be in the Tripoli municipality, where an informant remarked that the impact of LGCS has been evident to the local population but that the population may not have attributed it to LGCS’ efforts. Conclusion While most LGCS inputs seemed to be directed toward improving municipality performance in terms of service delivery, the concept of “responsiveness” in governance was rarely noted in informant comments. LGCS was seen by most informants as well-intentioned and ambitious, but it had yet to deliver tangible change in municipal responsiveness as many inputs were either poorly executed, incomplete, or inadequate. These could be the result of LGCS not consulting sufficiently with municipalities on their requirements before implementing activities, and its inability to follow through on commitments made. While knowledgeable informants were aware of LGCS’ efforts, there was little demonstrable impact on citizens’ perception of municipal governance in LGCS municipalities, based on Altai’s public perception surveys. This could be attributed, in part, to a natural lag between the time that municipalities begin being more responsive to citizens and when citizens observe that responsiveness or comprehend that it’s the result of LGCS’ efforts. EQ.3: How has the Activity contributed to Libya’s decentralization agenda? a. To what extent has the Activity enabled two-way national–local government coordination? Findings  Informants described decentralization as a long-term, complex process due in part to the country having been highly centralized since 1951 and resistance to distribution of authority from the center to the periphery. LGCS’ support for decentralization included drafting several operational policies and procedures that municipalities need to manage delegated responsibilities and support national and subnational level dialogues on the decentralization process. MOLG is currently reviewing LGCS’ draft policies and procedures. Despite these efforts, two senior MOLG officials reported that LGCS has had minimal impact on improving municipal governance, thus far.  When CSO informants were asked to comment on how LGCS has contributed to Libya’s decentralization agenda, a number said they did not know. Some had positive views, noting that LGCS has helped reduce the gap between the municipality and national government, especially on PWD rights, and that its work will help improve the decentralization process and transfer of competencies to the municipalities. One informant said that they were working with MOLG on the decentralization issue without LGCS’ involvement. Of the six CSO informants who responded to the question of whether LGCS had enabled two-way national government coordination, one had a very good impression of LGCS’ impact, two were uncertain, and three saw no effect.  Municipal informants had mixed responses on this question. Several said that LGCS was either having no effect on the decentralization process or they were unaware of what it was doing to support decentralization. Other municipal informants suggested the central government needed 5 to communicate more with them through a structured protocol and establish a system to monitor the performance of municipal governments.  Two informants reported that LGCS was playing a constructive role in the process by working with several stakeholders to implement the decentralization agenda. Several municipal informants echoed this view stating that LGCS had a good relationship with both levels of government, facilitated good communication and cooperation between them, and helped with the decentralization agenda.  This lack of knowledge among CSO and municipal informants about LGCS’ efforts to support the decentralization process is, in fact, intentional as LGCS does not want to be viewed as the driver of the decentralization process but rather assist the MOLG and other ministries to lead and be seen by the public as leading the process. Conclusion LGCS was drafting procedural documentation that municipalities could use to implement responsibilities associated with decentralization, and conduct senior-level stakeholder consultations on the issue. The variety of responses from both CSOs and municipalities indicate that LGCS’ strategy to not publicize their role in the decentralization process is succeeding. Among the few that were aware of LGCS’ role, most reported that LGCS was playing a constructive role in furthering the decentralization agenda. EQ.4: Has this Activity mitigated potential negative effects of decentralization (e.g., fostering further fragmentation along political/geographic/ethnic lines, further weakening, or impeding the development of Libya’s central state institutions)? Finding  Of the five CSO informants who answered this question, none provided any information that directly or indirectly answered the question regarding whether LGCS had mitigated potential negative effects of decentralization. In general, CSO informants were more focused on the grants and services delivery within their municipalities. Municipal informants mentioned that LGCS operated in challenging times with Libya dealing with the COVID-19 pandemic and conflict, and this could have negatively affected LGCS’ impact in this area. One informant noted that the peer-to￾peer exchanges helped narrow the distance between the eastern and western regions and improved prospects for reconciliation. Conclusion The question of potential negative effects of decentralization – fragmentation, etc. – did not elicit significant data from informants, but the expansion of peer-to-peer exchanges between regions could facilitate political reconciliation. 6 Figure 1: LGCS summary infographics 7 8 2. INTRODUCTION In May 2021, USAID Libya submitted a statement of work (SOW) to the Monitoring and Evaluation for Tunisia and Libya (METAL) Activity to conduct a midterm evaluation of the Local Governance and Civil Society Activity (LGCS), a $39.4 million effort implemented by Develop Alternatives. This report provides the findings, conclusions, and recommendations of the evaluation2 of the LGCS Activity (also called Taqarib – Coming Together). Table 1: Summary Information Activity Name Local Governance and Civil Society Activity (LGCS) “Taqarib” USAID Office USAID/LIBYA Implementer(s) DAI Global, LLC Cooperative Agreement # 72067019CA00001 Total Estimated Ceiling of the Evaluated Activity $39.4M Life of Activity July 8, 2019–July 7, 2023 Active Geographic Regions 30 municipalities nationwide Development Objective(s) (DOs) Effectiveness of key Libyan governance institutions improved External or internal evaluation? External Background on Context and Activity Libyan Context Libya can be described as a conflict-affected, oil-dependent rentier state. It has many of the governance challenges associated with rentier states, 3 including problems with a lack of legitimacy and accountability to taxpayers. In the decade since the revolution of 2011, Libya has been transitioning from the Gaddafi regime’s centralized dictatorship to a decentralized constitutional democracy. An example of this effort is the Ministry of Local Governance’s Law 59 of 2012, which aims to distribute a range of authorities and budgets from the central government to subnational units, many of which have limited capacity to take on these responsibilities. While Law 59 includes governorates as an intermediate administrative level between central and local governments, they have yet to be established. The most significant subnational governance institutions in terms of services to the population are the approximately 100 municipalities,4 which operate with varying levels of effectiveness. Municipal elections were held – with international support – in 2014 and again in 2019, during an internal conflict, growing external influence, and political instability. The national context appears to have stabilized over the past two years, making it possible for USAID and other donors to increase their support for the development of Libyan government institutions, and specifically for USAID Libya to focus on helping municipalities provide improved services to the public and support the decentralization process. 2 Work on this evaluation started July 18, 2021. 3 See Verkoren & Kamphuis 2013 – State Building in a Rentier State. 4 There are some 97 cities ranging in population from 6,000 to Tripoli, with over 2M. 9 The population is weary from a decade of conflict and a weak political leadership. To help strengthen the foundations supporting a more unified Libyan state, the LGCS Activity aims to empower local governments to tangibly improve service delivery and become more responsive to citizens’ interests, reinforce community bonds, promote citizen engagement, and oversight in public dialogue, and prioritize public resource allocations. Some 30 municipalities have been selected for LGCS support. LGCS Objectives LGCS has the following four major objectives: 1. Strengthen the capacity of targeted municipal institutions to deliver services and be more responsive to citizen needs. 2. Build the capacity of citizens, CSOs, private sector actors, and other formal and informal stakeholders to advocate and hold municipal and related national-level institutions accountable to deliver on their mandates. 3. Strengthen national and subnational decentralization policies, systems, and coordination. 4. Inform programming through a robust learning and analysis agenda. Development Hypothesis The development hypothesis underlying LGCS is that it will support the foundation of a more unified Libyan state by prioritizing local-level activities that foster stability and accountable governance while reinforcing positive linkages among the national government, subnational government administration, and citizens. This is based on the benefits of fostering good governance, which assumes that stability and progress are associated with citizen perceptions of state legitimacy, accountability, citizen engagement, and effective administration and service provision. Causal Model The following graphic (from the FY 2021 LGCS Activity Monitoring, Evaluation, and Learning Plan/AMELP) is a representation of the causal model informing LGCS activities. The assumption is that the interventions will contribute to the outcomes and ultimately, to the Activity’s impact on the country. 10 Table 1. LGCS Causal Model Theory of Change The LGCS theory of change is as follows: “IF municipal institutions visibly improve the delivery and responsiveness of services (Objective 1) and coherence of national and subnational government policy, systems, and coordination (Objective 3); and IF Libyan men and women are empowered with the skills and capability to influence government action and hold the government accountable (Objective 2); THEN citizens’ trust and confidence in their government will improve, thereby promoting stability and peace and strengthening the foundations of a more unified Libyan state.” LGCS Municipalities LCGS is targeted to assist 30 municipalities and is now partnering with all 30, listed below. Abusliem Alabyar Albawanees Albayda Albrayga Alkhums Assahel Azzawya Bani Waleed Baten Aljabal Benghazi Daraj Ghadamis Sebha Shahhat Suq Alkhamees Swani Bin Adam Tajoura Tripoli Center Ubari Wadi Etba Yefren Zliten Zwara 11 Figure 2. Libya Map and LCGS Partner Municipalities (FY 2021 Quarter 2) Objectives and Indicators A complete list of the indicators measuring and tracking the progress toward meeting LGCS objectives is included in the Performance Indicator Tracking Table in Annex 2. Objective 1: Strengthen the capacity of targeted municipal institutions to deliver services and be more responsive to citizen needs. Indicators: 1.1: Number of non-binding implementation agreements with municipalities 1.2: Percent change over the baseline score in the Municipality Performance Index (MPI) of partner municipalities 1.3: Number of Visible Improved Service (VIS) processes supported in partner municipalities 1.4: Number of citizen priorities reflected in municipality VIS plans 1.5: Number of participants enrolled in in-service trainings, educational courses, or other professional development opportunities with Taqarib-supported content 1.6: Number of individuals with improved skills following completion of USG-assisted workforce development programs 1.7: Number of municipality-led multi-stakeholder meetings, town halls, or similar events held 1.8: Number of individuals who participate in peer-to-peer learning exchanges 1.9: Number of host country COVID-19 preparedness and/or response plans in various phases (formally proposed, adopted, or implemented) supported with USG assistance 1.10: Number of private sector partnerships utilized addressing second-order COVID-19 impacts 1.11: Number of participants in USG-supported activities trained in community emergency response 12 Objective 2: Build the capacity of citizens, civil society organizations (CSOs), private sector actors, and other formal and informal stakeholders to advocate and hold municipal and related national-level institutions accountable to deliver on their mandates. Indicators: 2.1: Number of CSOs receiving USG assistance engaged in advocacy interventions 2.2: Percent change in Organization Performance Index (OPI) scores over baseline of partner CSOs 2.3: Percent of USG-assisted organizations with improved performance 2.4: Number of CSO-led multi-stakeholder meetings, town halls, or similar events held as part of municipal planning, budgeting, oversight, and other service delivery processes 2.5: Number of grants awarded to advocate for accountability or hold government institutions accountable 2.6: Percentage change in social accountability monitoring scores 2.7: Number of participants in the Champions for Change (C4C) program Objective 3: Strengthen national and subnational decentralization policies, systems, and coordination. Indicators: 3.1: Number of public policies introduced, adopted, repealed, changed, or implemented with citizen input 3.2: Number of Taqarib-supported multi-stakeholder dialogues among national actors regarding the decentralization agenda 3.3: Number of decentralized business processes mapped and analyzed 3.4: Number of national advocacy initiatives supported by Taqarib 3.5: Percent change in citizen perceptions regarding advancement of the decentralization agenda Objective 4: Inform programming through a robust learning and analysis agenda. Indicators: 4.1: Number of municipalities for which profiles have been completed and for which service-delivery and public perception data is tracked over time 4.2: Number of strategic review sessions conducted 3. EVALUATION PURPOSE AND EVALUATION QUESTIONS Evaluation Purpose The purpose of this midterm performance evaluation is described in USAID’s SOW (see Annex 1) as follows: “To inform the USAID Democracy and Governance team on how the USG can best support Libya’s democratic political transition in the midst of conflict and growing external influence. Recommendations will be used to inform the implementation of the existing Activity and future local governance activities, ensuring that USAID’s efforts are as effective as possible given the challenging operational environment.” 13 Evaluation Questions The SOW for the LGCS midterm evaluation includes four main evaluation questions: EQ1: To what extent has the Activity improved citizen participation in and oversight of municipal decision￾making and service delivery processes? EQ2: To what extent has the Activity enhanced the responsiveness of local government institutions? a. How has the Activity contributed to improved municipal service delivery performance? b. To what extent have the Activity’s interventions affected public perceptions of local government performance and legitimacy? EQ3: How has the Activity contributed to Libya’s decentralization agenda? a. To what extent has the Activity enabled two-way national–local government coordination? EQ4: Has this Activity mitigated potential negative effects of decentralization (e.g., fostering further fragmentation along political/geographic/ethnic lines, further weakening, or impeding the development of Libya’s central state institutions)? Additional Evaluation Areas of Inquiry (AOI) During the initial kickoff meeting, USAID agreed to explore additional areas related to the coordination of donor inputs to strengthen subnational governments, the relevance of the Activity’s design, and level of LGCS support to the MOLG. As a result of these discussions USAID added the following areas of interest to the evaluation: AOI 1: How appropriate and relevant is the Activity’s design? AOI 2: How is the Activity supporting MOLG? AOI 3: How are other donors supporting subnational governance in Libya? 14 Comparing Evaluation Questions and LGCS Objectives Table 2 below presents the alignment of the evaluation questions to LGCS’ objectives. Table 2. Comparing LGCS Objectives and Evaluation Questions Evaluation Questions LGCS Objectives 1. To what extent has the Activity improved citizen participation in and oversight of municipal decision-making and service delivery processes? 1. Build the capacity of citizens, civil society organizations (CSOs), private sector actors, and other formal and informal stakeholders to advocate and hold municipal and related national-level institutions accountable to deliver on their mandates. 2. To what extent has the Activity enhanced the responsiveness of local government institutions? 2. Strengthen the capacity of targeted municipal institutions to deliver services and be more responsive to citizen needs. a. How has the Activity contributed to improved municipal service delivery performance? b. To what extent have the Activity’s interventions affected public perceptions of local government performance and legitimacy? 3. How has the Activity contributed to Libya’s decentralization agenda? 3. Strengthen national and subnational decentralization policies, systems, and coordination. a. To what extent has the Activity enabled two-way national–local government coordination? 4. Has this Activity mitigated potential negative effects of decentralization (e.g., fostering further fragmentation along political/geographic/ethnic lines, further weakening, or impeding the development of Libya’s central state institutions? AOI 1: How appropriate and relevant is the Activity’s design? AOI 2: How is the Activity supporting MOLG? 3. Strengthen national and subnational decentralization policies, systems, and coordination. AOI 3: How are other donors supporting subnational governance in Libya? 4. Inform programming through a robust learning and analysis agenda. 15 4. EVALUATION DESIGN, METHODS, AND LIMITATIONS Data Collection Methods The ET collected information from five main sources: Key Informant Interviews (KIIs), and four secondary sources: a desk review of primarily LGCS documentation, METAL’s TPM site visit reports of DAI events, LGCS citizen perception survey data, and a review of relevant literature (see Annex 7 – Bibliography). As noted in the limitations section below, planned Focus Group Discussions (FGDs) were not conducted. Primary Data Sources The evaluation’s primary data sources were KIIs with LGCS staff, MOLG officials, donors working on subnational governance, CSOs, and municipal government personnel in six municipalities selected for their location, level of operating performance, and population diversity. LGCS facilitated municipality selection and key informant identification. Municipalities Selected The ET selected informants from six of LGCS’ 30 municipalities, two from each region with different levels of functionality (low, medium, and high), and two having minority populations. This was consistent with a stratified sampling approach (see methodology below). LGCS and the ET selected the municipalities based on LGCS’ experience working with the 30 municipalities in the country, and their perceptions of their relative levels of performance, and presence of minorities. Table 3. Municipalities Selected for Evaluation Municipal Council Region LGCS Perceived level of Functionality Comments Sabha South High Ubari South Low Includes minority (Tuareg and Tebu) Tripoli Center West High Zwara West Medium Includes minority (Amazigh) Assahel East Medium-low Benghazi East High The ET conducted a total of 40 KIIs, distributed as follows:  CSOs: 11  LGCS/DAI: 12, including 2 Altai subcontractors  Ministry of Local Government: 2  Municipality administration: 5  Municipality elected: 5  USAID: 1  Donors: 4 Although the ET made efforts to interview at least three informants from each municipality – elected and administrative personnel, and a CSO representative – this distribution was not possible due to several factors that were not in the ET’s control and are noted in the limitations section. These factors included difficulty in contacting identified KIIs, lack of interest in participating, and communication challenges. The list of informants can be found in Annex 5 – Sources of Information. 16 Table 4. Key Informant Groups and Locations Informant Type Number Interviewed Informant Locations CSO 11 Tripoli, Sabha, Zintan,* Benghazi, Bani Waleed,* Aubari Municipal Administration 5 Sahel, Sabha, Benghazi, Zwara Elected Municipal Member 5 Sabha, Tripoli, Aubari, Zwara MOLG Official 2 Tripoli DAI/LGCS 12 Tripoli, Tunis, Benghazi, Sabha USAID Official 1 Frankfurt Donor 4 Tunis * Grant recipient Evaluation Management The evaluation team consisted of four members: one expat, two Libyans, and one Tunisian member of the METAL staff. See Annex 6 for biographies and contact information of team members. Table 5. Team Member Roles and Responsibilities Name Responsibilities Andy Tamas  Lead the evaluation, including desk review, data collection and analysis, and report writing  Manage the team, including delegating responsibilities, training, and guiding team members, monitoring progress, and providing feedback on inputs  Serve as the primary liaison with USAID/ Libya and lead presentations Hala T. Bugaighis  Data collection  Translation Tamim Rashed  Data collection  Translation Ahmed Trabelssi  Data collection  Technical support  Report writing Data Collection For data collection the ET used the “long interview” technique (McCracken, 1988). This technique allows research participants to reflect upon the domain of discourse in a relatively unbiased and free￾flowing way. It involves asking two main types of questions: “grand tour” [general and nondirective, such as ‘could you tell me about how (name the ORGANIZATION) got started?’] and “planned prompts” [allows the researcher to delve into specific subjects]. See details on data collection below. Sampling Approach/Strategy The ET adopted a stratified sampling technique to capture the perspectives and experiences of key stakeholders involved in LGCS activities. Considering the time limitation of this evaluation, this approach enabled the team to prioritize interviewees who were the most “informative” and convenient, 5 while increasing representativeness. As a result, the evaluation relied on a snowball sampling technique, which was based on who was available and engaged in the LGCS Activity to find the respondents. The strata are the reported degree of functionality of a municipality’s governance6 (i.e., high, medium, and low) while clusters are municipalities within each region (i.e., east, west, and south). 5 Those who are conveniently located around a location or who have access to internet service. 6 Identified based on LGCS experience and local interviewers’ knowledge of municipality performance. 17 Research Methods (Data Collection and Analysis Details) Key Informant Interviews (KIIs) The ET conducted all KIIs remotely by mobile phone or media such as WhatsApp, Google Meet, or Zoom. Interviews with Libyan CSOs, municipalities, LGCS and government informants used a semi￾structured interview guide (see Annex 3) with closed and open-ended questions to steer the conversation. Interviewers were prepared to “drill down” on specific matters. Teams of two researchers conducted interviews – one engaged the informant in conversation while the other listened, took notes, and recorded the conversation. The interview team subsequently reviewed these audio recordings and translated them into English. They provided transcripts to the Team Leader for coding and analysis. The Team Leader conducted interviews with USAID, donors, and the LGCS COP in English. Focus Group Discussions (FGDs) The ET’s plans to conduct FGDs were not successful because identified informants were not interested in participating in a virtual group meeting. Secondary Data Sources The ET drew desk review data and information primarily from LGCS Activity files. Specific LGCS documents provided by USAID and the IP for this desk review included the LGCS contract and modification, annual work plans, the Activity Monitoring and Evaluation Plan (AMELP), and quarterly reports. The ET also reviewed technical papers, articles, documents, and reports provided by other agencies and derived from internet research as appropriate (see Annex 5: Sources of Information and Annex 7: Bibliography). Data Analysis and Synthesis The methodology for the desk review (analysis of documentation) was to gather available and relevant literature which the ET then reviewed to develop a comprehensive context for the Activity’s evaluation. The ET reviewed LGCS and other relevant documents during the inception phase and during fieldwork as the team obtained new documents. The ET coded and grouped data from CSO, municipality, LGCS, and MOLG interview transcripts by informant category and evaluation questions and printed out database reports for review to identify major themes in the text. The ET used a similar process for interviews with USAID and donor informants. Summarized coded transcripts are reported in the findings sections of this report. (See sample coded interview text and code list in Annex 4) Data Quality and Limitations This section highlights evaluation limitations, their implications for the evaluation, and how the ET mitigated them. Remote contact – COVID restrictions, poor connectivity Although conducting interviews in person is preferred, unfortunately, this was not possible with this evaluation. Due to travel restrictions caused by the COVID-19 pandemic and security issues, all project communication was remote, using tools such as Google Meet, Zoom, WhatsApp, Skype, or mobile phones. Significant challenges with connectivity issues and power outages contributed to informant fatigue and reduced the amount and quality of data the team collected. The limited mitigation measures available included conducting frequent (sometimes thrice-weekly) virtual team meetings. The ET also selected the questions asked of key informants from the long questionnaire to limit informant fatigue, which allowed informants to focus their responses on issues they saw as high priority and could readily discuss. 18 Government permission required – delayed municipal interviews As evaluation work got underway, the government issued directives to municipalities limiting their ability to interact with foreign donor agencies. Municipality-based informants required letters of permission before they were free to talk with ET members. This contributed to delays in the evaluation schedule. Mitigation measures included seeking timely distribution of letters of permission, and interviewing available informants from CSOs, LGCS, MOLG, and USAID early in the evaluation schedule, while waiting for municipality-based informants to receive permission to be interviewed. Informant engagement challenges Some informants identified by LCGS as being suitable for interviews were reluctant to participate, and potential FGD members were not interested in taking part in remote group interviews. This affected the number and range of sources of information for the evaluation and forced the cancelation of FGDs as noted above. Mitigation measures included identifying alternate informants in the affected municipalities. Incomplete use of interview guide Due in part to connectivity problems and the desire to limit informant fatigue, not all informants in a particular category were asked all the questions in the interview guide meant for their respondent group. This contributed to incomplete data collection, a limitation that was mitigated by doing whatever possible to ask multiple informants in the same categories the most relevant questions in the guide. Also, at the end of each interview, informants were asked if they had any other information to share, which could elicit relevant data that had not been provided earlier. Typical qualitative intercultural action research limitations Limitations associated with qualitative, intercultural action research include accuracy and completeness factors with translated interview data, possible coding errors in analysis, and selectivity in identifying patterns in the coded data to include in the report’s findings. Mitigation measures included having two interviewers working together to collect, record, and analyze Arabic interview data and subsequent translation into English for coding by the team leader. Both interviewers of local respondents were Libyan and fluent in English and Arabic and had considerable experience with development-related evaluation research. The ET mitigated the likelihood of coding errors in analysis by multiple reviews of the coded raw text associated with numbered questions in the interview guide (Annex 3) as required to limit errors where possible. The Team Leader controlled for errors in selecting parts of the coded text to incorporate in the report’s findings by assigning multiple codes to the same body of text when relevant so it would appear more than once in the database reports used as the basis of the findings section. This limited the likelihood that key information would be overlooked. Bias risks There are several noteworthy bias risks in data collection and analysis:  Recall bias occurs when interviewees may not recall events and experiences clearly, if at all. To mitigate this the ET conducted as many KIIs in each municipality as possible to triangulate responses to increase the validity of the evaluation findings, and probed respondents’ answers to ensure accurate recollections.  Response bias is the risk that key informants may be motivated to provide responses that would be considered socially desirable. The ET mitigated this risk by explaining evaluation requirements clearly at the beginning of each interview.  Selection bias is an inherent risk when the Activity implementer refers the ET to more sympathetic and supportive counterparts or recipients. To mitigate this risk, the ET identified persons having 19 some involvement with LGCS and arranged interviews directly without LGCS assistance. The ET sought LGCS assistance to connect with targeted informants only when they were hard to reach. Gender considerations Social inclusion (for women, girls, youth, etc.) was not a separate component of review, but rather a cross-cutting theme addressed in the evaluation. Efforts were made to ensure full participation by women in interviews, most of which were carried out by a female ET member. LGCS Performance Indicator Results Summary As part of its desk review, the ET analyzed LGCS’ performance indicators, which showed that of its 25 indicators in Year 1, 10 met or exceeded targets, five fell short, nine reported no data, and one had no target. In Year 2, nine met or exceeded their targets, nine fell short, four reported no data, and three had no targets. The complete LGCS Indicator Tracking Table for Years 1 and 2 follows below and is also included in Annex 2. 20 Indicator Baseline FY 1 Target FY1 Q1 FY1 Q2 FY1 Q3 FY1 Q4 FY 1 Actual FY 2 Target FY2 Q1 FY2 Q2 FY2 Q3 FY2 Q4 FY 2 Actual 1.1: Number of non -binding implementation agreements with municipalities 0 15 0 3 7 8 18 30 0 8 14 6 28 1.2: Percent change over the baseline score in the Municipality Performance Index (MPI) of partner municipalities 0 0 0 3 14 1 18 0 0 8 4.8% 6,6% 6.6% 1.3: Number of VIS processes supported in partner municipalities 0 15 0 3 7 8 18 30 0 8 14 6 28 1.4: Number of citizen priorities reflected in municipality VIS plans 0 15 0 2 7 8 17 30 0 8 14 6 28 1.5: Number of participants enrolled in in -service trainings, educational courses, or other professional -development opportunities with Taqarib -supported content 0 50 0 0 18 19 37 50 60 95 261 108 524 1.6: Number of individuals with improved skills following completion of USG -assisted workforce development programs 0 15 0 0 0 0 0 15 13 9 x x x 1.7: Number of municipality -led multi -stakeholder meetings, town halls, or similar events held 0 10 0 0 6 7 13 30 2 20 11 6 28 1.8: Number of individuals who participate in peer - to -peer learning exchanges 0 50 0 0 0 14 14 50 28 15 28 25 96 1.9: Number of host -country COVID -19 preparedness and/or response plans in various phases (formally proposed, adopted, or implemented) supported with USG assistance 0 5 0 0 0 1 1 15 13 3 0 8 16 1.10: Number of private sector partnerships utilized addressing second -order COVID -19 impacts 0 5 0 0 0 0 0 15 0 0 0 0 0 1.11: Number of participants in USG supported activities trained in community emergency response 0 50 0 0 0 127 127 600 187 248 95 211 741 2.1: Number of civil society organizations (CSOs) receiving USG assistance engaged in advocacy interventions 0 15 0 0 0 3 3 30 0 3 3 3 9 2.2: Percent change in Organization Performance Index scores over baseline of partner CSOs 0 0 0 0 0 0 0 0 0 0 0 0 0 21 Indicator Baseline FY 1 Target FY1 Q1 FY1 Q2 FY1 Q3 FY1 Q4 FY 1 Actual FY 2 Target FY2 Q1 FY2 Q2 FY2 Q3 FY2 Q4 FY 2 Actual 2.3: Percent of U.S. Government-assisted organizations with improved performance 0 0 0 0 0 0 0 0 0 0 31% 29% 29% 2.4: Number of CSO-led multi-stakeholder meetings, town halls, or similar events held as part of municipal planning, budgeting, oversight, and other service delivery processes 0 5 0 0 12 0 12 10 3 2 0 0 5 2.5: Number of grants awarded to advocate for accountability or hold government institutions accountable 0 5 0 0 0 0 0 5 0 0 0 0 0 2.6: Percentage change in social accountability monitoring scores 0 0 0 0 0 0 0 0 0 0 0 0 0 2.7: Number of participants in the Champions for Change (C4C) program 0 150 0 0 60 119 179 150 0 0 46 37 83 3.1: Number of public policies introduced, adopted, repealed, changed, or implemented with citizen input 0 1 0 0 0 0 0 1 1 0 4 7 12 3.2: Number of Taqarib-supported multi￾stakeholder dialogues among national actors regarding the decentralization agenda 0 2 1 0 0 1 2 2 1 1 0 0 2 3.3: Number of decentralized business processes mapped and analyzed 0 3 0 0 0 0 0 3 0 1 1 1 4 3.4: Number of national advocacy initiatives supported by Taqarib 0 3 0 0 0 3 3 3 0 2 1 1 4 3.5: % change in citizen perceptions regarding advancement of the decentralization agenda 0 0 0 0 0 0 0 0 0 0 4.1: Number of municipalities for which profiles have been completed and for which service delivery and public perception data is tracked over time 0 15 0 0 18 0 18 30 0 0 6 8 12 4.2: Number of strategic review sessions conducted 0 4 2 0 0 0 2 4 0 1 1 1 3 Objective 1: Of the 11 indicators, 5 exceeded targets, 3 fell short, 2 had no data, and 1 had no target. 22 Objective 2: Of the 7 indicators, 2 exceeded targets, 1 fell short, and 4 had no data. Objective 3: Of the 5 indicators, 2 met their targets, and 3 had no data. Objective 4: Of the 2 indicators, 1 exceeded its target, and 1 fell short. LGCS Year 2 Results Summary Objective 1: Of the 11 indicators, 5 exceeded targets, 4 fell short, 1 had no data, and 1 had no target. Objective 2: Of the 7 indicators, 3 fell short, 3 had no data, and 1 had no target. Objective 3: Of the 5 indicators, 3 exceeded targets, 1 met the target, and 1 had no target. Objective 4: Of the 2 indicators, both fell short. 5. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS The ET analyzed the collected information to identify fact-based quantitative and qualitative evidence, which is presented as concise findings in this section of the report. Conclusions are a synthesis or interpretation of the findings, while recommendations are proposed actions drawn from specific findings. EQ 1: To what extent has the Activity improved citizen participation in and oversight of municipal decision-making and service delivery processes? Because CSO–municipality engagement was relatively new in the country’s political experience, results in terms of citizen oversight efforts were limited. Findings For this question, the ET interviewed 11 CSO informants from 6 communities, 8 municipal informants from 6 municipalities, and LGCS staff.  LGCS has established 56 citizen charters that are signed by the mayor, CSO representatives, and the LGCS leadership. These charters serve as a commitment among the signatories to strengthen municipalities’ management capabilities, improve citizen participation and transparency in municipal deliberations, and ultimately result in the delivery of more and better services. LGCS continues to expand the citizen charters to additional municipalities.  CSO informants said LGCS was providing them financial and capacity development support to strengthen their operations and improve their relationships with municipalities, which helped them implement their own projects that served the public and increased their legitimacy.  LGCS’ Indicator Progress Update of 2021.10.07 (Annex 2) that reports on this evaluation question shows actuals falling below targets, indicating that LGCS thus far has had difficulty engaging CSOs in accountability and oversight matters. Relevant indicators, targets, and actuals are shown in the following table: Table 6. LGCS Objective 2 CSO Oversight Indicators and Cumulative Results as of Q4 FY 2021 Indicator Target FY 2021 Achievement as of FY 2021 2.1: Number of civil society organizations (CSOs) receiving USG assistance engaged in advocacy interventions 30 9 23 Indicator Target FY 2021 Achievement as of FY 2021 2.4: Number of CSO-led multi-stakeholder meetings, town halls, or similar events held as part of municipal planning, budgeting, oversight, and other service-delivery processes 10 5 2.5: Number of grants awarded to advocate for accountability or hold government institutions accountable 5 0 2.6: Percentage change in social accountability monitoring scores 0 0  A few CSOs reported fostering citizen participation in municipal governance (for example encouraging youth to observe and comment on municipal governance processes) and reported that municipalities felt they needed their support to strengthen their legitimacy and public image.  LGCS was helping CSOs increase their ability to collaborate with municipalities on issues such as children’s rights, youth empowerment, COVID-19 response, improving services for PWDs, and addressing environmental issues.  Although municipality representatives did not provide clear information about CSO oversight or advocacy efforts, most of the informants said they had good relationships with CSOs, and about half of the CSOs echoed this view. ● The Activity’s documentation, as well as informants’ answers to this question, seem to assume that “citizen participation and oversight of municipal decision-making and service delivery” is primarily the product of collaboration between organized CSOs and municipal authorities. It does not consider that the public also interacts with municipal authorities individually and in informal groups as well as through CSOs. LGCS’ Objective 2 mentions “citizens, CSOs, private sector actors, and other formal and informal stakeholders.” LGCS is exploring ways of broadening its approach to supporting citizen participation and oversight beyond working only with registered CSOs. ● LGCS’ Champions for Change (C4C) program is intended to bring together members of the community to design and implement projects benefitting the community. The feedback from participants of C4C training as reported in METAL’s site visit reports all showed that participants learned how to engage with others in the community to develop projects and apply for C4C grants. As one training participant commented, "I learned more details about the grant, how to apply for it and how the winner should follow-up after receiving the grant." ● CSO informants also said there were problems with how LGCS provides financial and programming support for CSOs, such as a slow bureaucratic approval process for grants, which limits CSOs’ effectiveness to engage with municipalities and on projects they implement. They attributed this problem more to USAID than to LGCS. Conclusions  While there seem to be several cases of good collaborative working relationships between municipalities and CSOs, this was not universal and does not necessarily extend to strengthening “citizen oversight of municipal decision-making and service delivery processes.”  LGCS’ efforts to increase citizen oversight and public engagement with municipality decision￾making have fallen short of meeting the Activity’s performance targets.  This line of inquiry seemed foreign to both the interviewers and informants, which can be attributed in part to having a history of no substantive public engagement in governance under the previous dictatorship and most CSOs being relatively new actors on the governance scene.  Strengthening citizen oversight would require CSOs to do more than just collaborate with municipalities, such as attend public hearings, demand new and/or improvements in services 24 and infrastructure, and comment on municipal budgets and procurements. Citizens’ relatively nascent experience participating in democratic governance and accountability processes may limit their ability to engage in these more meaningful ways that would influence municipal authorities’ behavior.  LGCS’ continued efforts to strengthen democratic processes, foster collaborative linkages among citizens and municipalities, support networking among municipalities and among CSOs, and train CSOs in applying citizen oversight are likely to improve the public’s oversight and service delivery engagement processes in the 30 municipalities in which LGCS operates. Recommendations LGCS (and USAID) should do the following: 1. Help broaden CSO collaboration with municipalities to include CSO participation in and “oversight” of municipal governance. 2. Expand LCGS’s support for citizen engagement and oversight of municipal governance from organized CSOs to include other less structured components of civil society. 3. Continue to support networking among municipalities and CSOs to foster sharing of experiences and peer-to-peer learning opportunities across the country. 4. Expand the use of citizen charters to all 100 municipalities and broaden the number of signatories to each charter by soliciting additional local CSOs and private businesses to join. EQ 2: To what extent has the Activity enhanced the responsiveness of local government institutions? For the following questions, the ET interviewed 11 CSO informants from 6 communities, 10 municipal informants from 6 municipalities, and 1 former USAID Libya DG Officer, and reviewed relevant LGCS documentation. Findings  The ET had difficulty getting a definitive picture from CSO and municipality informants of the effectiveness of LGCS’ efforts on the responsiveness of local government institutions. One of the few CSO informants who addressed this issue reported they had a very good experience with LGCS, saying, “Taqarib is working well on informing and providing awareness campaigns for the population to learn what the responsibility of the Municipalities are and the services they can provide the residents.”  More critical CSO informants had quite different views, one reporting that councilors did not respond to their efforts and that LGCS was having minimal impact on municipality performance (discussed further in the next section).  Altai consultants’ systematic surveys of 8,000 informants in 40 municipalities, of which 30 were LGCS municipalities and 10 were a control group of non-LGCS municipalities, reported citizens’ perception of responsiveness of municipal government in LGCS communities was marginally better than their perceptions of non-LGCS municipalities in Years 1 and 2, but in year 2, the difference in perceptions between LGCS municipalities and non-LGCS municipalities fell slightly, as seen in figure 3 below. 25 Figure 3. Altai Research Findings on Municipality Responsiveness to Citizens’ Needs Conclusions  While LGCS was making efforts to influence municipalities’ accountability to citizens, the concept of “responsiveness” was not widely noted in informants’ comments.  The low level of key informant responses on this issue was attributed in part to the country’s relatively recent experience with participatory democracy.  LGCS municipalities showed improvement in government responsiveness to citizen needs compared with the control group. The level of difference between LGCS municipalities and others in this area should continue to widen as LGCS’ assistance to CSOs and municipalities is fully institutionalized by those organizations. Recommendations 1. Ensure the alignment of LGCS’ efforts to foster municipal governments’ responsiveness to citizens with the reality of the population’s limited experience with participatory democracy. 2. Continue to provide municipal government stakeholders with capacity development training and organizational development procedures (such as establishing and working with citizens’ advisory committees) to help them be more overtly responsive to the public’s needs. 3. Provide CSOs and other elements of civil society with tools to express their needs in helpful ways that municipal authorities can appreciate and respond to as effectively as possible. 4. Design and implement public education programs to foster a spirit of collaboration between citizens and their municipal governments and manage the public’s expectations of their municipalities’ ability to respond to their needs. 26 EQ 2a. How has the Activity contributed to improved municipal service delivery performance? Findings  Virtually all municipality informants reported that LGCS’ impact on service delivery was minimal and unevenly distributed. Sample complaints included failure to complete work on projects before starting on other projects: “They need to stick to one project and complete it from A-Z, rather than what they are currently doing, which is basically doing a little thing here and a little thing there.” One respondent complained about equipment the municipality did not really need, such as a third Bobcat loader. There were also complaints that LGCS support was inadequate – a few brooms, wheelbarrows, and buckets. “We are not a poor country,” said one respondent. “[LGCS] needs to understand what we need.” A municipal informant from Sabha also criticized LGCS for its lack of material support, noting that the United Nations Development Programme (UNDP) was providing 1,200 solar panels to Sabha while LGCS provided just 30. One municipal informant remarked, “There was a project on supporting the road repair. They did provide us with machinery, but [it is] not suitable for the job. They gave us a small generator to power the machinery, [but] it is too small; they gave us a rolling machine, which is also too small for our requirements. I was planning to have our own road repair team rather than contracting it out. However, I am still using contractors to conduct road repairs as the items provided are just not suitable.”  These negative comments may be attributable to LGCS’ deliberate implementation strategy to layer visible improvement in services projects across multiple waves. This was to accommodate enormous implementation challenges brought on by conflict and the COVID￾19 pandemic. For example, Taqarib did not want a vendor’s problem with international shipping for an item to delay progress that Taqarib could make on another project. Taqarib acknowledges that the layered approach may frustrate certain counterparts in the short term but feels that these perceptions will improve by the end of the project, at which point the layered approach will have left behind much more tangible benefit than one that was purely sequential.  Most municipality informants were disappointed with the low level of material support received from LGCS, perhaps due to their unrealistic expectations not being met and not being consulted on the assistance to be delivered. One said that when they were introduced to LGCS in Tunis, the impression they were given was that there would be much more support than they had received. They were also critical that LGCS did not consult with the municipalities on their needs before developing a work plan and delivering assistance. As one municipal informant from Sahba commented, “Taqarib doesn’t know our requirements, and I do not know their capabilities as an organization. How can we work together on any project with this kind of arrangement? I need to know their capabilities and Taqarib needs to know our requirements as the starting point. This is not happening.” This finding is surprising given that before LGCS begins working in any municipality, it consults with municipalities and CSO representatives before developing and signing a Citizen’s Charter which specifies all the assistance that Taqarib commits to provide each municipality, including material equipment and supplies.  CSOs also said LGCS should visit communities to better understand their needs. CSOs saw a need for improved communication and collaboration with municipalities. As one CSO representative commented, “I think USAID can support us in building our relationships with the other municipalities and bridge that gap between the CSOs and the municipalities. Having USAID as a partner would be a good way also to really let USAID show their work on the ground.” 27  While most CSO respondents reported that LGCS material inputs had little or no impact on municipal service delivery, some considered LGCS to have had a positive impact on training of municipality staff by helping staff understand what their responsibilities were in serving the local community. They saw municipality staff as accepting of LGCS inputs but believed that LGCS could engage further to support the municipality. A CSO informant said, “I have very good experience with Taqarib, and I can say that the Taqarib is really an advanced NGO [nongovernmental organization] in comparison to the others in the country.” He said its impact could be further improved by reaching out to people who do not use social media, such as the elders. He attributed noticeable changes in the municipality to Taqarib’s support and other efforts.  While municipality informants expressed appreciation for LGCS’ intentions and found its COVID-related support helpful, informants wanted more in-depth and in-person (not remote) training and organizational development support, as was starting to be provided by other donors.  Two municipal council informants from Tripoli had diverse views. One stated that LGCS should be training elected officials and that the training needs to be more practical and in person rather than remote. The other member found the training to be useful and effective and appreciated how LGCS was bringing municipalities together to learn from one another.  Municipality informants had much to say about how LCGS could improve its effectiveness. They mentioned the need to manage expectations, to be more collaborative and strategic, to communicate more with beneficiaries early in the planning process, and the need to follow through and complete projects such as the installation of geographic information systems (GIS) in the municipalities. They also said LGCS should offer training in administrative skills such as budgeting and managing municipality affairs and should also provide a stipend for maintenance support. There was also a suggestion from al Sahel that LGCS open a local office to improve services: “We are communicating to the Benghazi office, [but] this is not right. We should have our own office with Taqarib. I think it does not work to have one Taqarib office for the whole of the east. There need to be more offices and one in the Green Mountains.” Conclusions  While there were positive comments from CSO and municipal informants about some of LGCS’ initiatives, such as training, many responses were negative. While the Activity was seen by most informants as well intentioned and ambitious, most felt it had yet to deliver tangible change. There were several comments that material inputs were either poorly executed, incomplete, or inadequate. These could be the result of LGCS not consulting sufficiently with municipalities on their requirements before implementing activities, and their inability to follow through on commitments made.  Although LGCS was not designed to be a provider of substantial equipment to support the infrastructure needs of municipalities, some of its partner municipalities expected it to provide more material support than it has thus far. This is surprising given that, prior to delivering any material assistance to a municipality, LGCS provides each municipality a menu of options from which municipalities and their residents can select and discuss.  There is a significant mismatch between CSO and municipality expectations of LGCS assistance and the level of support LGCS was designed to provide that is affecting CSO and municipality perceptions of LGCS effectiveness.  Municipalities and CSOs felt that LGCS needed to follow through and complete projects, provide more significant and practical support, become more familiar with conditions on the ground, foster CSO and municipality linkages and exchanges with other communities, and, above all, improve communication and collaboration to ensure inputs are well suited to the beneficiaries’ needs and contexts. 28  Participants did not like remote training and preferred in-person assistance that other donors were starting to provide. Recommendations 1. Become more familiar with conditions on the ground and improve communication and collaboration with partner municipal governments and CSOs with a view to providing more relevant and practical support. 2. Follow through and complete municipal projects before starting another. 3. Better document and share the results of LGCS inputs in terms of municipality performance improvement with partner organizations. 4. Consider shifting back from remote to in-person training methods as the COVID-19 and security situation improves. 5. Better manage expectations by meeting with partner municipal governments and CSOs and sharing annual work plans and objectives so that beneficiaries clearly know what they can expect LGCS to deliver. EQ 2b. To what extent have the Activity’s interventions affected public perceptions of local government performance and legitimacy? Findings  According to the Altai public perception survey, in Year 1 citizens who considered the quality of public service delivery to be good or average was six (6) percentage points higher in LGCS municipalities than in the 10 non-LGCS municipalities, but the difference between the municipalities dropped to 1 point in Year 2. Altai researchers noted that this might be associated with a general decrease in the public’s perception of overall government performance over the previous months. Figure 4. Altai Research Findings on Quality of Municipality Service Delivery  During KIIs on this question, CSO responses were mixed. Two rated LGCS’ efforts highly, one moderately, and two said there was no effect. Most municipal informants felt that the 8 When asked general questions related to sentiments towards municipal councils and local governance, opinions have remained somewhat negative. Y1Q3 – Y2Q4 DATA TRENDS Y1Q3 Y2Q4 24% 19% 33% 32% 42% 48% Taqairb Partner Municipalities Non-Partner Municipalities Good Average Poor How would you rate the quality of public service delivery in your community? 16% 17% 26% 24% 58% 59% Taqarib Partner Municipalities Non-Partner Municipalities 29 public had yet to see any visible impact of LGCS’ interventions to improve municipal government performance. As one informant commented, “It is still early, to be honest.” The one exception seemed to be in the Tripoli municipality, where an informant remarked that the results of LGCS’ inputs have been evident to the local population but that the population may not have attributed this to LGCS support. Conclusion While knowledgeable informants were aware of LGCS’ efforts, this awareness does not extend to the public at large. Survey results showed a slight improvement in public perception of service delivery in the targeted municipalities in Year 1 compared with the control group. However, the public perception of municipal performance in LGCS communities dropped considerably from Year 1 to Year 2, perhaps attributable to a general decrease in public satisfaction with the government over that time, as noted by Altai’s informants. Public perceptions of municipal performance in LGCS communities may increase in Years 3 and 4 as LGCS’ assistance at the community level becomes more evident to the public. Recommendations 1. LGCS should target assistance to areas that will demonstrate more immediate enhancements in municipal government service delivery to improve public perceptions of local government and its legitimacy. 2. LGCS should support strategic public communication initiatives to increase the public’s awareness of municipalities’ service delivery accomplishments. EQ 3: How has the Activity contributed to Libya’s decentralization agenda? MOLG is one of the main implementers of the state’s decentralization process, and it works with other government ministries to transfer resources, responsibilities, and authorities to the municipalities. The intent of Law 59 of 2012, which it administers, is to transfer some of Libya’s oil revenues to municipalities to support service delivery. The MOLG is at the center of the Libyan government’s decentralization transition. For the following questions, the ET interviewed nine CSO informants from four communities, seven municipal informants from four municipalities, two senior MOLG officials and three donor representatives. LGCS reports were also reviewed. Findings  When MOLG officials were asked about LGCS’ support on decentralization they said it did not appear to be a USAID priority, and that other donors were more effective.  To support the decentralization process, LGCS prepared draft policies and service delivery manuals for municipal operations in a variety of areas such as building use, donor policy, urban planning, procurement, recycling, recruitment, and job descriptions. These were to be provided to MOLG for approval. LGCS objectives 3.1, 3.2, 3.3, and 3.4 all relate to MOLG in some manner. The Activity has exceeded or met all its targets related to these indicators (see Annex 2). Despite these efforts, MOLG informant comments on the overall performance of LGCS were largely negative, and are summarized below:  LGCS inputs seem to be scattered, not linked to a clear strategy, and not systematic.  The contracting process needs to be faster: “Conditions in Libya change quickly. In six months, 70 percent of a project plan may be irrelevant.”  LGCS should work with and fund the MOLG directly to be more effective. “They need to put the MOLG in the center of their efforts toward the municipalities.”  Communication should be improved: “Items were delivered to Zawia Municipality, but … not properly handed over. The municipality complained that they had not received them and Taqarib reported that they had been delivered, so we had to get involved and sort these issues 30 out. It is down to the lack of communication. All parties, especially Taqarib, need to keep MOLG in the picture.”  Assistance should better align with MOLG priorities and strategy. The MOLG made it clear that they are “not starting from scratch,” and “don’t need beginner support.”  Planning and consistency can be improved: “We need them to be more organized with their approach and plan. LGCS might pilot a project with 20 municipalities. If it works, good. If it doesn’t, then they move on to the next pilot. What about the other 100 municipalities? I am asking them to stick to one methodology and implement it in all municipalities. I am not saying the issue is from USAID only, it is also a communication issue from both sides.”  Decentralization does not appear to be a USAID priority; other donors are focusing more on this. Decentralization is a “huge task that most likely won’t be implemented in the next 10 years.”  Most CSO informants said they had not worked directly on Libya’s decentralization agenda, but several had positive views about LGCS’ efforts on decentralization. They noted that they had conducted campaigns in their community on the reconciliation process, had been involved with the constitutional drafting process, and had conducted several workshops and assessments in their local community. This helped reduce the gap between the municipality and national government, especially on PWD rights, and its work will help improve the decentralization process and transfer of competencies to the municipalities. Another informant said that they were working with MOLG on decentralization without LGCS’ involvement. One mentioned that moving services from the central to the local government is a transformational change that is likely to create resistance from some quarters of the government.  Responses from municipal informants were mixed as some were positive, others were unaware of what LGCS was doing to support decentralization, and others reported that LGCS was having no visible effect in this area. Those with a positive view reported that LGCS was playing a constructive role in the process by working with several stakeholders to implement the decentralization agenda. As one of the informants commented, “I have been following the process and what has been presented to the MOLG as the plan, and I think that they (Taqarib) have good experience in this. So, I think they are following a good road on this.” Another said that the decentralization process was slow due to the current context. He noted, “The country has been centralized since 1951. They only know this system; they have grown accustomed to this. It is very difficult to change this overnight.” This view was underscored by a Benghazi municipal informant who commented that implementing decentralization is challenging because there is a lack of government consensus on implementing laws supporting decentralization and giving more authority and responsibilities to municipalities and the MOLG. Conclusions  LGCS drafted procedural documentation that municipalities could use to implement responsibilities associated with decentralization. Also, informants knew decentralization was an important issue and a few reported being involved with initiatives supporting the process. Some informants knew that LGCS was supporting the decentralization process, but the extent to which LGCS inputs were involved with those initiatives was not clear to them. Responses from both CSOs and municipalities indicate that LGCS’ decentralization activities are not well known. This implies that LGCS has not been effectively communicating its decentralization￾related initiatives across its network of partner organizations and municipalities.  MOLG officials suggested that LGCS provide more support to the Ministry, and that its assistance be better aligned with MOLG strategies. They also considered LGCS assistance to be poorly delivered and not linked to any strategy. 31 Recommendations 1. LGCS should regularly communicate with MOLG about its work plans and assistance and work with them and other central government agencies as appropriate to help them design and implement a strategic public communication campaign to promote the decentralization plan to the public. 2. LGCS should continue its decentralization support related efforts and ensure they are effective and systematically documented and implemented, shared with its partners, and provided to MOLG for use in other municipalities in the country. EQ3a: To what extent has the Activity enabled two-way national–local government coordination? Findings  MOLG respondents mentioned that decentralization did not appear to be priority for LGCS, and that it did not appear that LGCS had not done much to improve national-municipal government coordination.  The six CSO informants who responded to this question had mixed views. One had a very good impression of LGCS’ impact, two were uncertain, and three saw no effect. One who spoke positively said that the municipal council members and national government were working together to solve all the obstacles facing the municipality.  Municipal informants also had mixed views. Several reported that LGCS has not been able to connect effectively with MOLG or other parts of the Libyan government and had not seen LGCS do anything to improve national and local government cooperation. An informant with a more positive view considered LGCS to have a good relationship with both levels of government, and to have facilitated good communication and cooperation between them.  A related line of inquiry asked informants about what the central government required to increase its support for municipalities. Municipal informants said the government needs to communicate more, with a professional and structured protocol, and establish a system to monitor the performance of municipal governments. CSO informants focused on legislation, budget, and competency issues required to improve service delivery. They mentioned that the central government could provide legal support to municipalities and that it should encourage municipalities to work together, share experiences, and provide municipal council members with assistance in planning and execution. Conclusion LGCS’ efforts to foster two-way national–local government coordination was not shared widely with CSO partner organizations. Some municipal informants stated that LGCS was fostering good cooperation between national and local government authorities while others did not see that LGCS was doing anything to improve national and local government coordination. MOLG informants reported that decentralization did not appear to be an LGCS priority and that other donors were providing more effective support to the decentralization process. Recommendations 1. Provide technical assistance to assess and strengthen the central government’s ability to monitor municipality performance and provide support as appropriate. 2. Encourage MOLG to create networking and collaboration opportunities for municipalities and CSOs. 3. Develop and implement a strategic communication process to appropriately convey LGCS’ decentralization support initiatives among central and municipal governments, partner CSO organizations, and the donor community. 32 EQ4: Has the LGCS Activity mitigated potential negative effects of decentralization? (e.g., fostering further fragmentation along political/geographic/ethnic lines, further weakening, or impeding the development of Libya’s central state institutions)? Findings  Of the five CSO informants who answered this question, none provided any information that directly or indirectly addressed the issue of whether LGCS had mitigated potential negative effects of decentralization. In general, CSO informants were more focused on grants and services delivery within their municipalities.  Municipal informants mentioned that LGCS operated in challenging times with Libya facing the COVID pandemic and conflict, and this could have negatively affected LGCS’ impact. An informant from Tripoli Center mentioned that the peer-to-peer exchanges managed to narrow the distance between the eastern and western regions and improve prospects for reconciliation. Municipal informants from the South mentioned that the project did not respond well to decentralization￾related issues and that more effort was needed. Conclusion The question of potential negative effects of decentralization – fragmentation, etc. – did not elicit any data from informants. Based on the comment of one municipal informant it appears that peer￾to-peer exchanges of municipalities from different regions could reduce regional fragmentation and improve national unity. Recommendations 1. Expand LGCS’ use of peer-to-peer exchanges across the country. Institute other approaches, such as holding periodic meetings with all partner municipalities to exchange experiences and best practices. This will help to break down regional barriers and promote national unity. 2. Demonstrate ethnic diversity and gender equity in LGCS operations by employing personnel from a variety of Libya’s sub-groups and draw from their experience in development initiatives. 3. Better communicate LGCS’ efforts to mitigate the negative effects of decentralization to partner CSOs and municipalities. Additional Evaluation Areas of Inquiry (AOI) AOI 1: How appropriate and relevant is LGCS’ design? This area of inquiry was added to the evaluation at the initiative of the ET during discussion with USAID in the kickoff meeting and included in the Inception Report and Interim Presentation to obtain views from LGCS personnel on the appropriateness and relevance of the design of the Activity they were implementing. It also seeks to address the objectives of this evaluation related to possible changes in the Activity, and to generate information for USAID’s consideration in working in local governance in other fragile, conflict-affected states. The specific questions asked were as follows: 1. Do you think the LGCS project’s objectives, methods, and performance indicators are relevant and appropriate? 2. What are the main challenges you encountered during implementation? 3. What changes (if any) would you like to see in LGCS’ Activity design? 4. Could you share some of the Activity’s successes? 5. How can USAID’s project design and implementation processes be improved to better serve fragile, conflict-affected states? 33 These questions were asked of 10 LGCS/DAI personnel from four communities. Findings from each question are reported in sequence below, with overall conclusions and recommendations at the end of the section. AOI 1.1 Do you think the LGCS project’s objectives, methods, and performance indicators are relevant and appropriate? Findings  Most LGCS informant responses showed enthusiastic support for the Activity, as several informants reported that the objectives of LGCS were relevant to the Libyan context, that the activity was on track to meet its objectives, and that the flexibility of the award mechanism enabled LGCS to respond to the various requests of municipalities. However, one informant had a very different view of the design, questioning whether LGCS’ focus on municipalities was appropriate, suggesting it should focus more on central government ministries rather than municipalities. As he explained, “Taqarib should not be focusing their efforts on municipalities. Their focus should be more on the ministerial level.” He added, “Most municipalities in Libya do not have proper recognition – they are weak, they do not have a financial management system, they don’t even own the buildings they occupy in the various municipal councils. They do not have the tools to do their jobs effectively. It is too early to do projects with municipalities, they are too premature.” His response implies that he considers municipalities as not being at an adequate level of development to effectively receive LGCS assistance.  There was an expressed need to adjust LGCS priorities to better suit the context, recognize the reality of the country’s decentralization time frame, and address problems with the Activity’s equipment distribution and schedules. The view was that LGCS should avoid becoming entangled in the country’s political turmoil and take advantage of the operational flexibility from having a cooperative agreement with USAID. Sample comments: LGCS relevance: When asked whether the Activity’s goals and indicators were relevant in the Libyan context, an informant replied that LGCS was “very important and the decentralization program for Taqarib is fantastic. So, the goals and what they are working on are very relevant to the Libya context.” Another said, “I think we are fully on track and have advanced well. I think it is good that we are very specific and direct with our indicators and goals in our components. We know what we want now, it was well planned.” Equipment distribution and schedules: While most informants said they thought LGCS was doing good work, a few expressed concerns about the equipment distribution process and the time required to deliver on contracts. As an example, one informant noted that LGCS distributed the same amount of equipment to each municipality regardless of its size and needs. However, based on LGCS documents, each municipality received equipment and repairs that were based on the municipalities’ expressed needs. Because many municipalities needed roads repaired and streetlights, most received that same type of assistance. Whether they received the same number of streetlights and equipment for road repairs is not provided in the documentation. Another stated that delays in contract approvals and shipments of goods resulted in some material arriving well after their planned delivery date, negatively impacting the credibility of USAID and LGCS in the eyes of recipients. Flexibility from cooperative agreement: The Activity’s Cooperative Agreement with USAID contributed to its flexibility. The LGCS COP said, “The work plan can be negotiated in regular quarterly review meetings with USAID. This is a demand-driven project which tries to be responsive to municipalities’ needs.” AOI 1.2 What are the main challenges you encountered during implementation? Findings 34  LGCS informants identified a number of challenges during implementation: the COVID-19 pandemic, remote online training, the Activity’s internal communications, internet connectivity, exceedingly slow procurement, and improper level of material distribution to municipalities. Other challenges included establishing relationships with communities, difficult aspects of Libyan culture (e.g., mistrust of strangers), CSO registration requirements, low capacity of municipal government personnel, security, and political challenges, and effectively managing municipal government and CSO expectations. Comments on these were as follows:  COVID-19 and security impacts: The onset of the COVID-19 pandemic early in the Activity’s operations presented LGCS with major challenges on multiple fronts. Staff travel and in-person communication were curtailed, and training had to shift to virtual modes using online platforms such as Google Meet, Zoom, and others. The internet connectivity required for these platforms was often not functional or even available in some municipal areas. One informant said, “The really difficult challenge was the technical difficulties with remote training – we have bad connectivity as well as electricity issues, so it was difficult to implement the online components.” COVID also negatively impacted the quality of relationships that LGCS was starting to establish with municipalities and CSOs. As one informant remarked, “The COVID situation has been very difficult, because we lost contact with all our partners for a certain amount of time. So, we lost credibility and had to regain this.” COVID also affected logistics, which slowed deliveries of material to municipalities. Several commented that the fragile state of Libya presents LGCS with security and political challenges and makes it difficult to navigate the situation on the ground while others noted that the project’s design made it possible for LGCS to continue to operate in a turbulent political context. One remarked, “As we are technical, we do not deal with governments so the change from GNA [Government of National Accord] to GNU [Government of National Unity] did not affect us.  Internal Communication: There were also management-related communication problems among various parts of the LGCS team itself: “The lack of communication and miscommunication are a challenge within our organization. There are decisions that have been made in Tripoli and then come to us in Benghazi much later and this really affects us. We expect it will improve as Taqarib improves its processes, but the underlying issue is the communication between all the different Taqarib offices internally.”  Slow procurement: One of the most consistent findings of this evaluation were complaints about the Activity’s slow procurement processes, which delayed material deliveries and grant approvals. Many of the informants put the blame on USAID procurement procedures while others also blamed municipalities. One informant noted, “The biggest challenge that we are facing is that the processes are taking too long. Approvals are taking three months, especially when the municipality has been promising the local public that they are making changes. They are getting questioned by the public, then the municipality is coming to us and asking us what is taking too long. There is no reason for the processes to take this long to be honest. We should have tangible work on the ground much sooner than how we are doing now.”  Establishing relationships with communities: LGCS informants reported challenges in establishing effective and trusting working relationships with several Libyan communities, many of which were suspicious of outsiders. Considerable effort was required to overcome these barriers. This was noted by one informant as a characteristic of Libyan culture: “The first challenge is the mentality of the Libyans, the culture itself, and the demographics and differences between the regions. You can imagine we are constantly needing to gain the trust of the different stakeholders.” 35  CSO registration requirements: The government recently implemented a CSO registration requirement, which negatively impacted LGCS’ ability to work with CSOs. An informant spoke to this issue: “With regards to the CSOs, we are following the rules requiring LGCS to only work with registered CSOs.”  Low capacity of Libyan government personnel: One of the challenges in improving municipal and national government effectiveness is the low capacity of government personnel. One informant said there is a need to have “the correct employees in the right positions, who speak the right language, have computing skills, having the right skills. Most employees can’t write a letter, even in Arabic.” This contrasts with some previously noted partner informants’ complaints that LGCS training was “simple stuff,” and they required more in-depth technical support.  Expectation management: An issue that emerged throughout this evaluation was the need to manage expectations. An LGCS informant addressed this matter: “The municipalities all believe that what we are doing is not enough; the municipalities are requiring a lot of things because they have not been provided budgets, so they are expecting a lot more from Taqarib. We need to manage their expectations.” AOI 1.3 What changes (if any) would you like to see in the LGCS Activity’s design? Finding  LGCS informant responses to this question reflected several challenges noted above: to base material distribution on the needs and size of the community, better manage expectations, speed up the approval process, and reduce bureaucratic procedures. Additional suggestions included increasing LGCS’ focus on supporting decentralization, helping municipalities create a register of assets, encouraging recruitment of younger men and women in municipalities, improving coordination with stakeholders, and clarifying what USAID wants to achieve. AOI 1.4 Could you share some of the Activity’s successes? Finding  LGCS personnel had several success stories to recount. These included helping a municipality become more PWD-accessible, supporting municipal GIS systems, seeing LGCS-supplied equipment being used by the municipality to support a local clinic, and linking municipalities and CSOs. Others included supporting decentralization, receiving requests for training, seeing municipalities apply LGCS inputs to improve elder and citizen participation, and supporting multi-stakeholder dialogue. LGCS informant responses on some of these examples follow: AOI 1.5 How can USAID’s project design and implementation processes be improved to better serve fragile, conflict-affected states? Finding  Informants had several suggestions on ways to improve USAID’s effectiveness in Libya that could apply to USG operations in other similar situations. These included consulting more with beneficiaries and adjusting activities accordingly, ensuring valid contextual awareness, supporting reconciliation efforts, matching inputs with local capacity, and, as noted elsewhere, speeding up response and approval processes. 36 Conclusions These conclusions and the following recommendations are drawn from findings associated with the five sub-questions above. LGCS informants noted many positive aspects of LGCS as well as some challenges that needed to be addressed and offered several suggestions to address those challenges. Challenges were mainly focused on the need to better understand the local context so LGCS could more closely align its assistance with the capacity of national and municipal beneficiaries. Other challenges focused on the need to manage the expectations of national and municipal beneficiaries regarding the scope of LGCS and what it planned to deliver during its five-year term. Informants had several suggestions for improving the effectiveness of LGCS assistance: Recommendations 1. Improve communication with partners and adjust LGCS priorities to respond to the local context and align its assistance to the capacity of national and local governments. 2. Recognize the reality of the country’s decentralization time frame and adjust LGCS performance expectations accordingly. 3. Ensure equipment is distributed to municipalities based on their size and needs. 4. Return to in-person training and meetings as conditions permit. 5. Improve LGCS’ internal communication so that all staff have a common understanding of LGCS’ objectives, activities, procurements, communications with stakeholders, and assistance delivery mechanisms. 6. Promote exchanges of information among field specialists so they can share their experiences and successes working with CSOs and municipal officials in the target municipalities. 7. Regularly inform national and municipal-level stakeholders of LGCS objectives, scope, and work plan to better manage stakeholder expectations of what LGCS can deliver. 8. Assist municipalities to create a register of assets to increase transparency and reduce the potential for corruption. 9. Encourage recruitment of younger men and women in municipalities who are better prepared to interface with technology. 10. Expand the development and implementation of citizen charters in all target municipalities. AOI 2: How has LGCS supported the Ministry of Local Governance (MOLG)? As with AOI 1 above, this area of inquiry was added to the evaluation during the kickoff meeting at the initiative of the ET and included in the Inception Report and Interim Presentation. AOI 2 assesses central government institutional sustainability of LGCS inputs, a question not overtly addressed by the EQs in the SOW. The ET prepared the following sub-questions: 1. How have USAID and other donors had to adjust their programs in response to MOLG needs? 2. How has MOLG collaborated with USAID/LGCS activities? Are there ways in which that collaboration could have been improved? 3. How effective are LGCS’ programs with MOLG and the municipalities? 4. To what extent did LGCS contribute to decentralization in Libya? Data sources were limited to two senior level MOLG officials, and four donor informants. In some of the responses from the two senior MOLG officials interviewed, it was difficult to determine whether they were speaking only of LGCS or LCGS and USAID together. There may have been other communications directly between the Ministry and USAID that contributed to the views reported here. Informants’ comments addressed project planning and implementation issues, LCGS support for MOLG, LGCS effectiveness, communication issues, quality of decentralization support, and municipal capacity development. 37 Finding  To support the decentralization process, LGCS prepared draft policies and service delivery manuals for municipal operations in a variety of areas such as building use, donor policy, urban planning, procurement, recycling, recruitment, and job descriptions. These were to be provided to MOLG for approval. LGCS objectives 3.1, 3.2, 3.3, and 3.4 all relate to MOLG in some manner. The Activity has exceeded or met all its targets related to these indicators (see Annex 2). Despite these efforts, MOLG mentioned that LGCS should work with and fund the MOLG directly to be more effective, better align LGCS assistance with MOLG priorities and strategy, replicate proven pilot projects in all 100 municipalities, and make decentralization a USAID priority. Conclusion LGCS has done considerable work in drafting policies and manuals for municipal governments in several areas, but MOLG informants reported that it needed to do more to support the decentralization process and develop model municipalities that can be replicated throughout Libya. Recommendations 1. Review LGCS’ current operating plan and determine whether to refocus more of its assistance to enhance the capacity of MOLG to deliver technical assistance to all municipalities to improve their ability to deliver quality services. 2. Significantly improve communication and increase collaboration with MOLG and other central government entities involved with decentralization, municipal governance, and service delivery. 3. Continue to collaborate with other donors supporting MOLG and other central government agencies to further the decentralization process. AOI 3: How are other donors supporting subnational governance in Libya? This additional question was added based on ET input during the kickoff meeting with USAID and included in the Inception Report and Interim Presentation. Its intent was to provide basic information on what other donors are doing to support subnational governance in Libya. Two sub-questions were asked: AOI 3.1. What support are other donors providing to strengthen your municipality? AOI 3.2. How coordinated are the various donor activities supporting your municipality? For these two questions, the ET interviewed 10 CSO informants from 6 communities, 4 municipal informants from 2 municipalities, and 4 donor representatives from Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), the European Union (EU), Canada, and the German Foreign Office. Information was also obtained from an internet search. Conclusions and recommendations from the two sub-questions are consolidated below. Findings  Municipal and CSO informants listed several donors they are working with, including British BCI, Intersos, Red Crescent, UNDP, IOM, UNICEF, Supernova, Expertise France, GIZ, VNG, and the EU. Donor representatives said they were channeling most of their funds through multilateral agencies such as UNDP and the United Nations Children’s Fund (UNICEF), and focusing on capacity development, with many supporting central agencies. GIZ was early in the process of helping the government set up an Institute of Public Administration. 38  Another donor initiative is Phase 2 of the “Baladiyati” initiative7 involving the EU, UNDP, UNICEF, and the Italian agency AICS. This initiative is EU-funded at 26 million Euros and aims to improve basic services and promote sustainable economic development and social cohesion in 15 municipalities in the South of Libya. This initiative, which UNDP noted is being done in partnership with MOLG, is spending about 75 percent as much as LGCS’ $39.4 million budget and working in half as many municipalities.  Donor informants reported having challenges like those noted by LGCS, such as limited awareness of conditions on the ground, deciding where best to provide inputs in a politically complex and dynamic environment, difficulties with the central government’s decentralization process, and identifying initiatives that are likely to have sustainable beneficial outcomes on municipalities. The EU informant said the donor coordination process was ineffective, and that unlike in many other countries, senior-level Libyan officials were able to tell donors “no” when offered inputs they did not think were in the “best interests of society” – she saw this as a positive indicator of the capacity of the government’s senior officials.  One CSO informant commented, “To be honest we are not getting the effect from the Taqarib projects. The other donors have a great relationship with us, and we are nurturing this.”  Although some informants said there seemed to be no coordination among donors, LGCS says it coordinates its activities with all donors that work in its targeted municipalities. In addition, as noted earlier in the section on MOLG, the government is implementing measures to foster coordination of donor activity and align it with its own development priorities. Conclusion Several donors are working to support the decentralization process and strengthen municipality performance and face similar challenges in doing so. At the municipal level, LGCS is coordinating its work with other donors implementing municipal development programs. Recommendations 1. Continue to coordinate with other donor projects operating in LGCS-targeted municipalities to ensure donor assistance does not overlap and is complementary. 2. Consider working with MOLG and municipality networks to facilitate periodic donor meetings to exchange information and ensure their assistance to the MOLG, municipalities, and community organizations is well coordinated and effective. 7 Source: https://reliefweb.int/sites/reliefweb.int/files/resources/EN%20-%20Joint%20Press%20Release%20- %20Baladiyati%20Kickoff%20Phase%202%20-%203%20November%202021.pdf Accessed 2021-11-03. 39 ANNEX 1: EVALUATION SCOPE OF WORK SCOPE OF WORK Mid-Term Evaluation of Local Governance and Civil Society Program (LGCS) 40 I. PURPOSE OF THE EVALUATION This statement of work (SOW) for the Monitoring and Evaluation for Tunisia and Libya contract (METAL) outlines USAID/Libya’s need for a midterm evaluation of the Local Governance and Civil Society Program (LGCS) to inform the USAID Democracy and Governance team on how the USG can best support a democratic political transition in the midst of conflict and growing external influence. Recommendations will inform the team to make changes to the existing Activity, ensuring that USAID’s efforts are as effective as possible given the challenging operational environment. The main audience is the USAID/Libya Democracy and Governance (DG) team and the broader USAID-Libya team, though the reports may also be shared in full or in part with the Libya External Office and USAID implementing partners. II. SUMMARY INFORMATION Strategy/Project/Activity Name Local Governance and Civil Society Program (LGCS) “Taqarib” USAID Office USAID/LIBYA Implementer(s) DAI Global, LLC Cooperative Agreement/Contract # 72067019CA00001 Total Estimated Ceiling of the Evaluated Project/Activity(TEC) $39.4M Life of Strategy/Project/Activity July 8, 2019–July 7, 2023 Active Geographic Regions Active in 30 municipalities nationwide Development Objective(s) (DOs) Effectiveness of key Libyan governance institutions improved Required evaluation? Yes External or internal evaluation? External III. BACKGROUND The Local Governance and Civil Society (LGCS) Activity, locally known as “Taqarib” (Arabic for “coming together”), works toward a more unified Libyan state. The Activity supports community-led activities that improve service delivery and accountable governance and promote stronger links between government institutions and citizens. A. Description of the Problem and Context The LGCS Activity was established with the knowledge that Libyans are weary from a decade of conflict and tired of a political class that has overstayed its tenure. To help strengthen the foundations toward a more unified Libyan state, the LCGS Activity aims to empower local governments to tangibly improve service delivery, reinforce community bonds, promote citizen engagement in public dialogue, and prioritize public resource allocations. B. Description of the Intervention to be Evaluated and Theory of Change 41 The program was designed under the overarching theory of change that: IF municipal institutions visibly improve the delivery and responsiveness of services (Objective 1) and coherence of national and subnational government policy, systems, and coordination (Objective 3); and IF Libyan men and women are empowered with the skills and capability to influence government action and hold the government accountable (Objective 2), THEN citizens‘ trust and confidence in their government will improve, thereby promoting stability and peace and strengthening the foundations of a more unified Libyan state. IV. EVALUATION QUESTIONS Evaluators will review and finalize questions in collaboration with USAID prior to finalizing the evaluation design. CORE QUESTIONS: 1. To what extent has the Activity improved citizen participation in and oversight of municipal decision-making and service delivery processes? 2. To what extent has the Activity enhanced the responsiveness of local government institutions? a. How has the Activity contributed to improved municipal service delivery performance? b. To what extent have the Activity’s interventions affected public perceptions of local government performance and legitimacy? 3. How has the Activity contributed to Libya’s decentralization agenda? a. To what extent has the Activity enabled two-way national–local government coordination? 4. Has this Activity mitigated potential negative effects of decentralization (e.g., fostering further fragmentation along political/geographic/ethnic lines, further weakening or impeding the development of Libya’s central state institutions)? V. EVALUATION DESIGN AND METHODOLOGY USAID expects that, at a minimum, the evaluation team will do the following: ● Upon award, familiarize themselves with documentation about the Taqarib Activity and USAID’s current assistance in the Democracy and Governance area in Libya and the region. ● Review and assess the Activity’s performance reports and performance data. ● Test survey instruments for clarity prior to use. ● Meet and interview USAID project implementing partner staff, beneficiaries, partners, and host government counterparts at appropriate levels. ● Interview USAID staff and a representative number of experts working in the sector. Due to COVID-19, it is expected that interviews will be conducted remotely. The evaluation team must address this limitation and include appropriate mitigation measures in its evaluation design. The desk review includes at a minimum: ● USAID Local Governance and Civil Society “Taqarib” SOW 42 ● Taqarib/LGCS materials: Annual and Quarterly Reports; Annual Work Plans; Monitoring, Evaluation, and Learning (MEL) Plans; Data Quality Assessments (DQAs); sector assessments; trip reports; performance report, gender analyses; relevant sections of the Project Appraisal Document; and miscellaneous thematic reports from other sources. The contractor will submit the preliminary evaluation design for review by USAID. The evaluation Contracting Officer’s Representative (COR) must approve the evaluation design prior to field work commencing. VI. DELIVERABLES AND REPORTING REQUIREMENTS 1. In-briefing Given travel restrictions related to the COVID-19 pandemic, the evaluation team will conduct a virtual in-briefing with the DG Officers and local staff for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to adjust the SOW, if necessary. 2. Evaluation Inception Report The evaluation team must prepare an evaluation inception report that includes the following: 1. Draft evaluation team schedule, including key milestones and logistical arrangements 2. Members of the evaluation team, delineated by roles and responsibilities 3. Description of the proposed evaluation methodology; this section should include a detailed evaluation design matrix that links the evaluation questions from the SOW (in their finalized form) to data sources, methods, and the data analysis plan 4. List of potential key informants and proposed selection criteria and/or sampling plan, as applicable 5. Draft data collection tools; 6. Limitations to the evaluation design 7. Dissemination plan (designed in collaboration with USAID) The data analysis plan should clearly describe the evaluation team’s approach for analyzing quantitative and qualitative data (as applicable), including proposed sample sizes, specific data analysis tools, and any software proposed to be used, with an explanation of how/why these selections will be useful in answering the evaluation questions for this task. Qualitative data should be coded as part of the analysis approach, and the coding used should be included in the appendix of the final report. Gender, geographic, and role (beneficiary, implementer, government official, NGO, etc.) disaggregation must be included in the data analysis where applicable. All dissemination plans should be developed with USAID and include information on audiences, activities, and deliverables, including any data visualizations, multimedia products, or events to help communicate evaluation [findings/conclusions/recommendations]. See the Evaluation Toolkit for guidance on Developing an Evaluation Dissemination Plan. If applicable based on the Disclosure of Conflict of Interest Forms submitted with the awardee’s proposal, the evaluation design will include a conflict of interest mitigation plan. 3. Midterm Briefing and Interim Meetings The evaluation team is expected to hold a midterm briefing with the USAID/Libya Democracy and Governance (DG) team, Middle East Regional Platform MEL specialist, and other relevant USAID/Libya team members on the status of the evaluation, including potential challenges and emerging 43 opportunities. The team will also provide the evaluation Contracting Officer’s Representative (COR)/Agreement Officer’s Representative (AOR) with periodic briefings and feedback on the team’s findings, as agreed upon during the in-briefing. If desired or necessary, weekly briefings by phone can be arranged. 4. Preliminary Presentation for Recommendations Development The evaluation team is expected to hold a preliminary presentation by virtual conferencing software to discuss the summary of findings, conclusions and recommendations. Any presentations or workshops will be scheduled as agreed upon during the in-briefing. 6. Draft Evaluation Report The draft evaluation report should be consistent with the guidance provided in Section IX, Final Report Format, and must meet all evaluation report and quality requirements detailed in USAID Automated Directives (ADS) Chapter 201. The report will address each of the questions identified in the SOW. The submission date for the draft evaluation report will be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, the USAID/Libya team will have 10 working days in which to review and comment on the initial draft, after which point the COR will submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report within ten working days, and again the USAID/Libya team will review and send comments on this final draft report within ten working days of its submission. 7. Final Evaluation Report The evaluation team will be asked to take no more than 10 working days (or as agreed upon in the work plan) to respond to and incorporate the final draft evaluation report and presentation comments from the USAID/Libya team. 8. Submission of Final Evaluation Report to the Development Experience Clearinghouse Per USAID policy (ADS 201.3.5.18) the contractor must submit the evaluation final report and its summary or summaries to the Development Experience Clearinghouse (DEC) within three months of final approval by USAID. 9. Submission of Dataset(s) to the Development Data Library Per USAID’s Open Data policy (see ADS 579, USAID Development Data) the contractor must also submit to the COR and the Development Data Library (DDL), at www.usaid.gov/data, in a machine￾readable, non-proprietary format, a copy of any dataset created or obtained in performance of this award, if applicable. The dataset should be organized and documented for use by those not fully familiar with the intervention or evaluation. Please review ADS 579.3.2.2 Types of Data to Be Submitted to the DDL to determine applicability. 10. Other Deliverables a. Two-Page Summary Brief. Following final approval of the evaluation report, the contractor must produce a two-page brief of the evaluation that includes evaluation purpose and summarizes evaluation findings, conclusions, and recommendations. This brief should be visually appealing and, if appropriate, incorporate photos or other visuals. This brief may be shared with stakeholders. 44 b. Post-Evaluation Action Plan. Within 30 days of the approval of the evaluation report, the contractor must work with the USAID technical team to develop a “post-evaluation action plan” per the guidelines in ADS 201. VII. EVALUATION TEAM COMPOSITION The Contractor must propose an appropriate team for this assignment and provide information about evaluation team members, including their curricula vitae. The Contractor must explain how each evaluation team member’s qualification and expertise meets the requirements in the evaluation SOW and will contribute to the achievement of a technically sound, objective, and high-quality evaluation. Submissions of writing samples or links to past evaluation reports and related deliverables composed by proposed team members are highly desirable. The evaluation team must be approved by the METAL COR. Any substitutes to the proposed key personnel must be vetted and approved by the COR before they begin work. USAID may request an interview with any of the proposed evaluation team members via conference call, Skype, or other means. Per ADS 201.3.5.14, all team members must provide to USAID a signed statement attesting to a lack of conflict of interest or describing an existing conflict of interest relative to the project or Activity being evaluated (i.e., a conflict of interest form). VIII. DELIVERABLES AND TIMELINE The Contractor must provide the following deliverables. All written documentation for submission by the Contractor to USAID/Libya must be in English. This schedule is illustrative and will be updated (as necessary) in collaboration with USAID. Deliverables Due Date (Time frame) 1 Inception report containing work plan and evaluation design Prior to the start of the document review and meetings, the Contractor must submit to USAID an Inception Report for USAID approval. The report must include: ● Complete list of documents planned for review ● Complete list of planned interviews and proposed individuals ● Proposed list of questions for the interviews ● Detailed description of the data collection plan and analysis methodology ● Preliminary literature review USAID will have one week to review and provide comments and feedback. Work plan and evaluation design period: August 10, 2021 (July 27–August 10) USAID Review Period: August 17, 2021 (August 10–17) 2 PowerPoint presentation of Taqarib Mid-Term (MT) Evaluation preliminary findings, conclusions, and recommendations September 10, 2021 (August 17–September 10) 45 The Contractor must develop and deliver a PowerPoint presentation of the preliminary findings, conclusions, and recommendations of the Taqarib MT Evaluation to USAID/ Libya staff and other relevant stakeholders to be delivered to the USAID/Libya team. 3 Draft MT Evaluation Report Refer to section “Final Report Format and Supporting Data” below. Should be submitted as an MS Word document. Draft report will include feedback from PowerPoint presentation of preliminary findings, conclusions, and recommendations. USAID will 10 working days to review and provide comments/feedback. Draft Report Prep Period: September 24, 2021 (September 10-24) USAID Review Period: October 8, 2021 (September 24–October 8) 4 Final MT Evaluation Report Refer to section “Final Report Format” below. All comments provided by USAID should be addressed in the Final Report. The report should be submitted in MS Word and PDF formats. Final Report Prep Period: October 15, 2021 (October 8–15) IX. FINAL REPORT FORMAT 1. Abstract 2. Executive Summary 3. Evaluation Purpose 4. Background on the Context and Activity Being Evaluated 5. Evaluation Questions 6. Methodology 7. Limitations to the Evaluation 8. Findings, Conclusions, and Recommendations 9. Annexes See the Evaluation Toolkit for the How-To Note on Preparing Evaluation Reports and ADS 201mah, USAID Evaluation Report Requirements. The evaluation abstract of no more than 250 words should describe what was evaluated, evaluation questions, methods, and key findings or conclusions. The executive summary should be 2-5 pages and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, and conclusions (plus recommendations and lessons learned, if applicable). The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methods (e.g., in sampling; data availability; measurement; analysis; any potential bias such as sampling or selection, measurement, interviewer, response, etc.) and their implications for conclusions drawn from the evaluation findings. Annexes to the report must include the following: - Evaluation SOW (updated, not the original, if there were any modifications) - All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides - All sources of information or data, identified and listed 46 - Statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team, if applicable - Signed disclosure of conflict-of-interest forms for all evaluation team members, either attesting to a lack of or describing existing conflicts of interest X. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure quality. - Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or Activity. - Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. - The Executive Summary should present a concise and accurate statement of the most critical elements of the report. - Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. - Evaluation methodology should be explained in detail and sources of information or data properly identified. - Limitations to the evaluation should be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). - Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. - Conclusions should be specific, concise, and include an assessment of quality and strength of evidence to support them supported by strong quantitative and/or qualitative evidence. - If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. - If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. See ADS 201mah, USAID Evaluation Report Requirements for additional guidance. XI. OTHER REQUIREMENTS All modifications to the required elements of the SOW of the contract/agreement, whether in evaluation questions, design and methodology, deliverables and reporting, evaluation team composition, schedule, and/or other requirements will be agreed upon in writing by the COR. Any revisions made will be noted in the SOW annexed to the final Evaluation Report. XII. LIST OF ANNEXES 1. Activity Monitoring, Evaluation, and Learning (MEL) Plan 2. Year 1 and Year 2 Work Plans 47 3. Task Order and executed modifications 4. Data Quality Assessments (DQAs) 48 ANNEX 2: LGCS INDICATOR PROGRESS UPDATE 2021.10.27 Indicator Baselin e FY 1 Target FY1 Q1 FY1 Q2 FY1 Q3 FY1 Q4 FY 1 Actual FY 2 Target FY2 Q1 FY2 Q2 FY2 Q3 FY2 Q4 FY 2 Actual 1.1: Number of non-binding implementation agreements with municipalities 0 15 0 3 7 8 18 30 0 8 14 6 28 1.2: Percent change over the baseline score in the Municipality Performance Index (MPI) of partner municipalities 0 0 0 3 14 1 18 0 0 8 4.8% 6,6% 6.6% 1.3: Number of VIS processes supported in partner municipalities 0 15 0 3 7 8 18 30 0 8 14 6 28 1.4: Number of citizen priorities reflected in municipality VIS plans 0 15 0 2 7 8 17 30 0 8 14 6 28 1.5: Number of participants enrolled in in-service trainings, educational courses, or other professional-development opportunities with Taqarib-supported content 0 50 0 0 18 19 37 50 60 95 261 108 524 1.6: Number of individuals with improved skills following completion of USG-assisted workforce development programs 0 15 0 0 0 0 0 15 13 9 x x x 1.7: Number of municipality-led multi-stakeholder meetings, town halls, or similar events held 0 10 0 0 6 7 13 30 2 20 11 6 28 1.8: Number of individuals who participate in peer-to-peer learning exchanges 0 50 0 0 0 14 14 50 28 15 28 25 96 1.9: Number of host-country COVID-19 preparedness and/or response plans in various phases (formally proposed, adopted, or implemented) supported with USG assistance 0 5 0 0 0 1 1 15 13 3 0 8 16 1.10: Number of private sector partnerships utilized addressing second-order COVID-19 impacts 0 5 0 0 0 0 0 15 0 0 0 0 0 1.11: Number of participants in USG supported activities trained in community emergency response 0 50 0 0 0 127 127 600 187 248 95 211 741 2.1: Number of civil society organizations (CSOs) receiving USG assistance engaged in advocacy interventions 0 15 0 0 0 3 3 30 0 3 3 3 9 2.2: Percent change in Organization Performance Index scores over baseline of partner CSOs 0 0 0 0 0 0 0 0 0 0 0 0 0 2.3: Percent of U.S. Government-assisted organizations with improved performance 0 0 0 0 0 0 0 0 0 0 31% 29% 29% 49 Indicator Baselin e FY 1 Target FY1 Q1 FY1 Q2 FY1 Q3 FY1 Q4 FY 1 Actual FY 2 Target FY2 Q1 FY2 Q2 FY2 Q3 FY2 Q4 FY 2 Actual 2.4: Number of CSO-led multi-stakeholder meetings, town halls, or similar events held as part of municipal planning, budgeting, oversight, and other service delivery processes 0 5 0 0 12 0 12 10 3 2 0 0 5 2.5: Number of grants awarded to advocate for accountability or hold government institutions accountable 0 5 0 0 0 0 0 5 0 0 0 0 0 2.6: Percentage change in social accountability monitoring scores 0 0 0 0 0 0 0 0 0 0 0 0 0 2.7: Number of participants in the Champions for Change (C4C) program 0 150 0 0 60 119 179 150 0 0 46 37 83 3.1: Number of public policies introduced, adopted, repealed, changed, or implemented with citizen input 0 1 0 0 0 0 0 1 1 0 4 7 12 3.2: Number of Taqarib-supported multi-stakeholder dialogues among national actors regarding the decentralization agenda 0 2 1 0 0 1 2 2 1 1 0 0 2 3.3: Number of decentralized business processes mapped and analyzed 0 3 0 0 0 0 0 3 0 1 1 1 4 3.4: Number of national advocacy initiatives supported by Taqarib 0 3 0 0 0 3 3 3 0 2 1 1 4 3.5: % change in citizen perceptions regarding advancement of the decentralization agenda 0 0 0 0 0 0 0 0 0 0 4.1: Number of municipalities for which profiles have been completed and for which service delivery and public perception data is tracked over time 0 15 0 0 18 0 18 30 0 0 6 8 12 4.2: Number of strategic review sessions conducted 0 4 2 0 0 0 2 4 0 1 1 1 3 50 ANNEX 3: DATA COLLECTION INSTRUMENT Key Informant Interviews (KIIs): GUIDE for Interviewers Note: Not all questions were asked of each informant in each category. Key Informant Interview Record number: Name of the interviewer: Date of interview: Starting time: Ending time: Location: Introduction Hello, I am (……………….) and I am working with IBTCI, which is implementing the Midterm Performance Evaluation of Local Governance and Civil Society Program (LGCS). USAID and IBTCI team would like to learn about your perceptions of the impact of LGCS on Local Governance and the stability of Libya. Your knowledge will be very valuable as we evaluate the progress of LGCS. Your comments will not be attributed to your name, or your organization so be assured of the confidentiality of this information you provide. This interview will last approximately 45/60 minutes, and we will ask you few questions about how you perceive the impact of LGCS on Local Governance and Stability in Libya. Do you have any questions or concerns before we begin? Personal information of the interviewee Name: Position: Municipality / Ministry / NGO / DAI / Other: ☐ Man ☐ Woman Age: 1. How long have you been working professionally in your organization? 2. How did you first become involved with LGCS? 3. What is the type of cooperation with LGCS? ☐ Small Grant ☐ In-kind Grant ☐ Capacity Building ☐ Nothing 51 Questionnaire 1- For municipal Officers: Questions Impressions Answers/Comments Q1 To what extent has Taqarib support improved citizen participation in and oversight of municipal decision-making and service delivery processes? 1 = Things have gotten a lot worse after their support 2 = Things are a bit worse after their support 3 = No change after their support 4 = Things are a bit better after their support 5 = Things are a lot better after their support 1.1 Please describe your municipality’s relationship with CSOs and other citizen groups 1.2 Does your municipality make efforts to engage CSOs and citizens in its decision-making processes? Please describe. 1.3 How effective have LGCS’s activities been in increasing citizen participation in your municipal elections, decision￾making and operations? 1.4 What are the main benefits of your municipality’s signing the citizen charter with Taqarib? Q2 To what extent has the Activity enhanced the responsiveness of local government institutions? Note to researcher: The local government institutions are the MOLG and Municipalities. 1 = Things have gotten a lot worse after their support 2 = Things are a bit worse after their support 3 = No change after their support 4 = Things are a bit better after their support 5 = Things are a lot better after their support A How has the Activity contributed to improved municipal service delivery performance? B To what extent have the Activity’s interventions affected public perceptions of local government performance and legitimacy? 1= not at all 2 = slightly 3 = somewhat 4 = fairly well 5 = very well 2.1 Please describe the major differences between the roles of the central 52 Questions Impressions Answers/Comments government and the municipal council, and the services they provide 2.2 To what extent has LGCS support improved the capacity, performance, and sustainability of your municipal council? 1= not at all 2 = slightly 3 = somewhat 4 = fairly well 5 = very well 2.3 To what extent did LGCS contribute to supporting the municipal council to be more accountable and to better represent the interests of citizens? 1= not at all 2 = slightly 3 = somewhat 4 = fairly well 5 = very well 2.4 How effective is LGCS training to municipal officials, staff, and municipal council members to improve their ability to carry out their roles? 2.5 Has your opinion of the municipal council’s response to citizens’ needs changed since LGCS activities began? 2.6 How effective have been LGCS activities to increase citizens’ trust of municipal performance? 2.7 How effective has LGCS support to your municipality been? Is it enough or should there be more? Please describe. Q3. How has Taqarib’s Activity contributed to Libya’s decentralization agenda? 1= large negative impact 2= slightly negative impact 3= No change/impact 4= slightly positive impact 5= largely positive impact A To what extent has the Activity enabled two-way national–local government coordination? 1= not at all 2 = slightly 3 = somewhat 4 = fairly well 5 = very well 3.1 To what extent did LGCS contribute to the mission’s objective of supporting institutions of national and local government to be more accountable and better represent the interests of citizens? 1= not at all 2 = slightly 3 = somewhat 4 = fairly well 5 = very well 3.2 To what extent has LGCS support improved decentralization in Libya? Describe at least two activities. 1= large negative impact 2= slightly negative impact 3= No change/impact 4= slightly positive impact 5= largely positive impact 3.3 Has your opinion of the relationship between National and Local 1= large negative impact 2= slightly negative impact 3= No change/impact 53 Questions Impressions Answers/Comments governments changed as a result of LGCS activities? 4= slightly positive impact 5= largely positive impact 3.4 How effective are LGCS efforts to increase citizen trust and confidence in the performance of national and local governments? 1= large negative impact 2= slightly negative impact 3= No change/impact 4= slightly positive impact 5= largely positive impact Q4. Has LGCS Activity mitigated potential negative effects of decentralization (e.g., fostering further fragmentation along political/geographic/ethnic lines, further weakening or impeding the development of Libya’s central state institutions)? 1= large negative impact 2= slightly negative impact 3= No change/impact 4= slightly positive impact 5= largely positive impact 4.1 How did LGCS’ activities affect central-local government divisions in Libya? Please explain. 4.2 To what extent has LGCS support improved the development and performance of Libya’s institutions? 4.3 How effective are LGCS activities in increasing the participation of women in governments at the national and local levels? 4.4 How effective are LGCS activities in increasing the participation of marginalized groups in governments at the national and local levels? 4.5 Do you think that LGCS activities have benefited all regions of Libya? Follow up Q: if not, ask the informant to specify the reasons. 4.6 How can LGCS improve its activities in the future? 4.7 Please describe the support you receive from the central government to increase the effectiveness of municipal operations. 4.8 What do you think the central government agencies require to increase their ability to support municipal operations? 4.9 What other donors are providing support to strengthen your municipality? Follow up Q: if USAID is mentioned ask about its advantages compared to other donors. 54 Questions Impressions Answers/Comments 4.10 How coordinated are the various donor activities supporting your municipality? 2- For CSOs : Questions Impressions Answers/Comments Q1. To what extent has Taqarib support improved citizen participation in and oversight of municipal decision-making and service delivery processes? 1 = Things have gotten a lot worse after their support 2 = Things are a bit worse after their support 3 = No change after their support 4 = Things are a bit better after their support 5 = Things are a lot better after their support 1.1 Has your organization worked with the municipal council or government? 1.2 How open is the municipality to working collaboratively with you? 1.3 Please give examples of your relationship with the municipal council. 1.4 How easy is it for civil society to operate in the municipality these days? 1.5 Did your organization have any cooperation with the municipality before working with LGCS (Taqarib Project)? If so, please explain. 1.6 Please describe the response of the municipal council and government to engage with your organization? Please explain. 1.7 What issues has your organization faced in working with the municipality? 1.8 Has your organization signed the citizen charter with LGCS and the municipality? 1.9 What do you believe are the long￾term benefits of the LGCS project? 1.10 How can the LGCS project improve your organization in working with the municipality to benefit its citizens? 55 Questions Impressions Answers/Comments Q2. To what extent has the Activity enhanced the responsiveness of local government institutions? 1 = Things have gotten a lot worse after their support 2 = Things are a bit worse after their support 3 = No change after their support 4 = Things are a bit better after their support 5 = Things are a lot better after their support A How has the Activity contributed to improved municipal service delivery performance? B To what extent have the Activity’s interventions affected public perceptions of local government performance and legitimacy? 1= not at all 2 = slightly 3 = somewhat 4 = fairly well 5 = very well 2.1 How would you rate the municipal council’s response to citizen’s needs? 1= not at all 2 = slightly 3 = somewhat 4 = fairly well 5 = very well 2.2 What are your organization’s activities with LGCS to improve the responsiveness of municipal service delivery? 2.3 What are your organization’s activities with LGCS to improve the outreach and engagement of the municipal councils with citizens? Are you aware of any other entities or groups that assist the MC in the outreach and engagement activities? Q3. How has Taqarib’s Activity contributed to Libya’s decentralization agenda? 1= large negative impact 2= slightly negative impact 3= No change/impact 4= slightly positive impact 5= largely positive impact A To what extent has the Activity enabled two-way national–local government coordination? 1= not at all 2 = slightly 3 = somewhat 4 = fairly well 5 = very well 3.1 Does your organization advocate for decentralization in Libya? If so, please explain. 56 Questions Impressions Answers/Comments 3.2 Does your organization have any activities with the national government that are supported by LGCS? Please describe. Q4. Has LGCS Activity mitigated potential negative effects of decentralization (e.g., fostering further fragmentation along political/geographic/ethnic lines, further weakening or impeding the development of Libya’s central state institutions)? 1= large negative impact 2= slightly negative impact 3= No change/impact 4= slightly positive impact 5= largely positive impact 4.1 How can LGCS improve its effectiveness in the future? 4.2 Do you believe that LGCS activities improved the decentralization process in Libya? Please explain. 4.3 Does your organization implement any activities that include women? Please describe them. 4.4 Does your organization implement any activities that include marginalized groups? Please describe them. 4.5 Please describe the support you receive from the central government to increase the effectiveness of municipal operations. 4.6 What do you think the central government agencies require to increase their ability to support municipal operations? 4.7 What other donors are providing support to strengthen your municipality? 4.8 How coordinated are the various donor activities supporting your municipality? 3. DAI Project Personnel: 1. Do you think the LGCS project’s objectives, methods, and performance indicators are relevant and appropriate? 2. What are the main challenges you encountered during implementing the project? 3. What changes (if any) would you like to see in the LGCS Activity’s design? 4. Could you share with us successful examples and stories from the Activity 5. How can USAID’s project design and implementation processes be improved to better serve fragile, conflict-affected situations? 57 4. MOLG Personnel: 1. How have USAID and other donors had to adjust their programs in response to MOLG needs? 2. How has MOLG collaborated with USAID activities? Are there ways in which that collaboration could have been improved? 3. How often do USAID and other international projects share information formally or informally about the working environment, security, or political situation in Libya? 4. How effective are LGCS’s program with MOLG and the municipalities? 5. To what extent did LGCS contribute to the decentralization in Libya? 6. To what extent has LGCS support improved the capacity, performance, and sustainability of local administration in Libya? Conclusion Thank you very much for your time and for sharing your experiences with us. Do you have any additional comments or insight about the processes of stakeholder engagement and research prioritization? Do you have any questions for us? Again, thank you very much. Have a good day. Donor Activity information for LGCS evaluation Interview date: Informant name: Informant title/role: Informant organization: Informant location and contact information: Introduction: I am contacting you in my capacity as Team Leader of an interim evaluation of a USAID initiative strengthening municipalities in Libya - Local Governance and Civil Society (LGCS – also called Taqarib), and your assistance would be appreciated. We want to include summary information on other donors’ activities in this sector in our evaluation report. 1. How long has your organization been working in Libya? 2. What support has your organization provided for subnational governance in Libya? 3. Were you working with municipalities? If so, which ones? 4. What was the nature of your support? 5. Were you working with the central government agencies? If so, which ones? 6. What was the nature of your support for the central government agencies? 58 7. Did you collaborate with other donors? If so, which ones? 8. Does your organization plan to continue its support? If so, please describe. 9. Any other comments? Many thanks for your time, and for sharing your information for this evaluation. 59 ANNEX 4: RAW DATA SAMPLES Sample HyperRESEARCH database raw data printouts The following are samples of the database printouts that were used for analysis and reporting of informants’ responses. The answers respond to the questions listed in the KII questionnaire (see Annex 3). Raw data responses from MOLG informants to question, “Ministry 2:” “How has MOLG collaborated with USAID activities? Are there ways in which that collaboration could have been improved? Case Code 09 Mohamed Buohelga Ministry 2 Source Material: How can the cooperation between USAID and the MOLG be improved? We need them to be more organized with their approach and plan. The issue previously is that they pilot a project with 20 ministries and then if it works, good, and if it doesn’t then they move on to the next pilot. What about the other 100 municipalities? I am asking for them to ensure to fit to one methodology and implement it on all municipalities. I am not saying the issue is from USAID only, it is also a communication issue from both sides. 14. Maisa Ragibi - MOLG Ministry 2 Source Material: How do you feel USAID is able to work with the requirements of the ministry? There was no clear strategy on priorities or a plan for the requirements previously. However, this has since changed and the first thing that the ministry made clear to USAID in its meetings with the recent projects is making clear the strategy, the direction that the ministry is going toward and not all things that were discussed meant that the ministry is starting from scratch, there are several things that the ministry made clear are advanced and don’t need beginner support. So, the MOLG made it clear to USAID and other donors the exact priority requirements. 14. Maisa Ragibi - MOLG Ministry 2 Source Material: What is the cooperation between USAID and MOLG in general? There are old projects that have been finished now, but the relationship with USAID and MOLG is long-standing. Currently it is Pragma and Taqarib is all that I am aware of. 14. Maisa Ragibi - MOLG Ministry 2 Source Material: Do you feel Taqarib has been effective with regards to supporting the MOLG and municipalities? This is a critical project implementer, and we believe they have the potential to be effective. However, I must say that there is a lack of communication between the different entities. This has caused a lot of misunderstandings and resulting in issues. To give you an example, one of the project implementers has provided items to the Zawia municipality, they were delivered and have been kept in a room and not yet properly handed over. Zawia municipality has complained stating they did not receive it and Taqarib stating that it has been delivered so we had to get involved and sort these issues out. It is down to the lack of communication. All parties especially Taqarib need to keep MOLG in the picture. 14. Maisa Ragibi - MOLG Ministry 2 Source Material: Do you feel Taqarib efforts have been successful? 60 It is successful, but there is always room for improvement. They need to adjust their focus and communication efforts; I think we need to consult with them directly to be able to improve their efforts. 14. Maisa Ragibi - MOLG Ministry 2 Source Material: USAID in comparison to other donors? The main thing is from a technical perspective the support and funds needs to be directed to the MOLG especially with regards to supporting the municipalities. My message to USAID is that I would prefer that we increase our collaboration and communication between USAID and MOLG, I think the more there is collaboration and communication then there are better results for both sides. I think they need to put the MOLG in the center of their efforts toward the Municipalities. Municipal informants’ responses to questions Municipality 2 to Mun Q2A&B (See Questionnaire in Annex 3 above for details): Case Code 30. Ahmed - Awbari municipality Mun 2.2 Source Material: They are trying to support with things that really mean nothing. 31 Sarah Naami - Tripoli Center Council Mun 2.2 Source Material: For the elected members, they did not train us at all. They should be training the elected members; they should really need it as a priority. The training needs to be physical, not online, and practical. I think the training needs to support the development of smart cities. We need to be looking at these things to develop the future of the municipalities. Not some simple training. We need to be developing and building capacity. 32 Samir Kershman - Zwara Mun 2.2 Source Material: Do you feel Taqarib has been able to support the municipality services? They have not really improved our services. Even the park that they intended to create, they have not even implemented, it is still open I believe. And they started discussions with us on another project, but they didn’t even close out the first one. The individuals that came to us were very professional and capable individuals. 32 Samir Kershman - Zwara Mun 2.2 Source Material: Has Taqarib improved the municipality’s capacity? Very limited, a couple of workshops and trainings, but I don’t believe that has not been so effective. So yes, they did training, but I think it’s very limited. Case Code 18. Faisal - Tripoli Municipality Council Mun 2.4 Source Material: Do you feel the training provided was suitable for your requirements? I believe the training program was very useful to all, I liked how they involved the different municipalities and connected them so we can learn from each other. This was a very effective initiative that was well received by all. 24. Khaled Alhoudery - Sabha Municipality Mun 2.4 61 Source Material: Has Taqarib provided support to building the capabilities of the municipality? Yes, in this component, they have provided trainings to the staff and that was successful. 28. Naser Alawjaly - Benghazi Mun 2.4 Source Material: Do you feel that the training has supported the municipality? For this I have to say that the training was good, it was well implemented internally and was useful for our teams but again it was very small trainings and simple stuff. 29. Ahmed Ruhayem - DAI Mun 2.4 Source Material: All of the stakeholders in the municipalities prefer in person training, we do have very good participation rates online, however we have identified that the in-person training is much preferred. 31 Sarah Naami - Tripoli Center Council Mun 2.4 Source Material: However, I do have a point to note, I would like to say that this remote communication and training has not been effective, I want the meetings to be more face-to-face, the online platform is not effective. As a municipality, we have started to go back to the office, we are back to normal, they need to also start coming back to normal. In comparison to other projects, they have all started back. 31 Sarah Naami - Tripoli Center Council Mun 2.4 Source Material: For the elected members, they did not train us at all. They should be training the elected members; they should really need it as a priority. The training needs to be physical, not online, and practical. I think the training needs to support the development of smart cities. We need to be looking at these things to develop the future of the municipalities. Not some simple training. We need to be developing and building capacity. 32 Samir Kershman - Zwara Mun 2.4 Source Material: Has Taqarib improved the municipality’s capacity? Very limited, a couple of workshops and trainings, but I don’t believe that has not been so effective. So yes, they did training, but I think it’s very limited. Case Code 10. Mohamed Mehraz Mun 2.6 Source Material: Taqarib has not been able to bring together the Sabha municipality with the local population, there has been no result of this. 24. Khaled Alhoudery - Sabha Municipality Mun 2.6 Source Material: Do you feel Taqarib has supported the municipality for the local population to see this? No there wasn’t anything substantial for the local population to see. They have helped us with some trainings, nothing substantial like I said. There was supposed to be a project for the local park, renovating it, we started that project, but it is not yet completed, if this was completed for example that would be something that the local population can see. 27. Khalid Missallati - Benghazi Mun 2.6 Source Material: Do you feel Taqarib has supported or affected the Municipality? 62 No not at all. There is no effect. We need to face Taqarib and ask them what they have done for us! They have not done anything for us that is tangible. 28. Naser Alawjaly - Benghazi Mun 2.6 Source Material: Do you think that Taqarib has improved the relationship between the Municipality and the local population? Taqarib has not done much work to be able to say this, the municipality has not been affected much in its project implementation, so the local population are not seeing anything. Case Code 10. Mohamed Mehraz Mun 2.7 Source Material: Did the Taqarib project support the Municipalities work? The efforts by Taqarib have been very simple, they did not reach our expectations or the expectations of the population. It is perceived as a simple project and many of the tasks have not been implemented nor have they been completed. I received a letter from the Taqarib management to apologize to me stating they have not had the capability to complete the project. I don’t believe there are strong deliverables for the Sabha municipality by the Taqarib team. You can say they have been weak in delivering and communicating to us. They have also tried to provide us with things that we really did not have any use for, I did not feel there is no cooperation between us as the municipality and Taqarib. Taqarib brought me something ready that they thought that I need it, but I did not need it at all. They have not been coordinating with us. 10. Mohamed Mehraz Mun 2.7 Source Material: What should Taqarib do, to improve their work toward you? They should present options for requirements before just going to implement it. We should have a discussion first and consultative sessions first. They should listen to us first before going to implement a project. 10. Mohamed Mehraz Mun 2.7 Source Material: Taqarib does not seem to have any preparations or funding for works as they are not effective. I don’t believe they have much funding. To give you an example, UNDP has been implementing projects for 1,200 solar panels, in comparison to Taqarib 30 solar panels. It is incomparable to what others are doing in Sabha. I believe Taqarib has been focusing on Tripoli and its outskirts more from what I can see in the social media, we as Sabha are not getting at all the same treatment. 11. Osama - As Sahel Mun 2.7 Source Material: How can Taqarib support the Municipality further? There needs to be an awareness campaign to be implemented further, as there is a real lack of awareness of Taqarib and how to deal with them especially in the remote municipalities, I know that for Tripoli they do not have this issue, but for us it is important to educate and provide awareness for the locals and municipality members. They should also look at providing training in Budgets, how to manage the municipality and structure it. They should look to provide us with a stipend, where we can start providing requests for proposals for maintenance support. The increase in communication with the Municipality is important, they need to be open and transparent. In addition, they should look to have an office within the Municipality, and they should be more integrated with the municipality. This allows for sustainability and supports the awareness 63 campaign. We are communicating to the Benghazi office, this is not right, we should have our own Office with Taqarib, I think it does not work to have one Taqarib office for the whole of the east. There needs to be more offices and one in the green mountains as a whole this is important. 24. Khaled Alhoudery - Sabha Municipality Mun 2.7 Source Material: How did you first become involved with LGCS? We started with them in Tunis in March 2020, they presented their ideas and support for the municipality, and we were very happy with their presentation. It seemed very useful to our municipality. But when we started at the implementing phase, it was very simple, very shallow support. For example, they would provide us with some small tools to support repairing the roads, things like wheelbarrow, it wasn’t that much substantial support. When we met in Tunis, we were not told these were going to be simple, we thought it would be more than this, it was presented this way in Tunis by them. 24. Khaled Alhoudery - Sabha Municipality Mun 2.7 Source Material: Do you feel Taqarib can improve their support toward the municipality’s capabilities? There needs to be an assessment of the road map, if it needs changing then they need to change it, the strategy needs to be looked at. They are not discussing with us and really understanding what our requirements are. There needs to be more communication between us. 28. Naser Alawjaly - Benghazi Mun 2.7 Source Material: Do you Feel Taqarib has supported the development in the municipality? No there has been no development that has been noticed, Taqarib has weak capabilities and from what I have seen, they have not at all. 30. Ahmed - Awbari municipality Mun 2.7 Source Material: The project has not done anything. We agreed on some of the things they could support us with, they advised that they support us with the COVID pandemic. We also agreed on a project that would mean 30 streetlights to light up certain streets within the municipality. We were advised that they had limited resources, and this is all that they could support us with. However there has been no implementation of these things, we have just been talking about this. 30. Ahmed - Awbari municipality Mun 2.7 Source Material: Up until now they have not really given us anything. The project has not done anything. 31 Sarah Naami - Tripoli Center Council Mun 2.7 Source Material: Was Taqarib able to support the capability of the Municipality? Yes, they supported us in the trainings, and they were very supportive in the time of COVID also, they provided supplies. 31 Sarah Naami - Tripoli Center Council Mun 2.7 Source Material: Their efforts are limited, it is not that the local population has noticed a difference. 31 Sarah Naami - Tripoli Center Council Mun 2.7 Source Material: For Taqarib to be more effective, they need to also have some deeper ideas and communicate with us more to be able to support us practically and sustainably. 64 32 Samir Kershman - Zwara Mun 2.7 Source Material: Do you feel Taqarib has been able to support the municipality services? They have not really improved our services. Even the park that they intended to create, they have not even implemented, it is still open I believe. And they started discussions with us on another project, but they didn’t even close out the first one. The individuals that came to us were very professional and capable individuals. Case Code 10. Mohamed Mehraz Mun Q2 A Source Material: Taqarib doesn’t know our requirements, and then on top of this I do not know their capabilities as an organization. How can we work together on any project with this kind of arrangement? I need to know their capabilities and they need to know our requirements as the starting point, this is not happening. 10. Mohamed Mehraz Mun Q2 A Source Material: Do you feel Taqarib has improved the operations of the Sabha Municipality? There is a slight improvement but not to the extent that we were expecting. Taqarib has not been able to bring together the Sabha municipality with the local population, there has been no result of this. Taqarib does not seem to have any preparations or funding for works as they are not effective. I don’t believe they have much funding. To give you an example UNDP has been implementing projects for 1,200 solar panels, in comparison to Taqarib 30 solar panels. It is incomparable to what others are doing in Sabha. I believe Taqarib has been focusing on Tripoli and its outskirts more from what I can see in the social media, we as Sabha are not getting at all the same treatment. 11. Osama - As Sahel Mun Q2 A Source Material: How has Taqarib supported the local municipality? Taqarib brought 5 x wheelbarrows for the Sahel Municipality. There were discussions on how to distribute them, we asked them if there was going to be more coming, [but] we haven’t had any response from them. Other than this, there is nothing else that we have seen tangible from them. 11. Osama - As Sahel Mun Q2 A Source Material: What are the challenges that the municipality faces with regards to these projects? There is a lack of strong and skilled persons to ensure that the projects are implemented. 18. Faisal - Tripoli Municipality Council Mun Q2 A Source Material: How do you feel the Taqarib project has supported you? I have noticed that they are intending to do some work. Their intentions are in the right direction; however, I think they are not strong enough to really support us. Let me give you an example, there was a project on supporting the road repair, so the machinery that they should be providing should be suitable for the job, they did provide us with machinery, but they are not suitable for the job, they gave us a small generator to power the machinery, it is too small, they gave us a rolling machine, which is also too small for our requirements. Ultimately, I was planning to have our own road repair team rather than contracting it out, however I am still using contractors to conduct road repairs as the items provided are just not suitable. So, they are just sitting there not being used, as far as I am concerned the project is still not closed. The other issue is that they have a delay in the process, they are taking a long time to go from the planning stage to implementation phase. This is an underlying issue. They are taking too long to process 65 things. I would also like to comment on their communication with us. They should be collaborating with us more in the technical details so that we can really have tangible results, if they can talk more with us, we can let them know ok these things are suitable and these things are not, we should be part of the procurement process for getting these items to support our initiatives. We have technically skilled persons in our Municipality, engineers who can provide the technical knowledge of what we need for a sustainable project. 27. Khalid Missallati - Benghazi Mun Q2 A Source Material: Do you feel Taqarib has supported or affected the Municipality? No not at all. There is no effect. We need to face Taqarib and ask them what they have done for us! They have not done anything for us that is tangible. 28. Naser Alawjaly - Benghazi Mun Q2 A Source Material: Do you Feel Taqarib has supported the development in the municipality? No there has been no development that has been noticed, Taqarib has weak capabilities and from what I have seen, they have not at all. Case Code 18. Faisal - Tripoli Municipality Council Mun Q2 B Source Material: Do you think Taqarib has supported the Tripoli Municipal council services so that people take the municipal more seriously? This is what I know their main goal is as the Taqarib project, but I will be honest with you, there isn’t any result on the ground. The people have not seen anything tangible. 31 Sarah Naami - Tripoli Center Council Mun Q2 B Source Material: Did Taqarib support the municipality to improve its image with the local population? It has; however, I do have a point to note, I would like to say that this remote communication and training has not been effective, I want the meetings to be more face to face, the online platform is not effective. As a municipality, we have started to go back to the office, we are back to normal. They need to also start coming back to normal. In comparison to other projects, they have all started back. 31 Sarah Naami - Tripoli Center Council Mun Q2 B Source Material: Did Taqarib support the municipality to improve its image with the local population? It depends on municipality to municipality; I think there is support in Tripoli municipality and the effect has been able to show through to the local population. They haven’t been able to change the view of the local population to the point that what we are expecting. Their efforts are limited, it is not that the local population has noticed a difference. 32 Samir Kershman - Zwara Mun Q2 B Source Material: Do you think Taqarib has had any effect on the local population through the municipality? No, I do not believe Taqarib has supported in this. Taqarib has supported but not to this extent. Taqarib is working with us and is visiting us tomorrow to support the roads repair, however they have not provided such an extent of support to the point that the population has noticed. 66 ANNEX 5: SOURCES OF INFORMATION Municipalities Selected Included are two municipalities from each region with different levels of functionality (low, medium, and high); also, those with minorities. Municipality selection was facilitated by LGCS and by Libyan ET members familiar with circumstances in the country. Municipal Council Region Functionality Comments Sabha South High Aubari South low Includes minority (Tuareg and Tebu) Tripoli Center West High Zawara West Medium Includes minority (Amazigh) Assahel East Medium-low Benghazi East High Key Informants Interviews (KIIs) LGCS provided names and contact information for municipality and CSO informants contacted by the ET for this evaluation. Donors were identified by the ET. Informant name Informant type Title/Role Interview Date Category Location AlSaeh Abu Drbala Emhemmed Grants recipient (SEB-001) Child and Youth Friendly Baladyat Foundation: Local Development Experts Group - Tripoli 11/09/21 CSO Tripoli Salih Abduljalil Shahhat Alaswad Grants recipient (SEB-003) General Union of Sebha University Students: Baladiati – Sebha 11/09/21 CSO Sabha Hudi Mohammed Alsaqouri Grants recipient (BEN-001) Nesaa Raedat Association: Women Salon – Tripoli 11/09/21 CSO Tripoli Naji Aboulqasim Aboubakr Pokr Grants recipient (SEB-002) Friends of People with Disabilities Organization: Tayseer – Tripoli 11/09/21 CSO Tripoli Abdurrahman Alazhari Grants recipient (TRI-005) Ehsan Organization: Recycle IT –Tripoli 19/09/2021 CSO Zintan Salim Jummah Bin Hamid Grants recipient (BEN-003) Impact Platform: Reflection – Benghazi 21/09/2021 CSO Benghazi Salem Matouq Khamkhem Grants recipient (TRI-003) ALlibeya Organization: Menassa (Platform) – bani Waleed 21/09/2021 CSO Bani Waleed Abdulwahab al￾Kamel Non-Government stakeholder President of Ubari Business Union – Aubari 22/09/2021 CSO Aubari Saad Omar As-Saiti Grants recipient (BEN-002) LASTM: Reviving Neighboring Garden – Benghazi 25/09/2021 CSO Benghazi Thiria Wahiyba Grants recipient (TRI-001) O2 Organization: Regulating use of Plastic bags – Tripoli 23/09/2021 CSO Tripoli Hend Bishari CSO – C4C CSO Founder –Benghazi 11/09/21 CSO Benghazi Abouazom Aboujdiryha Sebha Regional Program Manager interacts with southern partner municipalities 10/09/21 DAI Tripoli 67 Informant name Informant type Title/Role Interview Date Category Location Mohammed Alkawash Tripoli Regional Program Manager interacts with western partner municipalities 13/09/2021 DAI Tripoli Hunter Keith Project manager LCGS Chief of Party 16-17/09/21 DAI Tunis Husen Gerbi Tripoli PDO 1 interacts with western partner municipalities 22/09/2021 DAI Tripoli Abdussalam Imhimmed Sebha PDO 1 interacts with southern partner municipalities 22/09/2021 DAI Sabha Otman Gajiji Senior Technical Advisor interacts with all partner municipalities 22/09/2021 DAI Tripoli Hassanin Hamid Taqarib DCOP interacts with all partner municipalities 22/09/2021 DAI Tunis NourAlddeen Alhouni Benghazi PDO 1 interacts with eastern partner municipalities 22/09/2021 DAI Benghazi Ali Sada Taqarib staff CSOs component manager 23/09/2021 DAI Tunis Ahmed Rauhym Taqarib staff MEL Manager 27/09/2021 DAI Tunis Danielle Doubt Project subcontractor - Altai Project Manager 04/10/21 DAI Tunis Kamran Parwana Project subcontractor - Altai Director 04/10/21 DAI Tunis Eng. Mohamed Abuhelga Libya National Government MOLG Advisor 20/09/2021 MOLG Tripoli Maisaa Ragibi Libya National Government MOLG Head of International Cooperation 22/09/2021 MOLG Tripoli Osama Elkezza Project Municipal Client rep Project Officer Director – As Sahel 19/09/2021 Mun Adm Sahel Khalid Al-Hadhiri Project Municipal Client rep Officer Project Engineer (Officer Project Director 2019–2020) – Sabha 23/09/2021 Mun Adm Sabha Khaled Al-Maslati Project Municipal Client rep HR & Training Director – Benghazi 26/09/2021 Mun Adm Benghazi Naser Aloajili Project Municipal Client rep City Manager – Benghazi 26/09/2021 Mun Adm Benghazi Assiel Alatoshi Project Municipal Client rep City Manager – Zwara 26/09/2021 Mun Adm Zwara Mohamed Mahrez Ali Project Municipal Client rep Officer Project Director – Sabha 20/09/2021 Mun Elect Sabha Faisal Shneiba Project Municipal Client rep Municipal Council member (ex-Project Officer Director) – Tripoli 22/09/2021 Mun Elect Tripoli Ahmed Matco Project Municipal Client rep Ubari mayor – Aubari 28/09/2021 Mun Elect Aubari Sarah Naami Tripoli Council elected member Elected Municipal Council Member – Tripoli 27/09/2021 Mun Elect Tripoli Samir Kershman Zwara Council elected member Elected Municipal Council Member – Zwara 27/09/2021 Mun Elect Zwara Joe Scheibel USAID - funder Head, Democracy and Governance, Libya – Tunisia (until July 2021) 10/09/21 USAID Frankfurt Merve Kania Donor (GIZ) GIZ Libya Decentralization and Municipal Support 10/06/21 Donor Tunis Nancy Bergeron Donor (Canada) Development Secretary, Libya Embassy 02/12/21 Donor Tunis 68 Informant name Informant type Title/Role Interview Date Category Location Marion Saurel Donor (EU) Program Manager, Delegation of the EU to Libya 01/12/21 Donor Tunis Mario Hauter Donor (Germany) Desk Officer, Federal Foreign Office Email 12/01/2021 Donor Germany Summary: Ministry of Local Government – 2 Municipality administration – 5 Municipality elected – 5 CSOs – 11 DAI – 12 (2 are Altai subcontractors) USAID official – 1 Donor representative – 4 Total: 40 Key Informants 69 ANNEX 6: EVALUATION TEAM MEMBERS’ QUALIFICATIONS ▪ Team Leader Dr. Andy Tamas is a governance specialist with over 40 years’ experience supporting organizational development in fragile and emerging contexts. His 2018 dissertation, Contextually Appropriate Institutional Development in Fragile States: Lessons from Policy Implementation in Afghanistan, drew upon decades of technical advisory and evaluation work in Iraq, Afghanistan, Qatar, Yemen, Libya, Egypt and elsewhere. This assessment of LGCS is the tenth USAID evaluation in which he has been involved, most often as Team Leader. ▪ Hala Bugaighis has 20 years’ experience with institutional development in Libya. She is a lawyer and member of the Libyan Bar Association. With her extensive background in project management, evaluation, performance management, and other areas, and her fluency in both Arabic and English, she brings outstanding capabilities to this evaluation. ▪ Tamim Rashed is British-educated Libyan consultant experienced in the monitoring and evaluation field in Libya. He has a Master’s in War Studies from Kings College London, and a bachelor’s degree in business information systems from the University of Wales. As a researcher and political analyst, he is committed to supporting the Libyan democratic transition, in particular the constitution building process. ▪ Isam Areebi, who unfortunately left the project at the half-way point, has a master’s degree in Engineering Management, and extensive experience with project evaluation and institutional development in Libya. He was instrumental in drafting and adjusting our semi-structured questionnaire interview guide, which formed the basis of our evaluation’s data collection process. ET contact information Expertise Name E-mail Phone Team Lead Andy Tamas andy@tamas.com +1 613 853 3805 Field researcher Hala T. Bugaighis hala.bugaighis@gmail.com 00 218 91 373 10 57 Field researcher Tamim Rashed tamim.m.rashed@gmail.com 00 218 91 560 55 58 M&E specialist\ Research assistant Ahmed Trabelssi (METAL Staff) atrabelssi@ibtci.com 0021629656749 70 ANNEX 7: BIBLIOGRAPHY LGCS Activity Documents 1. Altai Consulting 2021 - Y2Q4 SRS LGCS Taqarib Presentation 25SEP2021Indicator Progress Update_20210619 2. Taqarib - Quarterly Report FY21Q1 – Final 3. Taqarib - Quarterly Report FY21Q2 – Final 4. Taqarib AMELP Y2 Submission - 2020-09-30 (ver 003) 20.11.19 5. Taqarib Y1 Workplan - Final 6. Taqarib Y2 Workplan – Final 7. Signed Assistance_72067019CA00001 8. Signed Mod No. 01 Other Documents Chianca, T. (2008). The OECD/DAC Criteria for International Development Evaluations: An Assessment and Ideas for Improvement. Journal of Multidisciplinary Evaluation, 5(9). Collier, Paul 2009 - Wars Guns and Votes – Democracy in Dangerous Places. New York: Harper. Delegation of the European Union (2021). Strategic Evaluation of EU support to community stabilization, local governance and local development in Libya – Draft Synthesis Report SIEA-2018-1337. Accessed 20 May 2021. Elfeituri, N. (2020). Why Civil Society is Libya’s Best Defence against the COVID-19 Pandemic. https://merip.org/2020/07/why-civil-society-is-libyas-best-defense-against-the-covid￾19-pandemic/. Accessed 2022-01-15. Esper, H., Fatehi, Y. K., & Baylor, R. (2021). Developmental Evaluation in Theory versus Practice: Lessons from Three Developmental Evaluation Pilots. Journal of Multidisciplinary Evaluation, 17(40). EU Libya Office (2021). EU support to local governance and community stabilization. (PowerPoint presentation) June 2021: Baladiyati Steering Committee. GIZ. (2021). Building for Peace: Transformative Reconstruction in the MENA Region. Eschborn, Germany: GIZ. Government of Libya - Interim National Transitional Council (NTC) (2012) Law No. 59 of 2012 Concerning the Local Administration System. Houdret, A., & Harnisch, A. (2017). Decentralisation in Morocco: The Current Reform and its Possible Contribution to Political Liberalisation. Bonn: German Development Institute. Kuzel, A. J. (1992). Sampling in qualitative inquiry. Lucas, B. (2014). Capacity development at the national level in fragile and conflict-affected states. GSDRC - Applied Knowledge Services. McCracken, G. (1988). The long interview (Vol. 13). Sage Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: Jossey-Bass. METAL Site Visit reports of DAI provided training, March 2020 – December 2021. Muntaner, C., & Gómez, M. B. (2003). Qualitative and quantitative research in social epidemiology is complementarity the only issue. Gac Sanit, 17(Suppl 3), 53-57. National Municipal Councils Association (NMCA) - Libya. (2021). Libya Program Summary Description. Retrieved from https://knowledge-uclga.org/the-national-municipal￾councils-association-nmca?lang=fr. Accessed 2021-11-03 Paffenholz, T., & Spurk, C. 2006. Civil Society, Civic Engagement, and Peacebuilding. Working Papers - Conflict Prevention & Reconstruction, 36. 71 ReliefWeb. (2021). Baladiyati Project Phase 2 - Kickoff Event Press Release. Retrieved from https://reliefweb.int/report/libya/ministry-local-government-aics-undp-and-unicef￾hosted-kick-event-eu-funded-baladiyati Accessed 2021-11-03 Ruffa & Evangelista 2021 - Searching for a middle ground? A spectrum of views of causality in qualitative research. Italian Political Science Review. United Cities and Local Governments of Africa 2020 - Local Authorities Roadmap December 2020 (http://www.knowledge-uclga.org/IMG/pdf/libya_roadmap_2020.pdf) United States 117th Congress (2021) – Libya Stabilization Act https://www.congress.gov/bill/117th-congress/house-bill/1228 Accessed 2012-09-25 USAID 2015 - Qualitative methods - Module 15 USAID Evaluation Policy 2011 updated 2016 USAID Evaluation SOW Checklist Verkoren, W., & Kamphuis, B. (2013). State Building in a Rentier State: How Development Policies Fail to Promote Democracy in Afghanistan. Development and Change, 44(3), 501-526. World Bank Group. (2001). Administrative Decentralization. Public Sector Governance - Decentralization & Subnational Regional Economics.