This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Ritu Nayyar-Stone, Katie Mark, Adrian Lupusor, Ion Beschieru, Jessica Wallach, Samantha Downey, Anna Solovyeva, and Samantha Austin from NORC at the University of Chicago. DRG LEARNING, EVALUATION, AND RESEARCH (DRG-LER) II ACTIVITY COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 DRG LEARNING, EVALUATION, AND RESEARCH (DRG-LER) II ACTIVITY COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION May 11, 2022, Revised June 24, 2022 Prepared under Contract No.: GS-10F-0033M / 7200AA18M00016, Tasking N055 Submitted to: USAID DRG Center Submitted by: Ritu Nayyar-Stone, Team Leader Katie Mark, Local Governance Evaluator Adrian Lupusor, Local Government Finance Expert Ion Beschieru, Decentralization and Local Service Delivery Expert Jessica Wallach, Qualitative Specialist Samantha Downey, Quantitative Specialist Samantha Austin, Evaluation Support Anna Solovyeva, Quantitative Specialist Contractor: NORC at the University of Chicago 4350 East West Highway, 8th Floor Bethesda, MD 20814 Matthew Parry E-mail: parry-matthew@norc.org Cover Photo Credit: The cover photo was captured by Moldovan research firm IMAS during a site visit at a Citizen Information and Service Center instituted by the Comunitatea Mea Activity in Straseni and features the exterior of the Center. DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | i ABSTRACT NORC at the University of Chicago (NORC) conducted a mid-term performance evaluation of the United States Agency for International Development (USAID)/Moldova-funded Comunitatea Mea (CM) Activity. The evaluation addressed the following four main questions: 1) How is CM making progress toward the achievement of its goals and objectives? 2) To what extent were the approaches employed by CM effective in reaching the stated results? 3) How effective is CM’s phased approach? How feasible is the total number of LPAs over the course of the project? and 4) Looking forward and building on the successes and failures of CM, which areas should USAID support further to build stronger local governments and better local democracy where USAID has a competitive advantage? The evaluation team a mixed-methods approach using a combination of desk review, focus group discussions (FGDs), key informant interviews (KIIs), a telephone survey, and data gathered at Citizen Information and Service Centers (CISCs) instituted by CM. The evaluation found that CM was much appreciated by staff at Local Public Authorities (LPAs) due to its rich menu of assistance and flexibility, and successes in efforts to improve the quality of and access to services and own source revenues. Citizens also noted greater opportunity for engagement in governance. Several approaches were seen as successful and are recommended for continuation, however, smaller LPAs cited the lack of human and financial resources in sustaining several approaches after CM ends. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | ii TABLE OF CONTENTS ABSTRACT I EXECUTIVE SUMMARY 1 EVALUATION DESIGN 1 MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 2 EVALUATION PURPOSE AND QUESTIONS 7 PURPOSE 7 EVALUATION QUESTIONS 7 PROJECT BACKGROUND 8 METHODS AND LIMITATIONS 10 DESK REVIEW 11 QUANTITATIVE DATA COLLECTION – TELEPHONE SURVEY 11 QUALITATIVE DATA COLLECTION – KIIS AND FGDS 12 CISC OBSERVATIONAL DATA 13 LIMITATIONS 13 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 15 FINDINGS 15 EQ1: HOW IS COMUNITATEA MEA MAKING PROGRESS TOWARD THE ACHIEVEMENT OF ITS GOALS AND OBJECTIVES? 15 EQ2: TO WHAT EXTENT WERE THE APPROACHES EMPLOYED BY IREX/COMUNITATEA MEA EFFECTIVE IN REACHING THE STATED RESULTS? 27 EQ3: HOW EFFECTIVE IS CM’S PHASED APPROACH? HOW FEASIBLE IS THE TOTAL NUMBER OF LPAS OVER THE COURSE OF THE PROJECT? 36 EQ4: LOOKING FORWARD AND BUILDING ON THE SUCCESSES AND FAILURES OF COMUNITATEA MEA, WHICH AREAS SHOULD USAID SUPPORT FURTHER TO BUILD STRONGER LOCAL GOVERNMENTS AND BETTER LOCAL DEMOCRACY WHERE USAID HAS A COMPETITIVE ADVANTAGE? 40 CONCLUSIONS 43 RECOMMENDATIONS 46 ANNEXES 51 ANNEX A – EXTENDED METHODOLOGY 52 DOCUMENT REVIEW 52 QUANTITATIVE DATA COLLECTION – TELELPHONE SURVEY 52 QUALITATIVE DATA COLLECTION 53 ANNEX B – DETAILED ANALYSIS FOR EQ 1 58 ANNEX C – DETAILED ANALYSIS FOR EQ 2 63 ANNEX D – EVALUATION STATEMENT OF WORK 86 USAID/MOLDOVA MID-TERM PERFORMANCE EVALUATION SCOPE OF WORK 86 ANNEX E – DATA COLLECTION INSTRUMENTS 93 CATI 93 INTRODUCTION AND CONSENT STATEMENT – KEY INFORMANT INTERVIEWS 119 KII INTERVIEW GUIDE – LPA OFFICIALS 119 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | iii KII INTERVIEW GUIDE – IMPLEMENTING PARTNERS 124 KII INTERVIEW GUIDE – GOVERNMENT OF MOLDOVA 127 KII INTERVIEW GUIDE – LOCAL GOVERNANCE ASSOCIATIONS 130 KII INTERVIEW GUIDE – DONORS 134 FGD GUIDE – CSO GROUPS AND ACTIVE AND NON ACTIVE CITIZENS 137 QUESTION BANK 149 ANNEX F – SOURCES OF INFORMATION 155 DOCUMENTS REVIEWED 155 KEY INFORMANTS 157 PARTICPATING LPAS – CATI 160 ANNEX G – DISCLOSURE OF CONFLICTS OF INTEREST 162 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | iv TABLES Table I: Evaluation findings and conclusions ................................................................................................. 2 Table II: Evaluation Recommendations ...........................................................................................................5 Table 1: CM approaches by component.........................................................................................................9 Table 2: Telephone interview respondents................................................................................................. 12 Table 3: Statistically significant difference in “high” improvement in service quality across cohorts ..................................................................................................................................................18 Table 4: Statistically significant difference in “A lot” of increase in types of citizen engagement across cohorts ....................................................................................................................................22 Table 5: Main challenges in using CM approaches in the future .............................................................34 Table 6: Summary of approaches, limitations, and interest in continuing.............................................35 Table 7: Reasons for interactions with other LPAs, response by Deputy Mayors ............................39 Table 8: Types of interactions with other LPAs, response by Deputy Mayors...................................39 Table 9: Most frequently suggested new approaches, by objective .......................................................42 FIGURES Figure 1: Comunitatea Mea partner communities by cohort (Comunitatea Mea Annual Report, 2021) .......................................................................................................................................................8 Figure 2: Research design ..................................................................................................................................11 Figure 3: EQ1 Key findings................................................................................................................................15 Figure 4: Number of LPAs with high, medium, and low improvement in service quality..................16 Figure 5: Number of LPAs with high, medium, and low improvement in access to municipal services.................................................................................................................................................19 Figure 6: Visit time at the CISCs......................................................................................................................21 Figure 7: Number of LPAs with high, medium, and low increase in citizens’ engagement in local governance ...........................................................................................................................................22 Figure 8: Mostly Positive or no Impact of Various CM Activities related to Decentralization Policy (# of LPAs)...............................................................................................................................26 Figure 9: Factors contributing to an increase in own source revenues.................................................26 Figure 10: EQ2 key findings................................................................................................................................. 27 Figure 11: What is your LPA doing to collect feedback on its service quality from your citizens? (n=314).................................................................................................................................................30 Figure 12: Which of the following communication strategies have you initiated within your LPA? (N=323)................................................................................................................................................31 Figure 13: CALM challenges and capacity ........................................................................................................33 Figure 14: EQ3 key findings................................................................................................................................. 36 Figure 15: EQ4 key findings................................................................................................................................. 40 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | v ACRONYMS ADEPT Association of Directors of Environment, Economy, Planning & Transport CATI Computer Assisted Telephone Interview CDP Community Development Project CDCS Country Development Cooperation Strategy CDS Community Development Strategy CISC Citizen Information and Service Center CM Comunitatea Mea COG Community Oversight Group CSO Civil Society Organization CUPS Unified Centers for Public Services Delivery DC Decentralization Corners DRG-LER II Democracy, Human Rights, and Governance – Learning, Evaluation, and Research Activity EQ Evaluation Question ET Evaluation Team EU European Union FEsZ Free Economic Subzones FGD Focus Group Discussion FOCAS Functional Organization Capacity Assessment GESI Gender Equality and Social Inclusion GIS Geographic Information System GIS GoM Government of Moldova IFMC Internal Financial Management Control IMAS IMAS-CERCETARE ȘI STRATEGII SRL IMC Inter-Municipal Cooperation IMCS Internal Managerial Control System IP Implementing Partner IR Intermediate Result IREX International Research & Exchanges Board IT Information Technology KII Key Informant Interview LED Local Economic Development LPA Local Public Administration LTD Local Tax Development MEL Monitoring, Evaluation, and Learning MSPW Municipal Strategic Planning Workshop NDI National Democratic Institute CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | vi NGO Non-governmental Organization NICS National Internal Control Standards NORC NORC at the University of Chicago PAP Public Asset Management Plan PRM Performance and Risk Management RED Revenue Enhancement Development SIAP Service Improvement Action Plan SP Strategic Plan UNDP United Nations Development Programme USAID United States Agency for International Development CONTRACT NO. GS-10F-0033M / ORDER NO. AID-OAA-M-13-00013 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 1 EXECUTIVE SUMMARY As part of the DRG Learning, Evaluation, and Research II Activity (DRG-LER II), NORC was contracted to do a mid-term performance evaluation of the United States Agency for International Development (USAID) Comunitatea Mea (CM) Activity. CM is a $20.5 million five-year (2018-2023) flagship local governance program funded by USAID/Moldova and implemented by IREX and its partners Tetra Tech ARD, National Democratic Institute (NDI), and Urban Institute. CM’s goal is to strengthen local government to become more effective, transparent, and accountable to citizens through four primary components: 1) Improved quality and access to municipal services, 2) Citizens meaningfully engaged in good governance, 3) Decentralization policy advanced, and 4) Increased locally owned revenues and improved financial management practice. CM supports Development Objective 1 of USAID/Moldova’s Country Development Cooperation Strategy (CDCS): “Strengthened Participatory Democracy” through Intermediate Result 1.2 “Responsiveness of Targeted Public Institutions Strengthened.” CM partners with Local Public Administrations (LPAs) to implement a tailored package of assistance guided by the four primary components. At the time of the evaluation, CM had partnered with 87 total LPAs of varying sizes across the country. NORC partnered with IMAS, a Moldovan research firm to complete a mixed methods evaluation to assess the progress of CM in achieving its goals and provide recommendations to inform the strategic design of future USAID local governance programs in Moldova. The evaluation was guided by the following four main questions: 1. How is Comunitatea Mea making progress toward the achievement of its goals and objectives? 2. To what extent were the approaches employed by IREX/Comunitatea Mea effective in reaching the stated results? 3. How effective is CM’s phased approach? How feasible is the total number of LPAs over the course of the project? 4. Looking forward and building on the successes and failures of Comunitatea Mea, which areas should USAID support further to build stronger local governments and better local democracy where USAID has a competitive advantage? EVALUATION DESIGN The evaluation design included a document review of 66 documents as well as data collection using mixed methods. This included focus group discussions, key informant interviews, a quantitative telephone survey, and observational data gathered through site visits at Citizen Information and Service Centers (CISCs) instituted by CM. Data was collected between November 2021 and April 2022. Qualitative data collection was collected from a purposive sample of 15 LPAs across all three cohorts. It included 40 Focus Group Discussions (FGDs) (10 with CSOs, 15 with citizens active in governance, and 15 with non-active citizens), and 55 key informant interviews (KIIs) of LPA staff, IREX implementation team, USAID/Moldova, implementing partners and donors in the local governance sector, and select members of the government of Moldova (GOM). After data collection, transcripts were translated then analyzed and coded in Dedoose using content analysis. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 2 Observation data of CICSs was collected data from six LPAs (two from each cohort) using an observational checklist, a user survey, and five questions asked to CISC operator to assess the operationality and effectiveness of the CISCs in providing a one-stop-shop for citizens to submit administrative service requests to their LPA. Quantitative data collection included a survey programmed in Qualtrics and applied via telephone in 66 LPAs. 361 LPA staff completed the survey including Mayors, Deputy Mayors, Specialists in budget/finance/economy, Specialists on Communal Services, Secretaries of Local Council, and Other Service Specialists. Quantitative data was cleaned, then analyzed using statistical software. MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS A summary of main findings, conclusions, and recommendations from the evaluation are presented in the table below. Table I: Evaluation findings and conclusions FINDINGS CONCLUSIONS EQ1. How is Comunitatea Mea making progress toward the achievement of its goals and objectives? ▪ CM helped beneficiary LPAs improve the quality of and access to several services at least moderately. ▪ Where CISCs were established, citizens appreciated the improvement in administrative services. ▪ CM was successful in increasing citizen engagement in some areas and had mixed results in others. ▪ There were some successes regarding decentralization policy, but adoption of some drafts is pending or unlikely to happen soon. ▪ CM had a positive impact on fee and tax revenues of beneficiary LPAs, as well as asset management revenues, thourgh to a lesser extent. ▪ The capacity building component on financial management was highly appreciated by LPAs, but sustainability issues were identified. ▪ Overall CM helped LPAs improve the quality of and access to several services at least moderately through a variety of approaches. ▪ CM contributed to the improvement in LPAs’ transparency, and some initial improvements in meaningful citizen participation, engagement and trust. ▪ The civic engagement component was important for ensuring the accountability of LPAs in implementing the infrastructure projects, but little emphasis was placed on post-execution monitoring. ▪ In LPAs without a CISC the improvement in quality and access to administrative services as perceived by LPA officials and citizens is much less obvious compared with LPAs where CISCs have been created. ▪ CM’s input and assistance to GoM on advancing decentralization policy reforms was to a large extent demand-driven and resulted in a considerable number of drafts and proposals ▪ The lack of political will at the national level diminished the impact of the technical assistance provided by CM to the GoM in promoting decentralization reform and amalgamation; there was also limited involvement or use of CM partner LPAs in promoting decentralization reform policies; more could have been done. ▪ Most LPA officials believe that CM helped their LPA increase their own source revenues; there was appreciation for the diversity of approaches available to LPAs under this component. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 3 FINDINGS CONCLUSIONS EQ2. To what extent were the approaches employed by IREX/Comunitatea Mea effective in reaching the stated results? ▪ CM offered a rich menu and flexibility in choosing some approaches. ▪ Despite the advantages of flexibility, the multiplicity of approaches created some challenges. ▪ Starting with the strategic planning approach was helpful for many LPAs. ▪ Several CM approaches were effective in engaging citizens and getting their feedback ▪ The communication strategy was inclusive but faced some challenges ▪ Community Development Projects (CDPs) were appreciated but there were several challenges in their implementation. ▪ Evidence shows space for more efforts to connect, inform, and coordinate the work done by CM regarding decentralization at the national and local level ▪ Results were mixed regarding the capacity and use of CALM ▪ CM provided a flexible and tailored approach to specific needs and priorities of beneficiary LPAs for financial management. ▪ High interest from LPAs to continue using CM approaches, but they face challenges in ensuring their institutionalization and sustainability. ▪ The “menu” of approaches offered by CM was very rich and there was some flexibility for LPAs to choose from, but a minimum package of approaches was mandatory to get the funding for the infrastructure project (CDP) which was a big draw. ▪ The community development strategies (CDSs) are key strategic planning documents that can increase the capacity of LPAs to attract grants, financing and investments for development. But their opportunities can be tapped only if the LPAs continue keeping these documents up-to-date and stay active in their fundraising, advocacy and budgetary planning activities. ▪ CM focused primarily on providing LPAs informational, technical and financial support, which visibly outpaced the support to the civil society. ▪ Given the relatively small scale of CDPs, LPA were frustrated with CMs feasibility studies and engineering requirements, which caused delays in implementation and, given rising inflation, large discrepancies between the initial budgets and de facto spending. ▪ Often, the capacity-building activities were attended by LPA representatives whose specialization was not entirely relevant to the topic discussed. This undermined the impact of these activities. ▪ One of the missed opportunities of CM is not enough emphasis on ensuring proper peer-to-peer learning and knowledge dissemination based on the individual experience with the implementation of various CM approaches. ▪ CM work at the local level in the beneficiary LPAs and the work at the central level to promote decentralization policy reform seemed to be to a large extent treated as separate and isolated tracks. ▪ CALM was used for specific activities as a service provider and as a partner in others, but the involvement and use of CALM is seen as insufficient by many respondents, especially by LPA officials. ▪ Given the current limited LPAs’ capacities, the institutionalization or external ownership of various approaches to incentivize/remind LPAs is key for ensuring sustainability and use beyond the CM project lifespan. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 4 FINDINGS CONCLUSIONS EQ3. How effective is CM’s phased approach? How feasible is the total number of LPAs over the course of the project? ▪ Given the number of target LPA beneficiaries, a phased/cohort approach was the best solution ▪ There was mixed feedback regarding implementation differences between cohorts ▪ There is some sharing of lessons learned among LPAs, but it is organic and has not been sufficiently stressed by CM ▪ Given the large number of target LPAs a phased/cohort approach was essential. Some respondents felt that even 25/26 LPAs in each cohort was a lot to work with each year. ▪ There is mixed perception regarding implementation differences among cohorts. Some stakeholders felt that the earlier cohorts benefited from more time and supervision from the CM project ▪ There is some organic sharing of lessons learned among cohorts and partner LPAs, but more could be done. EQ4. Looking forward and building on the successes and failures of CM, which areas should USAID support further to build stronger local governments and better local democracy where USAID has a competitive advantage? ▪ Several CM approaches were effective and could be useful to continue supporting ▪ Stakeholders suggested potential new approaches to expand effectiveness ▪ Improved donor coordination would be fruitful ▪ Several CM approaches are considered well worth continuation, especially if adjustments are made to improve their effectiveness and impact. These include: − Support for the development and implementation of strategic plans – especially for smaller LPAs – would be valuable. − CDPs were also found to be very useful, but it would be good if some of the more burdensome requirements could be streamlined, and it would be desirable if they could be completed earlier in the project to allow time for establishing sound maintenance and monitoring processes. − There is continuing interest in direct assistance with infrastructure and services including SIAPs. − CISCs were very much appreciated, and where they are there is expectation that they are sustainable. Other donors are interested in contributing as well, and USAID has developed useful training and materials to support future CISCs. − Measures to increase citizen participation and support civil society are seen as successful and desirable and should be expanded. ▪ New areas to consider for future programming included youth engagement, local economic development, and strengthening IT and digitalization. ▪ If USAID seeks to continue to support decentralization reform, recommendations include building LPA and CALM advocacy skills to improve their effectiveness with the GoM; better leverage the experience and relationships from direct work with LPAs; and improve coordination with other donors. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 5 Table II: Evaluation Recommendations Recommendations EQ1  Focus on fewer approaches to achieve critical results within each LPA. Going forward the ET suggests that USAID select a few key results/outcomes and then have fewer approaches that target that result.  Expand the current work with citizens and CSOs with more targeted and successful tools, to build their capacity and incentives to engage in local governance.  Engage the LPAs more in the decentralization policy discussion with the national government to create a critical mass of grass-roots support for the proposed reform measures.  Create more synergies with other donor projects to improve results in select LPAs.  Institutionalize financial management approaches and provide refresher training to sustain results in increasing own source revenues. EQ2  Refine processes for attendance at training; and composition of COGs to ensure correct representation of LPA staff and citizens respectively.  Have fewer approaches so LPAs are not over-taxed and to ensure better results and sustainability.  Start the CDPs at an earlier stage of the project (to avoid delays in implementation, changes in prices, and allow sufficient time for post-execution monitoring).  For future communications work, efforts should emphasize outreach and provision of information rather than hosting chats or discussions, which require more active moderation which overtaxed staff are not equipped to provide. Support in building digital literacy and IT skills would also be useful.  Communicate the work on national decentralization reform to promote grass-roots support and provide more advocacy training for CALM and LPAs to improve relations with the central government and better advocate and lobby for legislative reform.  Start sustainability and institutionalization focus earlier, not in the last year of the project and involve local institutions in the development of the approach/tools to ensure ownership and buy-in for the approach.  Work more with LPAs to improve administrative services both with and without the CISC model.  Develop a training of trainers on good local government approaches/tools with consultants and develop a culture for paying for this service by LPAs.  Provide a balanced approach to supporting the local authority and citizens, by empowering civil society and combining the assistance to LPAs with accountability-enhancement measures.  Explore supporting the implementation of CDSs in smaller LPAs, their monitoring and keeping them up-to-date.  Emphasize the exchange of information and expertise among beneficiary LPAs, so they can learn from each other regarding the common challenges they face and practical insights on addressing them.  CALM should remain an important partner in CM and other potential local government focused USAID projects. EQ3  USAID could either continue a phased cohort approach or explore a hub and spoke model (where the “hubs” get more intensive support and are then supported in their assistance to “spoke” LPAs) to build the capacity and work with a larger number of LPAs. Both of these approaches would benefit from the following points.  Provide a subset of LPAs with more intensive assistance – especially those with more absorption capacity – while using them to reach out to and support/mentor another group of nearby LPAs.  Facilitate the communication and exchange of practices and lessons learned among LPAs (knowledge dissemination, peer-to-peer workshops, field visits, etc.), either using CALM or other local organizations and institutions.  Establish a strategy for graduating LPAs/strategies from getting assistance. EQ4  Continue to support the development and implementation of strategic plans.  If further support is given on CDPs, USAID should look into ways to simplify the approval process and provide some support in securing external funds. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 6  For future CISCs, USAID should work to collaborate with other donors, who have already indicated an interest, and suggest that they provide the major funding while USAID, reflecting its comparative advantage, would provide technical support and training, including sharing the operational manual CM has already developed.  A modified and more disaggregated form of FOCAS emphasizing use by individual LPAs may be more sustainable.  The youth sector is increasingly important, and engaging youth should be a priority for local government.  USAID should consider whether to take on decentralization reform given the lack of strong political will at the central level.  USAID should consider working to build diplomatic skills of LPAs and CALM to help them be more effective in advocacy.  Coordination with donors should be emphasized. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 7 EVALUATION PURPOSE AND QUESTIONS PURPOSE As part of the DRG Learning, Evaluation, and Research II Activity (DRG-LER II), NORC completed a performance evaluation to assess the progress of Comunitatea Mea (CM) in achieving its goals and provide recommendations to inform the design of future USAID local governance programs in Moldova. NORC worked with a local research partner IMAS, based in Chisinau, as well as two local experts who led data collection (quantitative survey, observational data, focus group discussions, and key informant interviews) for this evaluation. The evaluation data collection was carried out between November 2021 and April 2022 with draft and revised evaluation reports submitted in May and June 2022, respectively. The evaluation team included Dr. Ritu Nayyar-Stone (team lead), Katharine Mark (Local Governance Evaluator), Dr. Adrian Lupusor (Local Governance Finance Expert), Ion Beschieru (Decentralization and Local Service Delivery Expert), Anna Solovyeva and Samantha Downey (Quantitative Specialists), Jessica Wallach (Qualitative Specialist), and Samantha Austin (Evaluation Support). EVALUATION QUESTIONS NORC was tasked with answering the following evaluation questions: 1. How is Comunitatea Mea making progress toward the achievement of its goals and objectives? a. To what extent have the activities implemented under Comunitatea Mea achieved their expected results, against each of the four objectives? To what extent have they been able to collectively contribute to its Goal: “Local Government is more effective, transparent, and accountable to citizens”? 2. To what extent were the approaches employed by IREX/Comunitatea Mea effective in reaching the stated results? a. To what extent have CM approaches been implemented and shown benefits? − How effective is strategic planning as an entry point in the process of community development? b. To what extent have the approaches been institutionalized and are they sustainable? 3. How effective is CM’s phased approach? How feasible is the total number of LPAs over the course of the project? a. What is the optimal number of LPAs that USAID should build the capacity of over a project lifetime? 4. Looking forward and building on the successes and failures of Comunitatea Mea, which areas should USAID support further to build stronger local governments and better local democracy where USAID has a competitive advantage? a. What are other areas with a potential to generate development outcomes in the Local Governance area that USAID might expand its support to? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 8 PROJECT BACKGROUND CM is a $20.5 million, five-year (2018-2023) activity implemented by IREX along with its partners Tetra Tech ARD, National Democratic Institute (NDI) and Urban Institute. At the beginning of the evaluation the Activity was in its fourth year of implementation. CM aims to strengthen local government to become more effective, transparent, and accountable to citizens. The principal beneficiaries of the Activity are the first level LPAs that will engage civic groups and local businesses to work with local governments to develop their communities and improve public services. The assumption of this Activity is that by increasing the capacity of LPAs, including their ability to work inclusively with citizens and civil society, not only will services be improved, but checks and balances between the central and local governments will also be strengthened, improving the overall system of democratic governance in Moldova. The four primary components of CM are 1) improved quality and access to municipal services, 2) Citizens meaningfully engaged in good governance, 3) Decentralization policy advanced, and 4) Increased locally-owned revenues and improved financial management practice. Figure 1 provides a depiction of CM’s partner communities, as provided in the 2021 annual report. Figure 1: Comunitatea Mea partner communities by cohort (Comunitatea Mea Annual Report, 2021) CM is USAID/Moldova’s flagship local government program, providing comprehensive assistance to Level 1 communities, including towns, villages, and municipalities, primarily targeting local self-governing bodies. The program set out at its award to work with up to 100 communities, and currently works in CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 9 87, selected through three rounds of applications, and representing a range of sizes and geographical locations across Moldova.1 The program is also working with central government partners, such as the State Chancellery, Ministry of Finance, Ministry of Agriculture, Regional Development and Environment, Cadastral Agency, e-Governance Agency, the Congress of Local Authorities of Moldova (CALM), and other donor-funded programs to advance decentralization policy reform and increase LPA financial viability. CM used a variety of approaches/tools to achieve results under each of the four components. Table 1 below provides a listing and brief write up on the different approaches used by CM. While some approaches were used across all LPAs in cohort 1, 2, and 3, others were used only in targeted LPAs based on demand and capacity. Table 1: CM approaches by component COMPONENT APPROACHES Component 1: Improved quality and access to municipal services Technical assistance (workshops and individual assistance) to improve governance in local administrations and local service providers Technical support to LPAs through Municipal Strategic Planning Workshops (MSPW) and further implementation of strategic plans developed /updated Technical support to implement funded Community Development Projects (CDPs) / Intermunicipal cooperations (IMCs) Technical support to develop and use Service Improvement Action Plan (SIAP) Technical support to develop and operate Citizen Information and Service Centers (CISCs) Technical assistance to improve social services/ develop plans/Multifunctional Centers Component 2: Citizens meaningfully engaged in local governance Technical assistance to strengthen LPA public outreach, communication, and transparency Technical assistance to strengthen the capacity of civic groups / platforms to engage with local government Support and citizen-led monitoring of local government, public service, and budget Technical assistance to support innovative transparency and citizen engagement tools Technical support to civil society (civic groups) through Small Grants Scheme Technical support to local working groups 1 Of the 87 LPAs, 77 were selected through a competitive application-driven selection process, while 10 joined the program by virtue of being included in Inter-municipal Cooperation projects. The technical assistance offered to this group of LPAs was more limited than the assistance provided to the 77 LPAs who applied to the program and was targeted specifically at preparing them to establish and implement successful IMCs. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 10 COMPONENT APPROACHES Component 3: Decentralization policy reforms advanced Rapid assessment of existing and/or optional reform arrangements Policy and legislative technical support for advancing decentralization reform Strengthen local government associations’ role in representing interests of LPAs Support broad-based dialogue on decentralization reform Civil society monitoring and oversight of decentralization reform Technical support to/through library/ librarians/ Decentralization Corners (DC) Decentralization Corners Component 4: Increased locally￾owned revenues and improved financial management practice Technical assistance to increase locally owned revenues and improved financial management practice Technical support for transparent budgeting, financial management, including gender equality and social inclusion (GESI) Technical support to helping CM partner to develop public property management policies Technical support to helping CM partner to improve local financial management practices/ innovative tools Technical support on local economic development to LPAs Technical support to assist partners in local tourism development for the most feasible LPAs’ ideas METHODS AND LIMITATIONS The evaluation team (ET) employed a mixed-methods approach including a document review and qualitative and quantitative data collection to generate credible evidence to answer each evaluation question. The design included computer-assisted telephone interviews (CATI), key informant interviews (KIIs), focus group discussions (FGDs), and three types of data gathered from citizen information and service centers (CISCs) including an observation checklist, a user survey, and questions asked to CISC operators. IMAS, a Moldovan research organization, partnered with NORC to support focus group moderation and conduct telephone interviews. Local experts also worked closely with NORC to conduct KIIs and validate instruments used for data collection. Figure 2 provides a high-level overview of the research design. An extended methodology section can be found in Annex A – Extended Methodology. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 11 Figure 2: Research design DESK REVIEW The evaluation team reviewed all documents pertaining to CM provided by IREX including technical papers, quarterly and annual reports, annexes to reports, monitoring, evaluation, and learning (MEL) plans, and data collected by the activity such as results from Functional Organizational Assessments (FOCAS), Population Surveys, and revenue generation data. In total, 66 documents were reviewed to either provide context, or provide further evidence to support findings for each evaluation question. QUANTITATIVE DATA COLLECTION – TELEPHONE SURVEY Telephone interviews reached LPA staff in 66 LPAs with 361 total surveys completed. Five LPAs (Chișinău, Bălți, Telenesti, Cahul, and Pohrebeni) of the intended sample of 71 were not able to be interviewed due to a lack of response to multiple calls, refusals because of lack of knowledge of the CM project (due to employee turnover), and in one case refusal due to an unfavorable view of the CM project. Table 2 provides a breakdown of the respondents by category. Desk Review Review of key documents including CM quarterly and annual reports, performance monitoring plans, and data gathered by CM. Quantitative Approach Computer￾assisted telephone interviews (CATI) administered in 66 LPAs with 361 total responses. Key informant interviews (KIIs) KIIs with LPA staff from 16 LPAs, USAID staff, IREX staff, other donor organizations working in local governance, and GoM. 55 interviews completed. Focus group discussions (FGDs) 10 FGDs with CSOs or community groups and 30 with active and non-active citizens selected from 15 LPAs (six LPAs from Cohort 1, five LPAs from Cohort 2, and four LPAs from Cohort 3.) CISC Observational Data Observational data collected at 6 Citizen Information Service Centers (CISCs) using a user survey, observation checklist, and CISC operator questions. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 12 Table 2: Telephone interview respondents LPA OFFICIAL NUMBER Mayor 65 Deputy Mayor 21 Budget/Finance/Economy Specialist 53 Cadastral Engineer 17 Specialist on Communal Services 2 Secretary of Local Council 42 Other Service Specialists 161 Total 361 As seen above, service specialists were interviewed at the highest rate. These specialists included local councilors, administrators, and specific service specialists. QUALITATIVE DATA COLLECTION – KIIS AND FGDS KIIs. The ET conducted mostly in-person semi-structured KIIs with a wide breadth of stakeholders involved in the implementation of CM. This included intended participants such as LPA staff, as well as the implementation team from IREX and the USAID/Moldova management team. Additionally, the ET interviewed other stakeholders in the local governance sector of Moldova such as other implementing partners and Donors. The ET also interviewed select members of the Government of Moldova (GoM) to understand how CM interacted with the national government. In total, 55 KIIs were conducted. 30 men and 29 women2 were interviewed for a total of 59 respondents. Annex A provides a breakdown by number for each participant group. FGDs. Two trained and experienced IMAS moderators conducted a total of 40 FGDs with citizens and CSOs or community groups from 15 LPAs across all three cohorts. 294 total citizens and CSO members participated, including 183 women and 111 men. Two FGDs were conducted in each of the fifteen LPAs selected for qualitative data collection, one with citizens active in local government, and one with non-active citizens. In ten LPAs, one additional FGD was conducted with CSOs or informal community groups. Non active citizens were randomly selected from the locality 2 Two KIIs were with groups of 2 and 4 participants respectively, leading to over 55 respondents. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 13 CISC OBSERVATIONAL DATA The ET collected data in the form of user surveys, observation checklists, and five CISC operator questions from six CISCs in Ialoveni, Strășeni, Cimișlia, Drochia, Cărpineni, and Copceac. User Survey. The ET developed a survey questionnaire to be asked to citizen users of the CISC. This included questions focused on the experiences of citizens using CISCs to access administrative services. 19 total questionnaires were applied in five of six LPAs. Nine men and ten women completed the questionnaire ranging from the age of 20-76 years old. Checklist. CISC observers were asked to complete a checklist focused on the level of functionality of the CISC, measures to ensure the comfort of the user and accessibility of the CISC, as well as adherence to COVID-19 safety protocols such as social distancing, masks, and available sanitizer. Each CISC was observed in the late morning to late afternoon over several hours. Operator Questions. Five questions were asked to CISC operators who were sitting at the desks during the time of observation. LIMITATIONS The following limitations should be considered when interpreting results from this report: 1. Results Attribution. While findings sometimes state that support provided by CM contributed to and helped achieve results or project outcomes in the LPAs, we cannot pinpoint causality only on CM. A variety of other factors and/or assistance from other donor may have also contributed to results, and this evaluation does not use a randomized control trail methodology for attribution. 2. Recall Bias. As several questions raised during the interviews addressed issues that took place in the past, recall bias may have affected the responses provided due to discrepancies in the accuracy or completeness of recollections of past events. This may be especially true for Cohort 1 LPA Officials who have received less support from CM in 2019-2021. 3. Halo Bias. There is a known tendency among respondents to under-report socially undesirable answers and alter their responses to approximate what they perceive as the social norm, called halo bias. This manifests in responses from LPA Officials who may have a tendency to respond favorably to questions as beneficiaries of CM. Moreover, citizens may feel hesitant to share negative perceptions of their local government. The ET made efforts to gather information from a wide range of LPAs and types of respondents to adjust for this bias and triangulate responses among each group. 4. Sampling for FGDs with Active Citizens and CSOs. The ET sought to select 6-9 respondents for each FGD. Active citizens were sampled from lists provided of individuals involved in activities facilitated by CM such as Pune Umarul3, Citizen Oversight Committees, 3 The Pune Umarul methodology for engaging civic groups includes civic organizers identifying community members in CM localities who are interested in engaging in civic life and helping them to create an “initiative group.” The initiative group goes door-to-door canvassing residents to determine the issues that are of greatest concern, then CM guides initiative groups as they learn how to develop and carry out an advocacy campaign aimed at resolving the identified issue. CM assists initiative groups in setting achievable goals and building in opportunities for the broader community to participate in their advocacy efforts. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 14 Citizen Report Cards, or Service Improvement Action Plans (SIAP) working groups which were associated with engagement in local government. This does not sample for the entirety of the population of active citizens but focuses on active citizens that were engaged by CM/LPAs leading to possible selection bias. The ET also had difficulty accessing CSO groups. LPA officials were unable to provide contact information for these groups, so IMAS had to use their connections to source participants. The ET found that a significant number of CSO members and active citizens were different employees of the LPA such as managers of schools and kindergartens, councilors, etc. In one case, a CSO member claimed to be unemployed but was actually a counselor. The recruiters made efforts to identify the most unaffiliated people and invite them to FGDs, however, the affiliation of these two groups to the LPA could have produced biased responses and introduced a selection bias. 5. CISC Observations. CISC observers noted that once they were noticed, the behavior of staff members changed. Desks were staffed and citizens began to be served quickly. Observers also noted very few citizens visiting the CISC, and therefore were unable to reach a high sample size of users. The ET was unable to make generalizations based on all CISC users since the sample size was only 19. Additionally, data from citizens and operators were not collected at Ialoveni due to resistance from LPA Officials and a lack of operators available to be interviewed. 6. Survey Sampling. Our method of sampling for the telephone survey focused on reaching mayors first so that they could provide the contact information of their colleagues who might have been better informed about certain aspects of the CM activity. We were unable to successfully interview the mayors in six LPAs, which had implications on our ability to interview their colleagues. In one LPA, however, the mayor was able to provide contact information for fellow LPA officials despite their inability to be interviewed. Due to the sample exclusively covering LPA staff, the ET acknowledges a potential limitation of relying too heavily on their perspective. The ET noted that responses were overall very positive and may have been subject to a desire to make their own LPA look good. Accordingly, the ET adjusted for this by comparing survey responses to responses given in KIIs with other LPA staff, as well as responses from citizens and CSOs in FGDs. 7. Environmental Factors. Factors such as spikes in COVID-19 infections in our sample selection as well as the beginning of the 2022 Russian invasion of Ukraine led to difficulties in data collection. Multiple FGDs had to be delayed due to spikes in COVID-19 and multiple KIIs had to be shifted online. Moderators noted that conversations were initially derailed by discussion of the Russian invasion in late February and early March as this topic largely dominated public discourse. Additionally, due to the influx of refugees, local officials became busy and were more difficult to reach. The ET extended the data collection timeline and was able to conduct the planned FGDs and interviews, however, these environmental factors could have influenced the quality of data collected from respondents due to distractions or off topic responses. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 15 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS FINDINGS EQ1: HOW IS COMUNITATEA MEA MAKING PROGRESS TOWARD THE ACHIEVEMENT OF ITS GOALS AND OBJECTIVES?4 Figure 2: EQ1 Key findings ▪ CM helped beneficiary LPAs improve the quality of and access to several services at least moderately. ▪ Where CISCs were established, citizens appreciated the improvement in administrative services. ▪ CM was successful in increasing citizen engagement in some areas and had mixed results in others ▪ There were some successes regarding decentralization policy, but adoption of some drafts is pending or unlikely to happen soon. ▪ CM had a positive impact on fee and tax revenues of beneficiary LPAs, as well as asset management revenues, though to a lesser extent ▪ The capacity building component on financial management was highly appreciated by LPAs, but sustainability issues were identified In this section we present findings on the results achieved by CM across all 4 project components – improving quality and access to services, meaningfully engaging citizens in governance, strengthening decentralization reform, and increasing own revenues and financial management by the LPAs. We take a holistic approach to the anticipated results based on our understanding that CM built capacity, provided technical assistance and used several approaches/tools to achieve the above 4 component objectives at a very broad level, across all the related functions of the LPA that deal with service delivery, citizen engagement, decentralized functions and access to as well as use of revenues. A. CM helped improve the quality of and access to several local services at least moderately Quality of services. According to the telephone survey and backed by qualitative data, overall, LPA Officials believe CM helped beneficiary municipalities to improve the quality of services at least at a “medium” level for many areas.5 As shown in Figure 4 below, the majority of the LPAs indicated that municipal services with “high” improvement in quality due to CM assistance include street lighting (45 LPAs), schools/kindergartens (38 LPAs) and libraries (36 LPAs). The services with “low” improvements in quality are wastewater, centralized water supply, and markets. The remaining services show medium improvement. 4 Additional analysis for EQ1 can be found in Annex B. Here we summarize and present the most important findings. 5 Survey respondents were asked: “Have you seen any improvement in the quality of / access to the following services due to the work of CM?” CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 16 Figure 3: Number of LPAs with high, medium, and low improvement in service quality Source: Telephone survey of LPA officials Note: Based on the survey question “Have you seen any improvement in the quality of the following services due to the work of CM?” (1= “A lot of improvement in quality”, 2 = “Some improvement in quality”; 3 = “A little improvement in quality”; 4 = “No improvement in quality”). Numbers of LPAs for each service were computed from the original survey question in the following way: 1) Take the average value of officials’ responses per LPA; 2) recode the LPA average values on a 1-3 scale with 1 corresponding to high improvement, 2 to medium improvement, and 3 to low improvement; and 3) tabulate counts of LPAs. The improvements ascribed by survey responses to CM affected many more services than those receiving direct CM assistance (through, for instance, individual CDPs). This is consistent with CM’s results framework, in which a wide range of approaches under Results 1.2 through 1.5, strengthening inclusion, governance, administrative services, transparency, accountability, and planning, are intended to work together to achieve the Component 1 Objective: Improved Quality and Access to Municipal Services more broadly. 40 34 31 23 21 18 11 11 9 6 5 4 4 4 3 14 18 27 32 35 43 41 32 34 29 31 36 26 24 18 9 13 8 11 9 5 14 23 22 31 30 26 36 38 45 0 10 20 30 40 50 Markets Centralized water supply Wastewater Cemetery Transportation infrastructure Subsidies received from the LPA Permits (business, building) Solid waste collection Parks & sports facilities Street cleanliness Social services (health and education) Street & Sidewalk condition Libraries Schools / kindergartens Street lighting Number of LPAs High impr. (# LPAs) Med impr. (# LPAs) Low impr. (# LPAs) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 17 Survey findings are also supported by qualitative data. For instance, in an LPA with a CDP targeting rehabilitation of a local park, citizens in an FGD with active citizens, communicated considerable improvements in two other areas of municipal services - water supply and street lighting - in the last 2-3 years. Although this may have also been affected by other factors6, the overall finding based on all evidence is that other CM approaches may well have also contributed to improving the quality of (and access to) municipal services. Moderator: “I want to ask you if in the last three years you noticed any changes in these services. Have they improved, have they worsened?” One female and two male respondents replied: “They have improved.” “The roads were made, water was drawn, lighting. The people down the hill, who were not connected before, also got connected. […] the roads have been repaired, water has been drawn, the streets are illuminated.” [FGD Active Citizens] In a few cases, the enhanced communication between citizens and officials promoted by CM was noted to help improve services according to citizen needs. In an FGD with CSOs, one respondent noted: “It's because of the CM project that has been running for a year and a half and has fostered the possibility for the citizen to come with a problem and go to the town hall and solve it in record time.” Citizens actively engaged with their LPA also noted that Viber communication groups with citizens helped indicate that messages sent by citizens had been read by local officials, prompting a response regarding a timeline for resolution of questions. However, there were high levels of variation in citizen satisfaction with the quality of (or access to) municipal services between LPAs and within LPAs between citizens at the center of the locality versus the peripheries, with citizens at the peripheries receiving less quality or less access to services and infrastructure. One respondent in an FGD with non-active citizens said: “Towards the center, the roads are better than in the suburbs. […] There are some territories where there is not all lit. When we come from the center, here is illumination, but over there, it’s darkness, it seems that you are going in a great darkness, it seems like (going) from heaven to hell.” Many citizens specified garbage collection, street lighting and roads as services that they are satisfied with, but this varied by location. Despite some citizens saying that garbage collection has improved, others complained of it being poorly managed, or being required to pay for this service. This echoed findings from some KIIs that the willingness and readiness of the citizens to pay for the services is sometimes a real challenge. Some citizens complained about LPA officials not being responsive to their issues on municipal services. In one FGD with active citizens, the participants complained about the poor water quality and said that they cannot find the right officials to speak with about it. “I do not even call them, as I know it will be the same. I called and the lady there paid no attention. The people feel disappointed and they see no point in that, and they also have no time to leave work and go talk about these problems, to go there three times a day.” In several specific cases, citizens complained that even when municipal service capital investment projects were completed, their maintenance and upkeep have not been funded. “Ok, they put in new lights, they spent a lot of money from the budget, it's nice, it lights up, the road is visible. But, basically, over half a year - one out of two - they broke down. And nobody's taking action.” [FGD, Non-active citizens] 6 The question was more general, asking about improvements in last 3 years without reference to the CM project. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 18 Cohort comparison/variation: The evaluation team had a working hypothesis that Cohort 1 LPAs, which have had more time in the project and may have benefited from more intensive assistance, would show greater improvements in services. We therefore compared improvement in quality of services by the LPA cohort and found a general positive trend with high and statistically significant differences in service quality for three services: street lighting, street cleanliness, and the street and sidewalk condition (see Table 3 below). We do not find statistically significant evidence for differences across the cohorts in improved service quality for other services. Table 3: Statistically significant difference in “high” improvement in service quality across cohorts SERVICE COHORT 1 COHORT 2 COHORT 3 Street and sidewalk condition** 63% 52% 39% Street cleanliness* 64% 47% 48% Street lighting* 84% 70% 69% Source: Telephone survey of LPA officials Note: (***p<0.01, **p<0.05, *p<0.1) Information for street and sidewalk condition is based on 308 observations (Nc1=97, Nc2=94, Nc3=117). The difference across cohorts in the distribution of responses to the question is statistically significant at the 5% significance level based on the p-value of the Kruskal Wallis test. Information for street cleanliness is based on 311 observations (Nc1=98, Nc2=94, Nc3=119). The difference across cohorts in the distribution of responses to the question is statistically significant at the 1% significance level based on the p-value of the Kruskal Wallis test. Information for street lighting is based on is based on 311 observations (Nc1=98, Nc2=94, Nc3=119). The difference across cohorts in the distribution of responses to the question is statistically significant at the 1% significance level based on the p-value of the Kruskal Wallis test. Access to municipal services. The survey indicates that CM project had at least a moderate contribution to the improvement in citizen’s access to municipal services. As Figure 5 shows, the highest improvement in access is the assistance citizens get from the LPA administration. Local markets show the least improvements in access, as was the case with improvements in the quality of the service. Centralized water supply, however, show greater improvement in access than in quality. When comparing improvement in access to services by the LPA cohort, we did not find any statistically significant differences. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 19 Figure 4: Number of LPAs with high, medium, and low improvement in access to municipal services Source: Telephone interview of LPA officials Note: Based on the survey question “Have you seen any improvement in access to the following services due to the work of CM?” (1= “A lot of improvement in access”, 2 = “Some improvement in access”; 3 = “A little improvement in access”; 4 = “No improvement in access”). Numbers of LPAs for each service were computed from the original survey question in the following way: 1) Take the average value of officials’ responses per LPA; 2) recode the LPA average values on a 1-3 scale with 1 corresponding to high improvement, 2 to medium/moderate improvement, and 3 to low improvement; and 3) tabulate counts of LPAs. In FGDs, citizens and CSOs provided several examples of services in which access improved over the last three years. In some cases, the CM project is directly mentioned and in others, citizens provide feedback about changes in general over the last 1-3 years and not due to CM. “The whole city has had all the pipes changed and now we have water, we have water in the city, in the apartment from the third to the fifth floor, even the houses with nine floors, they didn't have water after ten pm, now it's 24/24, this has been a very successful project.” Mod: “Is there water everywhere or so-so?” “Everywhere, thanks Lord.” [FGD, Non-active citizens]. As with quality of services, there were high levels of variation in views of citizen access to municipal services between LPAs and within LPAs between citizens at the center of the locality versus the peripheries, with citizens at the peripheries receiving less access to services. In FGDs with active and non-active citizens as well as CSOs, respondents noted the discrepancy in access to sewage, repaired streets and water between the center and outskirts of the locality. Insufficient access to services in some LPAs for people with disabilities: The marginalized groups discussed by respondents included persons living with disabilities, ethnic minorities, youth and the elderly. Gender issues seemed less important to respondents and in a KII an LPA official noted that 34 15 13 8 8 5 1 20 33 34 27 24 32 13 9 18 19 30 34 29 52 0 10 20 30 40 50 Markets Solid waste collection Permits (business, building) Centralized water supply Libraries Social services (health and education) Assistance citizens get from the LPA administration Number of LPAs High impr. (# LPAs) Med impr. (# LPAs) Low impr. (# LPAs) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 20 there is more participation of women and young female adults in governance issues compared to men. In general, respondents said that there was not sufficient access to services for persons living with disabilities and the elderly. Some initiatives were underway, but they were underfunded and incomplete. In one FGD, in which the majority of the participants were visually impaired, respondents said that their needs were addressed in the last year and a half, and crosswalk signals now make a sound. However, in general, participants said that their town is not accessible, and they proposed initiatives, but nothing was done. They also mentioned that there are not enough social workers. The inclusion of youth in local governance showed mixed results. In some LPAs youth councils were involved in decision making and there were projects carried out by youth. In other LPAs however, respondents reported that youth were not represented because they choose to not get involved. Active citizens in an LPA noted that there is no adapted transport service for people with disabilities, although LPA officials of the very same LPA said that the CDP was installing bus stations and increasing the number of stops on the route which has improved access for vulnerable populations. In FGDs CSO and active citizen respondent explained that there is a lack of resources for LPAs to focus this issue. However, in 2 of the 6 KIIs with LPAs, officials noted significant impact of CM in involving persons with disabilities while resolving issues, including establishment of the CISCs. “It was directly related to CM. They taught us how much we need to be closer to this population, how much we need to be transparent with them and how much they need to be involved and integrated into our community, to work with them, to solve problems regarding this category of people as well.” B. Where CISCs were established, citizens appreciated the improvement in administrative services CM addressed the issue of quality and access to administrative services in two ways: (1) via the CISCs that were opened in 12 (mostly large) LPAs and (2) via training, capacity building, and support in reengineering the administrative services (such as elaboration of passports for administrative services, updating websites with the information about the requirements to get the administrative services, etc. on an on-demand basis to a limited number of LPAs per cohort). The CISC approach is also analyzed in EQ 2 of this report where we focus on different approaches used by CM; here we focus on results – dealing with improvements in administrative services. In the municipalities that have not implemented CISCs, the perception is that improvement of administrative services is more limited. For example, nonactive citizens in one large LPA without CISC said that they did not see any changes since CM started implementing the project. Citizens in many LPAs, when asked about access or quality changes for administrative services in their LPAs, referred to infrastructure or public utility services (gas, water, etc.) and not administrative services. On the other hand, some LPA Officials from non-CISC municipalities thought that these services (certificates, permits, etc.) were quite efficient and quick before the CM project started. In a FGD with active citizens from a LPA without a CISC, one citizen noted the following changes in administrative services: “Yes, yes. […] the willingness to have the problem solved faster […] They're more responsive now.” In locations that have CISCs, they are seen as making a substantial contribution in improving the quality and access to administrative services. According to the results of the users' survey in 6 selected CISCs, CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 21 on average, CISC visitors spent 4.9 minutes waiting to be served.7 As seen in Figure 6 below, the majority of visitors interviewed reported that their visit to the CISC took less time than their previous visit to the Primaria. Figure 5: Visit time at the CISCs Overall, the majority of surveyed visitors were very satisfied or satisfied (4 and 13 accordingly), with the knowledgeability of the CISC staff, with just one visitor being very dissatisfied and one neither satisfied nor dissatisfied. Additionally, 13 users said that they had achieved the goal of their visit, while 6 said it was still in progress. The challenges noted by LPA officials and citizens regarding the rendering of administrative services through CISCs mentioned in the EQ2 are also valid for administrative services delivered through the regular or non-CISC approach (local council secretaries or area specialists, such as land and cadastral engineers, social workers, etc.). The main challenges include low wages for civil servants leading to high staff turnover, difficulty to attract skilled staff, limited staff units, and lack of financial resources. C. CM was successful in increasing citizen engagement in some areas and had mixed results in others As revealed by Figure 7, and reported by LPA officials, a large number of CM LPAs indicate “high improvement” in transparency (providing access to information) and responsiveness to citizen needs. Approximately half indicate “medium improvement” in citizens' trust in their LPAs (38 LPAs) and citizen engagement with their LPA (35 LPAs). A “moderate improvement” in citizens' participation in the budgetary process is reported by 47 LPAs, the lowest marks, revealing a contrast between LPA responsiveness at one end, and active participation at the other. A potential explanation surfaced in FGDs, as several non-active citizens felt they don’t get sufficient information from and are not motivated to engage with the government. In the words of one participant: “we don't really get asked and nobody has the courage to go. They have their opinions and their decisions, we don't.” 7 See more details in the methodology and EQ2 sections on the locations selected for observation and surveys/questionnaires. 4 10 1 2 0 2 4 6 8 10 12 14 16 18 N/A or don't remember Less time About the same More time Compared to your last visit at the Primaria, did today’s visit take less or more time? (n=19) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 22 Figure 6: Number of LPAs with high, medium, and low increase in citizens’ engagement in local governance Source: Telephone survey of LPA officials Note: Based on the survey question “To what extent do you think the following have increased in your LPA as a result of CM assistance?” (1= “A lot”, 2 = “Some”; 3 = “A little”; 4 = “Not at all”). Numbers of LPAs for each form of engagement were computed from the original survey question in the following way: 1) Take the average value of officials’ responses per LPA; 2) recode the LPA average values on a 1-3 scale with 1 corresponding to high increase, 2 to medium/moderate increase, and 3 to low increase; and 3) tabulate counts of LPAs. When comparing various forms of citizen engagement in local governance by LPA cohorts, we find statistically significant differences for citizen engagement with LPA and citizens’ access to information about the LPA. See Table 4 below. We do not find statistical evidence for differences across the cohorts for other forms of engagement. Table 4: Statistically significant difference in “A lot” of increase in types of citizen engagement across cohorts SERVICE COHORT1 COHORT2 COHORT 3 Increase in citizens’ access to information about the LPA work*** 77% 68% 62% Increase in citizen engagement with the LPA*** 58% 43% 46% Source: Telephone survey of LPA officials Note: (***p<0.01, **p<0.05, *p<0.1) Access to Information: Based on the survey question “To what extent do you think the following have increased in your LPA as a result of CM assistance?” (1= “A lot”, 2 = “Some”; 3 = “A little”; 4 = “Not at all”). The graph is based on 344 observations (Nc1=107, Nc2=107, Nc3=130). The difference across cohorts in the distribution of responses to the question is statistically significant at the 10% significance level based on the p-value of the Kruskal Wallis test. Increase in Citizen Engagement: The graph is based on 344 observations (Nc1=107, Nc2=107, Nc3=130). The difference across cohorts in the distribution of responses to the question is statistically significant at the 10% significance level based on the p-value of the Kruskal Wallis test. 2 1 1 0 0 47 38 35 12 11 17 27 30 54 55 0 10 20 30 40 50 60 Citizens’ participation in the budgetary process Citizens’ trust in the LPA Engagement of citizens with your LPA Citizens’ access to information about your LPAs work The LPA’s responsiveness to citizen needs Number of LPAs High impr. (# LPAs) Med impr. (# LPAs) Low impr. (# LPAs) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 23 COVID-19 and citizen engagement. The restrictions imposed under the COVID-19 pandemic forced the LPAs to rethink and adapt their tools and approaches for engaging with citizens. There were winners and losers in this process. On the one hand, LPAs with good websites that were updated regularly were more transparent and saw more citizen engagement by publishing opportunities such as public hearings. Also, a large majority of LPAs, with CM’s guidance and support, created social media groups and pages to communicate with citizens. Thanks to these changes that were catalyzed by the pandemic, some citizens and CSOs reported that improved communication enhanced feedback and responsiveness on services by their LPA. Citizen engagement in drafting Community Development Strategies (CDS) was a major success of CM. The CDSs are key strategic planning documents that can enable LPAs to attract grants, financing and investments for development. CM provided support for the elaboration of these important documents and facilitated the engagement of civil society in the formulation of local challenges and priorities. During KIIs and FGDs, the majority of citizens stated that most recommendations made by citizens were taken into account, which ensures the inclusiveness and local ownership of CDSs. It is also important that the CDPs were identified according to the priorities stipulated in CDSs, ensuring the consistency among CM activities. A key respondent from an LPA noted that their CDP came directly from the action plans developed in their strategic planning process, which involved working groups composed of civil society organizations, LPA officials, and representative organizations for citizens. Citizen engagement was important for ensuring the accountability of LPAs in implementing the infrastructure projects. Community Oversight Groups (COGs) that monitored CDPs to support LPAs with project implementation and ensure transparency of the construction process provided an important mechanism for ensuring accountability, inclusiveness, and quality of these projects. Citizens were engaged with CDP implementation in two ways: (i) they were surveyed and consulted before the CDP was selected to gather their insights and opinions on what kinds of infrastructure projects are the highest priority, and (ii) CM organized monitoring groups with citizens to oversee implementation. Implicitly, it created a citizen-led mechanism for identifying and solving issues related to the quality of CDPs. As one LPA representative stated: "The citizens called and asked the LPA to come to the site when they were noticing some issues and asked various questions such as why the width of the street under construction was decreased. When issues were raised by citizens, they convened the monitoring committee (made up of citizens as well as LPA staff) to decide how to address them." [KII, LPA official] The key challenge is monitoring by COGs of CDPs in their post-execution phase, in order to assess the quality of works overtime. Since many CDPs are scheduled to be executed in the last year of CM implementation, it leaves little or no time at all to monitor the projects in their post-execution phase. In FGDs citizens also noted that the COGs stopped meeting once the infrastructure project was completed. Another missed opportunity to engage citizens was the establishment of COGs to monitor bank loans taken by LPAs for local development projects (and supported by CM) to ensure the accountability of LPAs in managing these funds. Citizen engagement showed mixed results in participatory budgeting. According to some LPAs and CSOs, CM contributed to improving the transparency of the local budgets and citizen participation in the formulation of local budgetary priorities. In the telephone survey of LPA officials, 94% of the respondents said that the communication strategy implemented with the help of CM resulted CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 24 in citizens understanding the local budget better. Main activities were holding public hearings, publishing clear and simply presented information about the budget, sharing the budgetary calendar and opportunities to engage for citizens, and allocating more money to citizen initiatives such as Pune Umarul. In a KII one LPA official said: “It was a very good and effective collaboration with the program team in Comunitatea Mea, the assistance was in terms of how to organize public hearings, how to form the budget, how to publish the budget, how to consult the citizens.” In another KII an LPA official noted that due to the CM project information regarding the draft budget was now published in social networks such as Facebook, and Viber communities and that “before joining this project, we also kind of didn’t do budget hearing with these presentations either.” However, other LPA officials, and CSOs said that it was often difficult to convince citizens to participate due to the existing culture, and that despite posting information on Facebook multiple times “about three citizens came to the hearings on their own initiative, the rest were from public institutions, also interested people, but not ordinary citizens.” One citizen said that there was insufficient outreach of invitations to public hearings. After the COVID-19 pandemic some LPA officials said they would continue holding public hearings both online and in-person instead of only in person: “In my opinion, the most successful and the best, and just in time was the hearing on the Facebook page of city hall. […] we’re going to continue with the public hearings on Facebook”. Often, citizens need an external push to get organized and engage in local decision-making processes. There were cultural norms that emerged from the qualitative data around citizen engagement; it seemed that citizens were often reluctant to actively engage in governance decisions by going to public hearings and making their voices heard. For example, in an FGD with non-active citizens participants stated that they will participate in community projects if the mayor recruits them, but they would not organize amongst themselves without an external push: “If [the mayor] says help is needed, people come to help, but otherwise […] Out of the blue to go and get to work, I cannot do that.” Instead, as noted in FGDs with CSOs and active citizens respectively, the approaches that were more effective involved surveys, or groups like Pune Umarul, going to citizens directly to solicit their opinions and assist them in bringing issues to the attention of their LPA. Having this external push, rather than asking citizens to get involved on their own accord, seemed to be most effective to encourage engagement. D. There were some successes regarding decentralization policy, but adoption of some drafts is pending or unlikely to happen soon. CM’s input and assistance to GoM on advancing decentralization policy reforms was to a large extent demand-driven and resulted in a considerable number of drafts and proposals but has translated less into adopted policy and legislative changes, especially regarding major issues such as territorial reform. GoM respondents expressed appreciation for CM’s assistance around decentralization legislation. In particular, CM’s organization of consultations with LPAs, the technical documents they produced, and their flexibility in responding to Ministry requests were very helpful and saved time for Ministry staff. Although it is not yet approved or publicly available, one GoM respondent said that CM’s assistance with the roadmap for decentralization, and their work on this with civil society, was the best activity on this front and CM’s analyses and recommendations will continue to be relevant after the end of the project. CMs work included for example, a draft methodology on tariff calculation for solid waste management (expected to be approved by the GoM according to IREX and Ministry of Environment officials, after the regulatory impact analysis report is approved), draft laws package for the territorial reform CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 25 implementation (unlikely to be adopted due to lack of political will), and regulations for special fee (adopted by GoM according to IREX). The lack of political will at the national level diminished the impact of the technical assistance provided by CM to the GoM in promoting decentralization reform and amalgamation. The country also experienced several political disruptions such as parliamentary elections, early elections, change of governments, changes in the structure of central specialized public administration (number of ministries), changes in senior and middle level civil servants in the ministries, etc. during the period of technical assistance provided by CM. Another way CM worked on advancing the decentralization reform policies was through development of policy papers (such as on IMC, an internal assessment of three LPA functions: urban planning, land management and urban green space management, an LPA costing study with findings, conclusions and policy recommendations related to LPA operational expenditures, staffing structures and provision of local services), analysis, recommendations and opinions on some draft laws and policies prepared by the government (such as on the medium-term fiscal and budgetary policy). There were a few examples of recommendations being taken over by the ministries (especially regarding the budgetary and local finance sphere), but for the other papers there is little evidence of follow up or reaction from the relevant ministries and authorities on the recommendations made. CM also supported CALM in their advocacy work, by providing financial support and expertise to prepare policy arguments for their discussions with relevant authorities. CM sought to include CSOs in the decentralization policies promotion work and selected the ADEPT Association to apply a tool tracking sentinel indicators of the decentralization (the first monitoring report was launched end-April 2022). Other initiatives aimed at supporting the research and capacity building of young professionals in the field of decentralization. In the telephone survey (see Figure 8 below), when asked to agree or disagree with a list of statements regarding the impact of CM activities related to decentralization policy, respondents from 61 LPAs agreed that CM activities that mostly had a positive impact were measures to enhance the fiscal base of LPAs, and 60 LPAs responded that the review and improvement in functional assignment of LPA had a positive impact. The measures related to the methodology for waste management tariffs, although not yet adopted by the Government are believed to have a positive impact by LPA officials in 42 LPAs. The work by CM on promoting territorial reform received mixed opinions among LPAs (31 said “mostly yes” and 33 that it had “mostly no” impact on the LPAs). CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 26 Figure 7: Mostly Positive or no Impact of Various CM Activities related to Decentralization Policy (# of LPAs) E. CM had a positive impact on fee and tax revenues of beneficiary LPAs, as well as asset management revenues, though to a lesser extent Except for one LPA, all other government officials responding to the telephone survey felt that working with CM helped the LPA increase its own source revenues. Figure 8: Factors contributing to an increase in own source revenues As seen in Figure 9 above, several factors contributed to an increase in own source revenues with the largest percent of LPAs indicating an increase in tax collection (79%), improvement in asset management (77%), adjusted tax base (73%), and new tax rates (55%). This increase in own source revenues is 33 23 6 5 31 42 60 61 0 5 10 15 20 25 30 35 40 45 50 55 60 65 Administrative Territorial reform Methodology for waste management tariffs Review / improvement of functionals assignments Measures to enhance the fiscal base of LPAs Number of LPAs Mostly yes (# LPAs) Mostly no (# LPAs) 21% 29% 48% 55% 73% 77% 79% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% New fees Increase in fee collection Introduced new taxes New tax rates Adjusted tax base Improved asset management Increase in tax collection CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 27 confirmed by budgetary data.8 For all cohorts, the total own revenues – minus asset management revenues – of LPAs improved significantly compared to the year before joining the project. Thus, beneficiary LPAs outperformed the country average in terms of growth of own source revenues: • Cohort 1: +47% compared to 34% for the country average • Cohort II: +57% compared to 31% for the country average • Cohort III: +34% compared to 25% for the country average As noted in CM Annual Report for 2021, one of the most remarkable successes of CM in extending the local tax base is the contribution to securing in Stefan Voda a €1.5M investment and 30+ jobs by providing LED support to the LPA to attract foreign investment in partnership. This resulted in Stefan Voda operationalizing its Free Economic subzone (FEsZ) and attracting its first major foreign investor – Swiss company Schoeni Food. CM’s support to LPAs on improving property valuation and management included help with the elaboration and implementation of Public Asset Management Plans (PAPs), including the implementation of innovative tools (e.g. GIS-local platform). This work was also coordinated with other important donors in the field mainly, the World Bank, Norwegian Mapping and Ceadastral Authority and the EU. Whereas most beneficiary LPAs increased their asset management revenues (includes rent, and revenues from the acquisition, holding, and disposition of property), it is difficult to assess the net impact of the CM project, both because asset management revenues vary dramatically among LPAs, and because on average the asset revenues improved for the rest of the LPAs in the country as well. Namely, the cohorts I and III underperformed the rest of LPAs (Cohort 1: 28,1% compared to 68% for the country average, and Cohort III: 18% compared to 50% for the country average).9 EQ2: TO WHAT EXTENT WERE THE APPROACHES EMPLOYED BY IREX/COMUNITATEA MEA EFFECTIVE IN REACHING THE STATED RESULTS? Figure 9: EQ2 key findings ▪ CM offered a rich menu and flexibility in choosing some approaches. ▪ Despite the advantages of flexibility, the multiplicity of approaches created some challenges. ▪ Starting with the strategic planning approach was helpful for many LPAs. ▪ Several CM approaches were highly effective in engaging citizens and getting their feedback. ▪ The communication strategy was inclusive but faced some challenges. ▪ Evidence shows space for more efforts to connect, inform, and coordinate the work done by CM regarding decentralization at the national and local level. ▪ CDPs were appreciated but there were several challenges in their implementation. ▪ Results were mixed regarding the capacity and use of CALM. 8 For this analysis, own source revenues were considered net of asset management revenues. Those are considered separately below. 9 Estimations were conducted based on the BOOST database of the Ministry of Finance. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 28 ▪ Results were mixed regarding the capacity and use of CALM. ▪ CM provided a flexible and tailored approach to specific needs and priorities of beneficiary LPAs for financial management. ▪ There is high interest from LPAs to continue using CM approaches, but they face challenges in ensuring their institutionalization and sustainability. CM deployed numerous approaches under each of its four project components. Given the multiplicity of approaches used under CM, and USAID’s interest in learning about the effectiveness, the ET has addressed this evaluation question by summarizing findings across all approaches and including some observations about specific approaches. Given the rich feedback on approaches, additional detail on approach-related findings is included in Annex C. A. CM offered a rich menu and flexibility in choosing some approaches CM offered a large variety of approaches and tools under its 4 components. Some were used across all participating LPAs -- for, example all were required to develop strategic plans and complete the FOCAS assessment -- while others depended on the preferences and capabilities of individual LPAs. A minimum package of tools and approaches were also a precondition for CDP implementation (such as strategic plans, capacity building, training, etc.). IREX staff considered the tailoring of their assistance to the needs of specific LPAs as an important approach of the project. Some LPAs also considered this flexibility in applying different approaches and tools as a strength. B. Despite the advantages of flexibility, the multiplicity of approaches created some challenges Even though the variety and specificity of approaches were appreciated by some LPAs, smaller LPAs with limited staff and capacity sometimes found the multi-faceted aspect of the project to be a challenge. In a KII a GoM official said: “As far as I know, the project was interacting with big municipalities, like Straseni which is a municipality with quite a big absorption capacity and then it goes very ok […] When you interact with smaller municipalities, then things go more complicated and that's a major problem in absorption capacity.” The very small number of staff in some of the smaller LPAs was also mentioned as a problem. Another GoM official also noted that “size adjustment is hard to do” and approaches applied to an LPA of 40,000 require adjustments based on the realities of an LPA of 3,000, and another GoM respondent said “what is not correct is when there is no capacity to adapt this complexity to the size of the locality.” It may be that CM could have played a more active role in helping limit the number of activities taken on by smaller LPAs. In fact, more than ¾ of LPA respondents across the board said that limited human resources would be a challenge in continuing CM approaches into the future. (See Table 5 in section J below.) Only a few respondents suggested that number of different approaches might be detrimental to quality of the program. In a KII a donor staff said “The weak point, I don’t know, the weak point might be the fact that they work too much in localities requiring different approaches and different intervention mechanisms. I do not know to what extent this is always achievable enough or to what extent this is done with the assurance of a certain level of quality, so to speak.” C. Starting with the strategic planning approach was helpful for many LPAs CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 29 respondents mentioned that the strategic planning directly or indirectly helped to identify the priority area for CDP funding or other CM support. Importantly, many stated that the developed strategic plans helped the LPAs to pursue external funding for projects beyond CM, as the inclusion of the priority in the strategic plan is a precondition (or an advantage) for getting the funding. An interviewed LPA official underlined the usefulness of the strategic plan for this purpose: “The strategic development plan, we exploit it practically on every project. Now, there is such a tendency to ask for priorities (to be included in the strategic plan) and it turns out that they are in it, and we cooperate with this plan practically almost always.” [KII, LPA Official] The usefulness of strategic planning is also supported by quantitative data. In the telephone survey LPA officials in 50 LPAs felt that strategic plan workshops and implementation of the plans had a “high” contribution to an improvement in services, 15 LPAs felt it had a “moderate contribution and only 1 LPA felt it had a “low” contribution. Some respondents brought up that LPAs had developed strategic plans before CM, either on their own or as part of other projects funded by external donors, and therefore in cases where LPAs are capable of undergoing the strategic planning process on their own, the project should focus resources on other activities. A non-USAID donor noted in a KII that strategic planning is only a good entry point if the municipality did not already have a local strategic plan; if they did, the better approach would have been to analyze it and provide assistance based on the existing plan and strategic planning phase could have been skipped to preserve resources. However, other respondents think that CM’s process has added value as it assessed whether strategic plans were in place and not expired and updated them to fill any gaps. In addition, compared to other previous processes, strategic plans developed with CM support were considered to be participatory and genuinely encouraged citizen engagement. One local official noted, regarding other previous strategic plans, “(Referring to previous planning experiences) they made the strategy, but only theoretically, they didn’t ask citizens’ opinions… it was a very superficial one… our strategy, that has been made with Comunitatea Mea project, it was created from zero and it really involved citizens. […] I consider it a very effective one because it addressed the real and current problems of the locality.” [KII, LPA Official] In the telephone survey of LPA officials, the following challenges were noted in implementing the CDS: lack of sufficient funds to implement the activities with the LPA (76%), insufficient LPA staff (60%), insufficient expertise (39%), lack of support from local citizens (36%), and lack of motivation among LPA staff (33%). In a KII one LPA official noted that strategic plans can “remain on paper” depending on the management of the LPA and local councilors and called for regulations to ensure that financial resources are allocated in line with each LPA’s strategy. Officials in 3 additional LPAs also noted in KIIs that there is insufficient funding to implement their strategic plans in full. One LPA official estimated that about 20% of their strategic plans are feasible within their local budget, and another 20% could be implemented with help from the District Council; but the remaining 60% would only be feasible if they found external funding sources. Another official mentioned that inflation has worsened this discrepancy between plans and funding sources. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 30 D. Several CM approaches were effective in engaging citizens and getting their feedback CM assisted LPAs with a wide variety of approaches to foster citizen engagement and get their feedback. As seen in Figure 11 below, the majority of LPA officials noted the use of citizen surveys, office hours, and the suggestion box to get feedback from citizens. In FGDs CSO members also mentioned surveys when asked about changes in levels of citizen engagement in their LPA. One stated: “I think that Comunitatea Mea has increased the communication access between LPAs and citizens. Even through the counter, through surveys people are cooperating and getting involved.” Figure 10: What is your LPA doing to collect feedback on its service quality from your citizens? (n=314) Source: telephone survey of LPA officials. CM approaches such as the MSPW, participatory and public budgeting, citizen oversight groups of the CDP, SIAPs and Pune Umarul all involved citizens in governance even when engagement was not the primary aim. In a KII one donor official noted that LPAs involved in CM had higher levels of citizen engagement compared to other LPAs. However, feedback in FGDs with non-active citizens and CSO indicated that citizens were often reticent to engage actively by, for example, actually going to public hearings. In an FGD one non-active citizen said “If she [the mayor] says help is needed, people come to help, but otherwise […] Out of the blue to go and get to work, I cannot do that.” More effective approaches were surveys and Pune Umarul which involved going to the citizens directly to solicit their opinions. E. The communication strategy was inclusive but faced some challenges Figure 12 below presents the different communication strategies used by the CM LPAs. The majority noted using multiple methods to communicate with their citizens, including the use of bulletin boards, Facebook page, public hearings, infographics about the local budget, improved webpage and Viber communities. Most in-person communication such as public hearings, visiting the Primaria to speak with 1% 2% 34% 41% 54% 73% 88% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Nothing Don't know Other, please specify Feedback at CISCs Suggestion box Office hours Citizen survey CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 31 officials or going door to door for a petition changed to online modalities after the onset of COVID-19 such as Facebook and Viber groups. Figure 12: Which of the following communication strategies have you initiated within your LPA? (N=323) Citizens also liked online platforms for communication because they are more inclusive; a citizen remarked, “In a meeting in the house of culture, everyone will not be able to express their opinion, but online is very good. Everyone can express their opinion.” [FGD, Non-active Citizens]. Also, CSOs mention that questionnaires or surveys administered online and shared through social media can have more participation than in-person initiatives: “We initiate these questionnaires […] They are published on our Facebook page, on our web page and people see them, they answer, ….they sit at their laptop and it's simple, you answer the questions. They are active.” [FGD, CSO]. On the other hand, online tools have their limitations and challenges. An important challenge for online communication modalities revealed via KIIs is that many LPAs struggled with digital equipment, infrastructure, and technological literacy for LPA staff which limit the effectiveness of online communication. Staff in several cities didn’t even know they had a website. Other challenges are related to the use of social media and in FGDs with citizens (active and non-active as well as CSOs) respondents felt that social media pages for LPAs need more moderation, for two reasons: 1) the tone of the conversations online has led to LPA officials ignoring the pages; and 2) there are examples of citizens spreading misinformation (“gossip”). There were several examples of negative citizen feedback leading to LPA officials giving up on this modality of communication or deleting posts. “The group became a big fear in terms of gossip, stuff. […] The Viber group, the Viber community, yes. We're not going to continue it, we're going to continue with the public hearings on Facebook. This group on Viber community as a tool for an effective communication with citizens. It has somehow lost its value.” [KII, LPA official] F. CDPs were appreciated but there were several challenges in their implementation CDPs are among the most desired components by LPAs, as revealed by many KIIs with LPAs. Citizens were engaged with CDP implementation by first being surveyed and consulted before the CDP was selected to gather their insights and opinions (via surveys and public hearings) on what kinds of 82% 83% 84% 88% 90% 90% Viber communities Improved functionality of the webpage Developed infographics about the local budget Have bulletin boards to share information with citizens Developed a Facebook page for the LPA Increased frequency of public hearings CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 32 infrastructure projects are the highest priority, and secondly participating in citizen oversight groups to oversee implementation. However, there were several challenges in implementing the CDPs. (i) The process of approvals was time-consuming and there was a strong perception among respondents that the feasibility studies and technical documents were not proportional to the small scale of the CDP, (ii) Rapid inflation caused a mismatch between initial budget submissions and the price of materials during implementation. While this was an exogenous constraint; it was aggravated by the slow approval process of CDPs. Nevertheless, the flexibility of CM and USAID, which allowed for budget increases above the initial $60,000 ceiling to ensure that projects could still be implemented successfully was highly appreciated (iii) smaller LPAs with a limited tax base faced difficulties in raising their own contribution to the CDPs. (iv) in several LPAs where FGDs were conducted with active citizen groups, the ET learned that COGs sometimes comprised of staff from the Primaria, who were requested to join the groups but not aware of their role and responsibilities, (v) monitoring of the CDPs ended when the infrastructure project was completed, and citizens were not engaged on oversight of operations and maintenance. One complaint brought up by an LPA official from cohort 1 was that each LPA – regardless of size – was given the same funding for the CDP grant: “I've tried to tell them that it's not fair. In my opinion it's not fair to take this approach, so 60,000 for all the communities, there are larger communities, there are smaller communities, the approach should be directly proportional to the size of the locality per capita.” A GOM official noted that this projects “create a positive atmosphere and give impetus to initiate certain reforms, …” it would be useful if there was a way to “ensure synergy with nationally funded projects.” This may be difficult given CM’s focus on local prioritization for these projects, but may be worth exploring to increase sustainability of the approach. G. Evidence shows space for more efforts to connect, inform, and coordinate the work done by CM regarding decentralization at the national and local level The KIIs across the board revealed that more could have been done by CM to create synergies between the national and local level on strengthening decentralization. As noted in CM’s annual report and by IREX staff, the project involved selected partner municipalities in various round tables and discussions related to decentralization reform (such as the on the draft waste management tariffs calculation methodology, on local taxes, and organized debates in several locations on the administrative territorial reform that were also attended by LPA officials of the respective municipalities, etc), However in KIIs with multiple officials in 6 LPAs (total of 19 KIIs) the majority of officials interviewed by the ET were not aware of or had very little information about the work done by the project at the national level. Similarly, most of the GoM officials, and many interviewed donor representatives had very little information about the work done at the local level in the partner municipalities. A few GoM officials specifically mentioned some drawbacks due to limited coordination of CM’s work at the local level with relevant national authorities -- specifically duplication of training for LPAs and limited institutionalization of some approaches. One GoM official stated that USAID could improve governance in Moldova by addressing communication gaps between different levels of government (national vs local public authorities) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 33 H. Mixed results regarding the capacity and use of CALM10 CALM is an important partner in the CM project design and implementation and a Memorandum of Understanding was signed between the two organizations. CM supported and impacted CALM’s organizational development and capacity in tasks such as providing services to their members, advocating for their interests, etc. CALM was used to implement CM activities and to disseminate the project results beyond CM partner LPAs. As seen in Figure 13 below 41 LPAs felt that the capacity of CALM to represent LPA interest to the central government had improved only “somewhat”, and 46 said that CALM had “somewhat” communicated the results of CM LPAs to other LPAs. Many LPAs said that CALM has limited financial and human resources (61) and that its management needs improvement (57). I. CM provided a flexible and tailored approach to specific needs and priorities of beneficiary LPAs for financial management The tailored approach of CM to the local needs of LPAs applies across all project activities, but it was especially appreciated under component 4, given the support provided by CM staff to LPAs on revenue enhancement development (RED), local economic development (LED) and local tax development (LTD) (e.g. tailored investment offers or LTD concepts focused on local peculiarities). CM also provided a multitude of training and workshops on local public finance management, National Internal Control Standards (NICS), Internal Managerial Control System (IMCS), Performance and Risk Management 10 In addition to assistance provided by CM, CALM received ongoing and concurrent support by numerous other donors, including Swiss Development Cooperation, Council of Europe, and the European Union. CALM Challenges CALM Capacity 30 43 57 61 0 20 40 60 It was not interested enough to get involved in the CM activities It was not engaged enough by the CM project Its management needs improvement It has limited financial and human resources Number of LPAs 3 46 41 20 22 0 10 20 30 40 50 CALM and related information tools, networks have been used to communicate results from LPA's CM experience to other LPAs outside CM project Improved capacity of CALM to represent LPA's interest in relation to the central government Number of LPAs A lot (# LPAs) Somewhat (# LPAs) A little/not at all (# LPAs) Figure 11: CALM challenges and capacity CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 34 (PRM) etc. Follow up technical assistance and mentorship was provided to those who requested it due to limited time and resource constraints. Most interviewed LPAs highlighted the usefulness of capacity building activities on local financial management. LPA officials especially appreciated the intensive training for LPA staff in asset management, own source revenues, and budgeting. Several KII respondents called for trainings to be institutionalized, and for refresher trainings in the future to ensure that information is retained and used. In a KII one LPA official noted “From the discussion it was not clear how the information from trainings is being applied in practice, The LPA seems to need some additional assistance with the implementation of the knowledge learned from the trainings (probably, that is the case for most LPAs).” At the same time, some interviewed LPA representatives affirmed that often the wrong individuals were delegated to attend some of these trainings (usually, the focal point person participated in all trainings). These cases were especially common in the LPAs that perceived these trainings as a formal requirement for a CDP. A challenge for the enforcement and sustainability of internal financial management control (IFMC) is that currently the Ministry of Finance collects reports on internal managerial controls only from raions and not from the LPAs. CM has recommended digitizing the IFMC scores and has had meetings with MoF to help LPAs implement IFMCs. J. High interest from LPAs in continuing to use CM approaches, but they face challenges in ensuring their institutionalization and sustainability While most LPAs say they plan to continue using CM approaches, the limited staff available in the LPAs and low wages seems to be a common challenge hindering the efficient use and sustainability of various approaches piloted within CM and offered to partner LPAs for further use. CMs decision to focus on sustainability and institutionalization only in the fourth year of implementation has further limited this result. Finally, another issue that was raised was that across the board is that LPAs did not have sufficient staff to take on all that CM had to offer. As seen in Table 5 below, a majority of the LPA officials cited limited human and financial resources as impediments to their ability to sustain the different approaches. Table 5: Main challenges in using CM approaches in the future CM APPROACH/TOOL LIMITED FINANCIAL RESOURCES LIMITED HUMAN RESOURCES Strategic Planning Workshops (n=289) 88% 78% Strengthened LPA public outreach (n=306) 81% 76% Citizen led monitoring of LPA (n=297) 79% 76% Revenue enhancement development (n=208) 77% 79% Improved public property management (n=197) 80% 79% Transparent program-based budgeting (n=209) 50% 78% CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 35 Table 6: Summary of approaches, limitations, and interest in continuing APPROACH LIMITATIONS INTEREST IN CONTINUATION Implementation of Strategic Plan Insufficient funding in the local budget to implement. Have the capacity and will continue updating and using strategic plans after CM. A strong incentive is that this is a condition for applying for donor and national funds’ investments/projects. MSPW Lack of funding, skilled staff. 47 LPAs plan to continue use; 19 have an interest in using them. CISCs Costs of equipment likely to be prohibitive. Low staff salaries which limit LPA ability to attract qualified candidates. Where CISCs have been established they are likely to continue; other LPAs would be interested but would need funding from other donors such as the World Bank, Slovak AID, or EU. FOCAS 43 LPAs said it took too much time; lack of staff capacity and high turnover. Likely to need external incentives to continue in some format (such as from GOM or donors), as well as technical support. Communication Strategies Limited digital skills in some LPAs, CDP implementation delays impact on the LPA-citizen relations and communication on social media. 65 officials said that it was “somewhat likely” or “very likely” that their LPA would continue updating and using. CDPs Co-financing, inflation due to delays. Strong interest; but need for donor funding. Internal Financial management Control Those delegated to attend trainings were not always the most appropriate staff; staff capacity; need for trainings to be institutionalized and refresher trainings. It appears that the larger LPAs are most likely to be interested in continuing, as the smaller ones have more limited capacity. Public property asset management Lack of staff capacity and insufficient delimitation of LPA land. 41 LPA are likely to continue. Transparent program￾based budgeting Short training, limited citizen engagement, low financial autonomy. 47 LPA are likely to continue. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 36 EQ3: HOW EFFECTIVE IS CM’S PHASED APPROACH? HOW FEASIBLE IS THE TOTAL NUMBER OF LPAs OVER THE COURSE OF THE PROJECT? Figure 12: EQ3 key findings ▪ Given the number of target LPA beneficiaries, a phased/cohort approach was the best solution ▪ There was mixed feedback regarding implementation differences between cohorts ▪ There is some sharing of lessons learned among LPAs, but it is organic and has not been sufficiently stressed by CM A. Given the number of target LPA beneficiaries, a phased/cohort approach was the best solution The ET asked key informants from USAID, IREX, GoM and IREX sub-contractors about the effectiveness of CMs phased approach and the feasibility of targeting work with 100 LPAs. All agreed that given the number of target LPAs for CM, a phased approach was the only option. In a KII, one subcontractor noted that working with 25/26 LPAs in each cohort was difficult to manage, and that more in-depth work could be done by focusing on a smaller number of LPAs, perhaps a maximum of 15; another option is to increase the total funding for the work. A respondent from GoM also asserted that the cohort approach was very necessary, “because if you start with a hundred Local Public Authorities at once, I don't think the project either... It had to hire a super team to handle it and then it would be very complicated.” An IREX subcontractor noted that it was unclear from the RFP if the focus was on helping the neediest LPAs or accelerating the progress of the strongest ones. “If we do work in 77/800, there will be change in all […] The people in Cohort 1 were thinking of administrative hassles and came up with the menu. These were not wrong things but there was no strategy behind it. Are you going to try to lift up 897 LPAs each? Or help those willing and able to lift themselves? What’s the minimum number of people to learn something?” [KII, IREX Subcontractor] Another key respondent from the GoM, when asked what else USAID could do to help LPAs, however, called for coverage of all LPAs with technical support and training. They explained that CM covered only some municipalities, similar to the Norway project (150 municipalities), so there is a need to cover the remaining. Inter-Municipal Cooperation. Under the current highly fragmented territorial organization of Moldova and reluctance of the key stakeholders (including CALM and GoM) to undertake territorial￾administrative reform, IMCs in recent years have been seen by many (donors, national policy makers, CALM and LPAs) as a promising avenue. As noted earlier, CM changed its strategy of working with a 4th cohort of LPAs and instead focused on supporting LPAs interested in forming IMCs for specific joint investments. This new activity comprised of two subcomponents: (1) analysis of the national policies and legislation and making recommendations for more conducive environment for the cooperation CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 37 among LPAs to jointly deliver public services11 and (2) employing the IMC approach to extend the number of beneficiary LPAs beyond the three LPA cohorts (as there wasn’t sufficient time in the project to go for the fourth cohort as initially planned). One of IREX’s subcontractors noted that Mayors’ attitudes towards protecting their sovereignty were a challenge for IMCs and that in Moldova, IMCs require a contractual relationship that is “limited to thinking of water or sharing of trucks.” This contractor felt that a study of the range of IMC service activities would “widen the discussion of amalgamation or not.” A respondent from USAID said that IMCs have not proceeded according to expectations; they hoped that IMCs would gain momentum early on, but there were challenges in initiating this in the country. Challenges with the management of IMCs led to one of them falling through. As noted by one LPA official from cohort 2: “Now, we've been involved in inter-municipal projects. We did a tremendous amount of work, there was an application process as part of Comunitatea Mea. We did a huge, huge job by our city hall with the neighboring city halls of [X LPA] and partially X district, where we persuaded the mayors that we needed this project. We ordered a feasibility study, project estimates documentation, we waited for how long, six months, but even after six months we received a letter "Sorry, your project is good, but the donor ran out of money." [KII, LPA Official, Cohort 2]12 Despite the challenges of managing IMCs there are a couple of IMCs in Moldova that are considered successful. LPA officials praised CM’s IMC with Ungheni, and an implementing partner respondent also noted that the Edinet IMC initiated under another project was a positive example for water transportation. Overall feedback from the qualitative data suggests that the IMCs are more complex and require more time compared to other CDPs implemented only in one LPA. A key informant from IREX mentioned that: “One of the challenges we use for criteria is having 7-10k inhabitants because they have resources, trained staff who can manage these projects. For challenges, at [the] beginning there was not a realization from partners that the process is lengthy and detailed. Many steps, requires inventory, making sure it considers capital. At beginning, it seemed like LPAs thought it would take less time and that there’d be quick results.“ CM also prepared a guide on initiating IMCs among its targeted LPAs, but the methodological support and the IMC projects themselves were initiated quite late in the project cycle leaving little time for implementation and efficacy of the approach. Sustainability of IMCs: Several other donors were mentioned by respondents as implementing IMCs in Moldova, such as UNDP and EU4Moldova. The Ungheni LPA in particular has a history of participating in IMCs before the CM project. In a KII an LPA official from Ungheni noted that since the LPA has a strong operator for public utilities, 8 surrounding rural LPAs came to an IMC agreement with Ungheni to 11 Regarding the policy paper on IMC prepared by the project, it is not clear from CMs annual reports and other sources how it was used and whether the Government or other national partners have implemented any of the recommendations. 12 However, USAID indicated that no LPA was told that there is insufficient funding for an IMC. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 38 extend their services to their territories, including waste management, water supply, tourism, public lightning, including the public transportation service. While CM is supporting the public transportation and construction of bus stations for the IMC through their CDP, EU4Moldova is supporting other parts of the agreement such as waste management and water supply. A key informant from IREX explained that when they decided to implement IMCs, they received a lot of discouragement from other international donors and experts who believed that this approach has not historically been successful in Moldova, and the Mayors with whom they consulted were also pessimistic because they could not work with mayors from other political parties. However, they pushed forward with this approach since it has a potential to deliver better services to citizens (as well as address supply chain fragmentation issues and create economies of scale). This respondent said that the IMCs initiated by CM have taken a lot of effort, and even though they deliver better services to citizens they are not efficient. Another informant from IREX explained that the size and complexity of IMCs as opposed to CDPs made this approach unrealistic for implementation. “It is a big elephant […] and a process that needs continuous reflection.” B. There was mixed feedback regarding implementation differences between cohorts Given the phased nature of technical assistance, cohort 1 LPAs received the most “supervision” from CM, even though the intensity of technical assistance to those decreased in each subsequent year. However, there were mixed views from respondents about whether different cohorts enjoyed significantly different benefits or opportunities from the CM project. The COVID-19 pandemic also muddied the waters, making it difficult to compare between cohorts. One LPA official from cohort 1 noted: “We were the very first to get that information. We received a lot of new information for our work, which we had never even imagined until then, and did not know that it could be done. And all these training seminars, especially in the first year, when we were only 20, there weren't many of us. Accordingly, I think we had more attention.” This respondent also acknowledged the role of the pandemic in hampering implementation for later cohorts. “There were more opportunities then, because there was no pandemic,… all these actions took place in physical presence. I participate in the current online meetings, trainings, seminars as well, but that's not the same, I will tell you. It's maybe 50%, maybe even less than 50% of utility. We had more opportunities, more utility in the first two years, when we worked directly with representatives, experts, and other specialists who either invited us there or came to us here.” The online format of trainings for later cohorts made it more difficult for the LPAs to absorb the information and could also have played a part in perceived lower levels of individual, targeted attention for them. A key respondent from USAID attested that support from the project was not distributed equally; those in the beginning benefited from more time spent; however, they felt the cohort approach was necessary or else the project would have been unmanageable. Another USAID respondent noted that there was some questioning from cohort 3 LPAs about why they had not been able to make the same progress as cohort 1 LPAs; but this is to be expected given that cohort 1 has had much longer time to receive support from the project. In key informant interviews IREX staff felt that lessons in implementing the project with cohort 1 were incorporated in work with subsequent cohorts. A respondent from GoM however, felt that there were not significant differences in implementation between the cohorts: “So, I don't think the later entrants are disadvantaged […] If the timetable is respected from the start, I didn't see a problem […] once there are delays, maybe we will end up with a certain disadvantage for those who came last but, as far as I know and the last discussions I had with them, they have somehow made up CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 39 for the delays and most probably even the cohort three will not suffer, I mean they are somehow on schedule already.” [KII, GoM] In terms of the number of cohorts, a USAID respondent predicted that with the current pace of implementation, it was unlikely that CDPs could be completed for a 4th Cohort within the project timeline, which is one reason the 4th cohort was replaced with IMCs. C. There is some sharing of lessons learned among LPAs, but it is organic and has not been sufficiently stressed by CM The CM project had quarterly meetings of LPA mayors but did not sufficiently stress additional sharing of lessons learned. In the telephone survey Deputy Mayors were asked a set of questions on interaction with other LPAs involved in CM. Out of 21 Deputy Mayors, 20 indicated that they interacted with other LPAs involved in CM for a variety of reasons (multiple responses were allowed). As seen in Table 7 below, “to share our experience” got the maximum response with 19 counts. In addition, 13 Deputy Mayors mentioned interaction to implement intermunicipal cooperation initiatives. Table 7: Reasons for interactions with other LPAs, response by Deputy Mayors REASONS FOR INTERACTION WITH OTHER LPAS # OF LPAS To receive training or other technical assistance together 18 To share your experiences 19 To provide help to each other 17 To implement Inter Municipal Cooperation initiatives together 13 Source: Telephone survey of LPA officials Note: N=21 Table 8: Types of interactions with other LPAs, response by Deputy Mayors INTERACTIONS WITH OTHER LPAS # OF LPAS In small meetings with 3 or fewer other LPAs 14 In large meetings with more than 3 other LPAs 12 Shared materials (guides or training material) 15 On-going personal contact 18 Source: Telephone survey of LPA officials Note: N=21 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 40 EQ4: LOOKING FORWARD AND BUILDING ON THE SUCCESSES AND FAILURES OF COMUNITATEA MEA, WHICH AREAS SHOULD USAID SUPPORT FURTHER TO BUILD STRONGER LOCAL GOVERNMENTS AND BETTER LOCAL DEMOCRACY WHERE USAID HAS A COMPETITIVE ADVANTAGE? Figure 13: EQ4 key findings ▪ Several CM approaches were effective, and could be useful to continue supporting ▪ Stakeholders suggested potential new approaches to expand effectiveness ▪ Improved donor coordination would be fruitful A. Several CM approaches were effective and could be useful to continue supporting As described earlier, many of CM’s approaches were greeted favorably and were seen as having helped improve both quality of and access to services, as well as deepening citizen engagement in local governance. To determine which approaches USAID should continue supporting, the ET looked at the most effective approaches that LPAs seemed eager to continue, and also took into account the assistance they would require, and to what extent that assistance might be most effectively delivered by USAID. Strategic planning. The development of strategic plans (SPs) was widely appreciated and 71 % of LPAs plan to continue implementing their Plans, although many felt they would need financial and (to a lesser degree) technical help. Respondents felt that the SP process was especially useful in contributing to citizen participation and engagement, but several noted that they would have needed help in obtaining funding to move forward with elements in the plan. Some of the larger LPAs already had their own SP, and several key informants suggested it might have been better to build on the SPs already in place . A donor suggested USAID may want to consider in future to support the initiation of strategic planning in smaller LPAs, where no strategic plan has been done before. Community Development Projects. CDPs were viewed very positively, but there were some caveats. A number of respondents commented on the time-consuming processes that were required; one LPA representative said they thought feasibility studies weren’t needed. One key informant pointed out that there were some inconsistencies with Moldovan procurement law. LPAs also found it challenging to find funds for their contributions and would have appreciated help in that area. Meeting legislative requirements, and delays in procurement meant that implementation started relatively late in CM’s period of performance, leaving insufficient time to ensure that strong O&M plans and citizen oversight was well established. For future deployment, USAID may want to bear in mind some adjustments. One LPA interviewee remarked: “The CDP is so big and lengthy in time that it started to lose institutional memory…it was delayed too much. The project was designed 4 years ago, and now many things changed in the field, the placement of different fences, a building was built, etc. there are some small issues, but I am sure they will be solved. The project was delayed too much, though. There are very high requirements in the legislation for the construction of the X” CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 41 FOCAS. Although LPA respondents noted that they found FOCAS quite time-consuming (2/3rd in the telephone survey), some informants clarified that it is not so much giving the ratings as in applying the scores and finding it difficult to pull together staff in one room. They found it more efficient to discuss each aspect rather than assigning scores. A less aggregated format for the tool addressing each aspect separately may be more useful for the individual LPAs. In its current format, it is useful for resource allocation and measuring activity performance. CISCs. Despite some challenges with the CISCs, they received positive reaction from LPAs in the telephone survey, in KIIs, and from citizens in FGDs. Several donors are interested in the CISCs; SlovakAid has already been a co-funder, and according to interviews both the World Bank and the EU are planning to provide funding in the future. USAID has developed strong expertise in the development of the Centers, and in technical support and training, as well as developing an operational manual. A key informant from IREX noted that refresher trainings were already needed for CISCs. Providing support in this way might be a very cost-effective way to maximize USAID’s impact. Communications. Most LPAs appreciated the communications assistance they received, and most implemented several new measures, some spurred by the challenges faced due to the COVID-19 pandemic. This assistance was appreciated for facilitating citizen engagement, transparency, and better public relations, and most respondents seem committed to using these measures. Further assistance in communications would be useful, including efforts to build digital literacy. An exception to this might be support to interactive platforms which began during COVID-19 as welcome replacements for in-person encounters but proved to be problematic, as some became sites for unpleasant exchanges / acrimonious, and in some instances, sources of misinformation. This was difficult to handle, especially in LPAs where staff had limited IT skills / digital literacy. Accordingly, it may be wiser to focus on supporting internet sites as a means of sharing information or soliciting private feedback, rather than provide an open forum that would require much more active moderation than the LPAs could provide. Citizen Participation. CM was successful in increasing citizen participation, and there was strong interest in continuing those efforts. One donor encouraged extending these successes to other LPAs, and also suggested involving vulnerable populations more actively. CM citizen engagement emphasized empowering citizens to engage in the local decision-making process rather than oversight of the activity of LPAs. For example, while there was encouragement of citizen participation in budgetary hearings and participatory budgeting, this left a gap in monitoring of budget execution or oversight of procurement. This might be a useful component to add given the risk of corruption. Other activities. In addition to those raised explicitly in the survey, several activities were named unprompted when asked what other approaches were most effective in meeting project goals: building civil society capacity, providing direct assistance with infrastructure and services, and the SIAP. These should also be considered for inclusion in future activities as well. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 42 B. Stakeholders suggested potential new approaches to expand effectiveness In the telephone survey, LPA officials were asked directly whether there were other sorts of assistance that would be helpful in meeting each of CM’s three objectives (see Table 9). Table 9: Most frequently suggested new approaches, by objective TO IMPROVE SERVICES N=127 INCREASE CITIZEN ENGAGEMEN N=78 INCREASE LOCAL REVENUES N=60 ▪ Direct support on communal services, 31% ▪ Legal assistance, 6% ▪ Communications, 5% ▪ IT / Software, 4% ▪ TA, education, training, 33% ▪ Financial assistance, 17% ▪ Communications, 27% ▪ Direct support on communal services, 15% ▪ Youth engagement, 6% ▪ Technological assistance, 4% ▪ TA, education, training, 26% ▪ Financial assistance, 14% ▪ Legal assistance, 12% ▪ Economic development, 11% ▪ Property related, 10% ▪ Support with taxation, 10% ▪ Improve financial management, 7% ▪ TA, education, training, 27% ▪ Financial assistance, 10% Source: Telephone survey of LPA officials There was substantial interest in receiving more training and technical assistance, and requests for direct financial support, but there were a number of specific ideas. Of the 127 who responded regarding service improvement, 31% mentioned the need for direct support on communal services (including waste management, infrastructure), while 6% mentioned legal assistance. The 78 who responded with suggestions on how to increase citizen engagement were more divided. The most frequent request was for assistance with communications, 27% both web and non-web modalities, but again there was substantial request for support on communal services. Economic development and youth engagement were also mentioned, and legal assistance was requested across two of the objectives. Several key informants, including citizens, CSOs, LPA staff, and implementers also mentioned youth as being a missing piece of the current project and suggested ways to include them in the future, including expanded civic education and creation of job opportunities. Local economic development activities were begun later in the project but seem to have been met with great interest. Another promising focus area would involve strengthening IT systems, interoperability and maintenance. Digitalization for services would also be very beneficial. Decentralization. Reactions to CM’s decentralization activities were mixed, with some stakeholders feeling that this was not a fruitful area, given the lack of political will at the national level. Nevertheless there were some suggestions for new approaches that might be effective. Two donors felt more could have been done on advocacy. One observed that neither LPAs nor CALM seemed very effective advocates and described them both as being overly “aggressive” and that they would benefit from building some skills in diplomacy for more effective advocacy with the central government. LPAs and CALM as well as other stakeholders (CSOs, CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 43 experts, and academic community) could benefit from more direct support to help them advocate and lobby for the adoption of specific draft laws, for example. There was limited evidence of engaging LPAs in the work at the central level, which was a missed opportunity. Donor coordination could also be useful in engagement with the GoM and the Parliament. C. Improved donor coordination would be fruitful Deputy Mayors were asked a set of questions on other programs and international donors providing assistance to their LPAs during the last five years. Of 22 Deputy Mayors, 16 indicated that their LPAs received assistance from other programs and/or international donors. Of those, 13 believed that work with CM helped them leverage and obtain additional funding from another donor, while two disagreed with that statement. Several different donors and programs were named. Four respondents mentioned the European Union and three mentioned the UN (including UNDP). A number of bilaterals were also mentioned, including the Polish Government (mentioned by 3), Slovakia (2), and Romania, Switzerland, and Austria, mentioned by one each. Five mentioned Migration and Local Development, a joint program supported by Switzerland and the UNDP. Donors felt that more coordination would be helpful. One commented that communication was especially low and could greatly enhance the project. CONCLUSIONS EQ1: HOW IS COMUNITATEA MEA MAKING PROGRESS TOWARD THE ACHIEVEMENT OF ITS GOALS AND OBJECTIVES? • Overall CM helped LPAs improve the quality of and access to several services at least moderately through a variety of approaches. Some services showed a high improvement and differences in improvement of services across the 3 cohorts was also statistically significant for street and sidewalk conditions, street cleanliness and quality of street lighting. • CM contributed to the improvement in LPAs’ transparency, and some initial improvements in meaningful citizen participation, engagement and trust. The COVID-19 pandemic however, undermined the quality and intensity of civic engagement in the local governance despite CMs efforts to adapt. • The civic engagement component was important for ensuring the accountability of LPAs in implementing the infrastructure projects, but little emphasis was placed on post-execution monitoring. The CM design also did not include citizen engagement in monitoring LPA budget execution and public procurement in general. The qualitative data indicated that often, citizens need an external push in order to get organized and engage in local decision-making processes. • In LPAs without a CISC the improvement in quality and access to administrative services as perceived by LPA officials and citizens is much less obvious compared with LPAs where CISCs have been created. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 44 • CM’s input and assistance to GoM on advancing decentralization policy reforms was to a large extent demand-driven and resulted in a considerable number of drafts and proposals. However, since the project was more reactive rather than proactive the number of drafts and proposals did not translate fully into adopted policy and legislative changes, especially regarding major issues such as territorial reform. • The lack of political will at the national level diminished the impact of the technical assistance provided by CM to the GoM in promoting decentralization reform and amalgamation; there was also limited involvement or use of CM partner LPAs in promoting decentralization reform policies; more could have been done. • Most LPA officials believe that CM helped their LPA increase their own source revenues; there was appreciation for the diversity of approaches available to LPAs under this component. EQ2: TO WHAT EXTENT WERE THE APPROACHES EMPLOYED BY IREX/COMUNITATEA MEA EFFECTIVE IN REACHING THE STATED RESULTS? • The “menu” of approaches offered by CM was very rich and there was some flexibility for LPAs to choose from, but a minimum package of approaches was mandatory to get the funding for the infrastructure project (CDP) which was a big draw. • The CDSs are key strategic planning documents that can increase the capacity of LPAs to attract grants, financing and investments for development. But their opportunities can be tapped only if the LPAs continue keeping these documents up-to-date and stay active in their fundraising, advocacy and budgetary planning activities. • CM focused primarily on providing LPAs informational, technical and financial support, which visibly outpaced the support to the civil society. • Given the relatively small scale of CDPs, LPAs were frustrated with feasibility studies and engineering requirements, which caused delays in implementation and, given rising inflation, large discrepancies between the initial budgets and de facto spending. The delays in CDP implementation to the last year of the project does not leave any time for post-execution monitoring. Nonetheless, the CDPs were greatly appreciate. • Often, the capacity-building activities were attended by LPA representatives whose specialization was not entirely relevant to the topic discussed. This undermined the impact of these activities. Capacity building activities are also the most effective when combined in a logical temporal sequence of training, technical assistance and mentorship. CM did not follow this approach consistently since only a small fraction of beneficiary LPAs were selected for the next phase of capacity building activities. • One of the missed opportunities of CM is not enough emphasis on ensuring proper peer-to-peer learning and knowledge dissemination based on the individual experience with the implementation of various CM approaches. • CM work at the local level in the beneficiary LPAs and the work at the central level to promote decentralization policy reform seemed to be to a large extent treated as separate and isolated tracks. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 45 Respondents at the local level were not aware or very little informed of the work done by the project at the central level and vice versa. CM’s input and assistance to GoM on advancing decentralization policy reforms was to a large extent demand-driven and resulted in a considerable number of drafts and proposals but has translated less in adopted policy and legislative changes, especially regarding major issues such as the territorial reform. • CALM was used for specific activities as a service provider and as a partner in others, but the involvement and use of CALM is seen as insufficient by many respondents, especially by LPA officials. Internal challenges related to CALM’s capacities and specific positions and communication style of the organization’s management and secretariat might have been a limiting factor precluding more (efficient) use and cooperation. • Given the current limited LPAs’ capacities, the institutionalization or external ownership of various approaches to incentivize/remind LPAs is key for ensuring sustainability and use beyond the CM project lifespan. Focusing on this in the final year of the project when various tools were already finalized undermined possible cooperation and synergies. EQ3: HOW EFFECTIVE IS CM’S PHASED APPROACH? HOW FEASIBLE IS THE TOTAL NUMBER OF LPAS OVER THE COURSE OF THE PROJECT? • Given the large number of target LPAs a phased/cohort approach was essential. Some respondents felt that even 25/26 LPAs in each cohort was a lot to work with each year. With GoM’s request to work with even more LPAs it is clear that a larger number of beneficiaries would require an appropriate increase in activity budget. • There is mixed perception regarding implementation differences among cohorts. Some stakeholders felt that the earlier cohorts benefited from more time and supervision from the CM project. Others felt that lessons learned from work with cohort 1 and 2 made work with cohort 3 more efficient. • There is some organic sharing of lessons learned among cohorts and partner LPAs, but more could be done. The quarterly meetings of mayors from all beneficiary LPAs were useful, but the project did not seek to create more opportunities for meetings and synergies, especially among LPA technical staff. EQ4: LOOKING FORWARD AND BUILDING ON THE SUCCESSES AND FAILURES OF COMUNITATEA MEA, WHICH AREAS SHOULD USAID SUPPORT FURTHER TO BUILD STRONGER LOCAL GOVERNMENTS AND BETTER LOCAL DEMOCRACY WHERE USAID HAS A COMPETITIVE ADVANTAGE? • Several CM approaches are considered well worth continuation, especially if adjustments are made to improve their effectiveness and impact. These include: − Support for the development and implementation of strategic plans – especially for smaller LPAs – would be valuable. o CDPs were also found to be very useful, but it would be good if some of the more burdensome requirements could be streamlined, and it would be desirable if they could be CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 46 completed earlier in the project to allow time for establishing sound maintenance and monitoring processes. o There is continuing interest in direct assistance with infrastructure and services including SIAPs. o CISCs were very much appreciated, and where they are there is expectation that they are sustainable. Other donors are interested in contributing as well, and USAID has developed useful training and materials to support future CISCs. o Measures to increase citizen participation and support civil society are seen as successful and desirable and should be expanded. • New areas to consider for future programming included youth engagement, local economic development, and strengthening IT and digitalization. • If USAID seeks to continue to support decentralization reform, recommendations include building LPA and CALM advocacy skills to improve their effectiveness with the GoM; better leverage the experience and relationships from direct work with LPAs; and improve coordination with other donors. RECOMMENDATIONS EQ1: HOW IS COMUNITATEA MEA MAKING PROGRESS TOWARD THE ACHIEVEMENT OF ITS GOALS AND OBJECTIVES? • Focus on fewer approaches to achieve critical results within each LPA. Going forward the ET suggests that USAID select a few key results/outcomes and then have fewer approaches that target that result. • Expand the current work with citizens and CSOs with more targeted and successful tools, to build their capacity and incentives to engage in local governance. Also provide greater support to LPAs and citizens to implement alternative/digital instruments of community engagement (given the ongoing pandemic) and involve citizens/CSOs in post-execution monitoring of infrastructure (CDP) projects as well as monitoring the budget. If possible, explore funding a separate CSO project that works closely with CM to improve citizen engagement. • Engage the LPAs more in the decentralization policy discussion with the national government to create a critical mass of grass-roots support for the proposed reform measures. This is specifically important given the lack of political will regarding decentralization. • Create more synergies with other donor projects to improve results in select LPAs. • Institutionalize financial management approaches and provide refresher training to sustain results in increasing own source revenues. Also ensure that training is complemented with practical technical assistance to implement approaches/tools/concepts. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 47 EQ2: TO WHAT EXTENT WERE THE APPROACHES EMPLOYED BY IREX/COMUNITATEA MEA EFFECTIVE IN REACHING THE STATED RESULTS? • Refine processes for attendance at training; and composition of COGs to ensure correct representation of LPA staff and citizens respectively. • Have fewer approaches so LPAs are not over-taxed and to ensure better results and sustainability. • Start the CDPs at an earlier stage of the project (to avoid delays in implementation, changes in prices and allow sufficient time for post-execution monitoring). Future USAID projects should adjust the procurement requirements for CDPs to the scale of projects: small-scale projects should be subject to lighter conditions, to ensure a faster start and execution. More emphasis should be put on monitoring by the project team and citizens/community, during the execution and post-execution phases. It may also be fruitful to explore ways of linking local projects to national government plans. • For future communications work, efforts should emphasize outreach and provision of information rather than hosting chats or discussions, which require more active moderation which overtaxed staff are not equipped to provide. Support in building digital literacy and IT skills would also be useful. • Communicate the work on national decentralization reform to promote grass-roots support and provide more advocacy training for CALM and LPAs to improve relations with the central government and better advocate and lobby for legislative reform. Better information and coordination with central level policymakers in planning and implementing approaches and activities at the local level could prevent duplications and ensure possible synergies and better alignment with national policies (such as regionalization of public services, balanced territorial development, etc.). • Start sustainability and institutionalization focus earlier, not in the last year of the project and involve local institutions in the development of the approach/tools to ensure ownership and buy-in for the approach. • Work more with LPAs to improve administrative services both with and without the CISC model. Also, examine if the CISC model is appropriate for small LPAs in cooperation with the State Chancellery, Agency for Public Services, E-Gov, CALM, LPAs and relevant donors and the relevance of the CUPS (Unified Centers for Public Services Delivery) initiative. • Develop a training of trainers on good local government approaches/tools with consultants and develop a culture for paying for this service by LPAs. • Provide a balanced approach to supporting the local authority and citizens, by empowering civil society and combining the assistance to LPAs with accountability-enhancement measures. Thus, support to LPAs on the budgetary process, tax management and fiscal space, CIP etc. can be combined with citizen-led monitoring and evaluation of the budgetary execution, engaging citizens in monitoring the public procurements and empowering citizens to evaluate the previous and ongoing execution of capital expenditures, etc. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 48 • Explore supporting the implementation of CDSs in smaller LPAs, their monitoring and keeping them up-to-date. It will ensure that these documents remain relevant, useful and local stakeholders retain ownership over them. • Emphasize the exchange of information and expertise among beneficiary LPAs, so they can learn from each other regarding the common challenges they face and practical insights on addressing them. • Despite its challenges, CALM should remain an important partner in CM and other potential local government focused USAID projects. The associations’ function as a platform for the transfer, preservation and further dissemination of CM legacy is a strong asset that should be used, especially keeping in mind the new electronic platform developed for CALM with Swiss funded Ma-Implic project support. Where possible and appropriate, diversify the sustainability market by strengthening and using other institutions such as the Public Administration Academy, certain national think tanks (NGOs) specifically strong in the field and specialized networks (for instance association of water operators, etc.). EQ3: HOW EFFECTIVE IS CM’S PHASED APPROACH? HOW FEASIBLE IS THE TOTAL NUMBER OF LPAS OVER THE COURSE OF THE PROJECT? • USAID could either continue a phased cohort approach or explore a hub and spoke model (where the “hubs” get more intensive support and are then supported in their assistance to “spoke” LPAs) to build the capacity and work with a larger number of LPAs. Both of these approaches would benefit from the following points. • Start/continue to work with larger LPAs who have the absorption capacity and have them mentor smaller LPAs. Shift focus to smaller LPAs once larger LPAs have graduated from getting assistance (see below). • Establish a strategy for graduating the LPAs/approaches from getting assistance. This implies shifting technical assistance focus to different approaches once initial approaches get institutionalized within the LPA. We also suggest focusing on smaller LPAs and encouraging stronger/larger LPAs who have “graduated” to take on the role of mentoring. • Require more sharing of information and lessons and the use of CALM to disseminate information. Emphasize the exchange of information (challenges and practical solutions) and the expertise among beneficiary LPAs. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 49 EQ4: LOOKING FORWARD AND BUILDING ON THE SUCCESSES AND FAILURES OF COMUNITATEA MEA, WHICH AREAS SHOULD USAID SUPPORT FURTHER TO BUILD STRONGER LOCAL GOVERNMENTS AND BETTER LOCAL DEMOCRACY WHERE USAID HAS A COMPETITIVE ADVANTAGE Several CM activities should be supported in the future, but approaches can be adjusted to improve effectiveness. • Continue to support the development and implementation of strategic plans by focusing development efforts on supporting smaller LPAs, while building on existing SPs where they exist. Once SPs are in place, provide ongoing support, including in the identification of external funding. • If further support is given on CDPs, USAID should look into ways to simplify the approval process and provide some support in securing external funds. Moreover, CDPs should be done early enough in the process so that strong operation and maintenance plans are in place and citizen oversight is well established. This may require setting a strict deadline so that no projects are scheduled to begin implementation in the last two years of the project. • For future CISCs, USAID should work to collaborate with other donors, who have already indicated an interest, and suggest that they provide the major funding while USAID, reflecting its comparative advantage, would provide technical support and training, including sharing the operational manual CM has already developed. • A modified and more disaggregated form of FOCAS emphasizing use by individual LPAs may be more sustainable. Several new approaches or opportunities have been identified for consideration • The youth sector is increasingly important, and engaging youth should be a priority for local government. This could be done through strengthening targeted outreach, through civic education, recruiting youth volunteers or interns, and through such initiatives as expanding job opportunities. Similarly, there appears to be strong interest and need for work in local economic development, and this would be a productive area of focus. • USAID should consider whether to take on decentralization reform given the lack of strong political will at the central level. If this course is chosen, the new program should more explicitly link the two sides of the project and take advantage of its strong relationship with LPAs, for example through using experience and evidence from LPAs to provide information to policy makers and through engaging LPA representatives in the process. • USAID should consider working to build diplomatic skills of LPAs and CALM to help them be more effective in advocacy. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 50 Coordination with donors should be emphasized • Given the number of donors and donor programs in Moldova, many of them in related areas, USAID should stress the need for donor coordination. This can help improve synergies, reduce overlaps, and help LPAs access resources. • Clear and consistent communication with donors and other stakeholders is important. Communication can also serve to ensure that program outcomes are shared. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 51 ANNEXES CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 52 ANNEX A – EXTENDED METHODOLOGY DOCUMENT REVIEW The evaluation team (ET) reviewed all documents pertaining to CM provided by IREX including technical papers, quarterly and annual reports, annexes to reports, monitoring, evaluation and learning (MEL) plans, and data collected by the activity such as results from the Functional Organizational Assessments (FOCAS), Population surveys, and revenue generation data. A list of documents reviewed can be found below in Table A.1 In total, 66 documents were reviewed to either provide context, or provide further evidence to support findings for each evaluation question. Table A. 1: Desk Review Documents DOCUMENT TYPE NUMBER OF DOCUMENTS Annual Report 4 Quarterly Report 4 FOCAS Data and Supporting Documents 6 Population Survey Data and Supporting Documents 12 Revenue Generation Data 1 Annual Implementation Plans 4 Annexes to 2021 Annual Report 34 2021 Revised MEL Plan 1 Total 66 QUANTITATIVE DATA COLLECTION – TELELPHONE SURVEY Computer-Assisted Telephone interviews (CATI) were intended to reach LPA officials in 71 of the 77 LPAs that CM was actively working with upon the start of the evaluation (October, 2021). Six LPAs were not selected in the sample to avoid repeated questioning as the ET had selected them to conduct KIIs with LPA officials. 66 LPAs were successfully reached with 361 total surveys completed, achieving the goal of 355 completed surveys. Five LPAs (Chișinău, Bălți, Telenesti, Cahul, and Pohrebeni) were not able to be interviewed due to a lack of response to multiple calls, refusals because of lack of knowledge of the CM project (due to employee turnover), and in one case refusal due to an unfavorable view of the CM project. Different numbers of officials were targeted in each LPA depending on the populations size. For smaller LPAs (<3000 population) three officials were targeted, but in larger LPAs, up to seven officials were targeted. On average, five officials from each LPA were reached. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 53 NORC programmed the survey in Qualtrics, trained IMAS supervisors, and then initiated a piloting phase in which the survey was tested by IMAS enumerators. The Qualtrics program allowed the team to predetermine which modules to present based on information included in the respondent database or on real-time screening questions, or to allow respondents to select modules from a list. The survey was made available in Romanian and Russian and tailored for each type of respondent IMAS enumerators contacted LPA staff over phone and applied the questionnaire after obtaining informed consent from respondents. QUALITATIVE DATA COLLECTION Qualitative data collection was based on KIIs and FGDs to provide concrete examples that illustrate in greater detail the influences and results of CM activities. KII and FGD guides were developed in collaboration with local experts and translated into Romanian and Russian. Given the continuation of the COVID-19 pandemic, precautions were taken to ensure respondent and interviewer safety. Several KIIs and one FGD were conducted online to adjust for higher rates of infection in certain LPAs. In these cases, the ET used Zoom teleconference software or phones to conduct interviews. 16 LPAs were selected to conduct KIIs and FGDs in using the following criteria: • A selection of rural and urban • Variable population size • Varying activities implemented by CM • Varying overall capacity based on 2020 FOCAS score13 • Notable FOCAS score change from 2019-2020 (most improved amongst cohort members, least improved, etc.) • Representation from each cohort (6 from cohort 1, 5 from cohort 2, and 5 from cohort 3) • Representation of LPAs with ethnic minorities 13 FOCAS is a tool developed by Tetra Tech ARD to assess the capacities of organizations. The overall goal of using the FOCAS tool for the CM project was to assess partner LPAs’ baseline capacity for effective management and service provision, and to identify needs to improve municipal services. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 54 Table A.2 provides an overview of all LPAs selected for KIIs and FGDs and selected key characteristics. Table A. 2: LPAs selected for qualitative data collection LPA COHORT POPULATION URBAN/RURAL 2020 FOCAS SCORE RATING NUMBER OF ACTIVITIES INITIATED BY CM Cahul 1 30,018 Urban High 13 Strășeni 1 18,376 Urban High 18 Ungheni 1 30,804 Urban High 12 Ursoaia 1 2,465 Rural Medium 14 Ciorești 1 4,337 Rural High 14 Tomai 1 4,263 Rural Low 17 Cimișlia 2 11,997 Urban High 17 Jora de Mijloc 2 3,543 Rural Medium 14 Ceadîr￾Lunga 2 16,605 Urban Medium 11 Drochia 2 13,150 Urban High 13 Pănășești 2 3,000 Rural Medium 16 Cărpineni 3 8,358 Rural Medium 15 Pelivan 3 3,257 Rural Low 12 Soroca 3 22,196 Urban High 13 Copceac 3 9,138 Rural Medium 13 Chișinău 3 469,402 Urban N/A 5 KEY INFORMANT INTERVIEWS The ET conducted mostly in-person semi-structured KIIs with a wide breadth of stakeholders involved in the implementation of CM. This included intended participants such as LPA staff, as well as the implementation team from IREX and the USAID/Moldova management team. Additionally, the ET interviewed other stakeholders in the local governance sector of Moldova such as other implementing CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 55 partners and Donors. The ET also interviewed select members of the Government of Moldova (GoM) to understand how CM interacted with the national government. In total, 55 KIIs were conducted. Table A.3 details the number of respondents by organization. Table A. 3: KII Respondents ORGANIZATION(S) NO. TYPE OF KII RESPONDENT USAID/Moldova 3 Project Management Specialists, Office Director IREX Staff 6 Chief of Party, Deputy Chief of Party, Component 1-4 leads Donors active in local governance sector 6 World Bank, SDC, SlovakAid, MIDL, Council of Europe Other IPs in local governance sector 5 UNDP, GiZ, Promolex Solidarity Fund, Ma Implic Project Government of Moldova 10 Ministry of Finance, State Chancellery, Agency for Land Relations and Cadaster, Ministry of Environment, Ministry for Agriculture, Regional Development and Environment IREX Subcontractors 2 Urban Institute, Tetra Tech LPA Officials 27 Mayor, Deputy Mayor, Accountant, Cadastral Engineer, Service Specialists, Secretary of Local Council Total 59 FOCUS GROUP DISCUSSIONS The ET conducted a total of 40 FGDs with citizens and CSOs or community groups from LPAs across all three cohorts. Two FGDs were conducted in each of the fifteen LPAs selected for qualitative data collection, one with citizens active in local government, and one with citizens not active in local government. In ten LPAs, one additional FGD was conducted with CSOs or informal community groups. Table A.4 details the selection criteria for each of these groups. The ET made specific considerations for FGD recruitment to ensure FGDs provided a representation of the intended sample population. This included recruiting a mix of ages (18-64), representation of men and women, representation from a mix of occupations (except in government/political jobs, media or market research occupations), and representation of participants with disabilities. One focus group only included people with visual impairments. The recruitment strategies for each of the FGD groups were as follows: • CSOs and community groups: The IMAS recruitment team used the contacts provided by CM. These contacts were unable to provide contact information for individuals engaged in these groups, so IMAS used contacts from the LPAs to find members. • Active citizens: these groups included people from the community who are/were involved in any civic engagement activities (Pune Umarul, Citizens Oversight Committee, Citizen report cards or SIAP working group). IMAS used the focal point contacts and the contacts of citizen oversight groups CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 56 to access these participants. In addition, CSO members provided contacts of active citizens from their community. • Non-active citizens: These participants were recruited using the IMAS recruiting network by randomly selecting people from the locality. FGDs offered an opportunity to discover rich information on complex questions related to information that respondents may have found more or less useful, and the environmental, organizational and individual-level factors that contribute to such views. The focus group structure allowed for feedback among participants, therefore generating responses that may not have emerged from the KIIs or web survey. Although the results are not generalizable to the broader population, they provide invaluable points of departure for the evaluation, as well as source material for vignettes to illustrate more general conclusions. Table A. 4: FGD Selection GROUP NO. SELECTION Active Citizens 15 Citizens engaged in CM’s activities with LPAs such as Pune Umarul14, citizen monitoring groups, GDD, participatory budgeting, and others. Non-Active Citizens 15 General citizens not actively engaged in local government activities. CSOs or community groups 10 CSOs or informal community groups engaged in local governance advocacy. Total 40 CISC OBSERVATIONAL DATA CISCs are one stop shops for administrative services implemented in partnership with the LPAs, Slovak Aid, USAID, and the e-Gov Agency. Five CISCs located in the town halls of LPAs were launched in mid￾December 2020, with an additional seven launched in December 2021. These centers aim to improved administrative services, increase LPAs transparency, reduce corruption and waiting times, and save citizens money. The ET collected data from six CISCs including two LPAs from each cohort. Selected LPAs included Ialoveni, Strășeni, Cimișlia, Drochia, Cărpineni, and Copceac. CISCs in Cărpineni, and Copceac were more recently launched in December of 2021. Data collection included user surveys, the completion of an observation checklist, and five questions asked to CISC operators. User Survey. The ET developed a survey questionnaire to be asked to citizen users of the CISC. This included questions focused on the experiences of citizens using CISCs to access administrative services. 19 total questionnaires were applied in five of six LPAs. Nine men and ten women completed the questionnaire ranging from the age of 20-76 years old. In Ialoveni, the questionnaire was unable to be applied due to the limited number of visitors observed at the CISC and the difficulties encountered 14 The Pune Umarul methodology for engaging civic groups includes civic organizers identifying community members in CM localities who are interested in engaging in civic life and help them to create an “initiative group.” The initiative group goes door-to-door canvassing residents to determine the issues that are of greatest concern, then CM guides initiative groups as they learn how to develop and carry out an advocacy campaign aimed at resolving the identified issue. CM assists initiative groups in setting achievable goals and building in opportunities for the broader community to participate in their advocacy efforts. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 57 gaining cooperation of staff members. Observers in Ialoveni encountered repeated questioning and were unwelcomed. The observer noticed that citizens seemed afraid or uncomfortable when asked to complete the survey. To maintain respondent safety and observer safety, the survey was not applied. Table A.5 details the distribution of surveys completed by LPA. Table A. 5: User Survey Responses LPA NUMBER OF SURVEYS Strășeni 3 Copceac 5 Cimișlia 5 Drochia 4 Cărpineni 2 Total 19 Observations. Observers visited CISCs between the hours of late morning to late afternoon for a period of two to four hours. The earliest arrival time was 9:41 AM, and the latest was 11:00 AM. The observer stayed until 3:20 PM at the latest. These hours were deemed to be the hours when the CISC was most likely to be fully operational with an operator at the designated desks. Each CISC was visited several times on different days of the week to ensure the observations aligned with the on average "normal" situation at each CISC. The observer was asked to complete a checklist focused on the level of functionality of the CISC, measures to ensure the comfort of the user and accessibility of the CISC, as well as adherence to COVID-19 safety protocols such as social distancing, masks, and available sanitizer. Operator Questions. Five questions were asked to CISC operators who were sitting at the desks during the time of observation. These five questions were asked in five of six of the selected LPAs. Questions were not asked in Ialoveni due to difficulty gaining cooperation and no operators present at the desks. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 58 ANNEX B – DETAILED ANALYSIS FOR EQ 1 Figure B. 1: Mean Scores for the Level of Improvement of the Quality of Municipal Services Source: Telephone survey of LPA officials Note: Based on the survey question “Have you seen any improvement in the quality of the following services due to the work of CM?” (1= “A lot of improvement in quality”, 2 = “Some improvement in quality”; 3 = “A little improvement in quality”; 4 = “No improvement in quality”). We computed the average (mean) score for each service. We classified services with the average score less than 1.5 as having high improvement, services with the average scores between 1.5 and 2.5 as having moderate improvement, and services with the average scores over 2.5 as having low improvement. N=361. 2.6 2.6 2.6 2.3 2.2 2.2 2.0 1.9 1.9 1.7 1.7 1.7 1.6 1.6 1.4 Markets Centralized water supply Wastewater Transportation infrastructure Cemetery Subsidies received from the LPA Permits (business, building) Solid waste collection Parks & sports facilities Street & Sidewalk condition Social services (health and education) Street cleanliness Schools / kindergartens Libraries Street lighting CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 59 Figure B. 2: Improvement in quality of street lighting by the LPA cohort (% of respondents) Note: Based on the survey question “Have you seen any improvement in the quality of the following services due to the work of CM?” (1= “A lot of improvement in quality”, 2 = “Some improvement in quality”; 3 = “A little improvement in quality”; 4 = “No improvement in quality”). The graph is based on 301 observations (Nc1=97, Nc2=88, Nc3=116). The difference across cohorts in the distribution of responses to the question is statistically significant at the 5% significance level based on the p-value of the Kruskal Wallis test. Figure B. 3: Improvement in quality of street cleanliness by the LPA cohort (% of respondents) Note: Based on the survey question “Have you seen any improvement in the quality of the following services due to the work of CM?” (1= “A lot of improvement in quality”, 2 = “Some improvement in quality”; 3 = “A little improvement in quality”; 4 = “No improvement in quality”). The graph is based on 311 observations (Nc1=98, Nc2=94, Nc3=119). The difference across 84% 70% 69% 11% 16% 16% 4% 7% 10% 1% 7% 4% Cohort 1 Cohort 2 Cohort 3 A lot Some A little Not at all 64% 47% 48% 27% 32% 33% 9% 0% 17% 15% 4% 4% Cohort 1 Cohort 2 Cohort 3 A lot Some A little Not at all CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 60 cohorts in the distribution of responses to the question is statistically significant at the 1% significance level based on the p-value of the Kruskal Wallis test. Figure B. 4: Improvement in quality of street and sidewalk condition by the LPA cohort (% of respondents) Note: Based on the survey question “Have you seen any improvement in the quality of the following services due to the work of CM?” (1= “A lot of improvement in quality”, 2 = “Some improvement in quality”; 3 = “A little improvement in quality”; 4 = “No improvement in quality”). The graph is based on 308 observations (Nc1=97, Nc2=94, Nc3=117). The difference across cohorts in the distribution of responses to the question is statistically significant at the 5% significance level based on the p-value of the Kruskal Wallis test. Figure B. 5: Mean Scores for the Level of Improvement in the Access to Municipal Services Note: Based on the survey question “Have you seen any improvement in the access to each of the following services due to the work of CM?” (1= “A lot of improvement in access”, 2 = “Some improvement in access”; 3 = “A little improvement in access”; 4 = “No improvement in access”). We computed the average (mean) score for each service. We classified services with the average score less than 1.5 as having high improvement, services with the average scores between 1.5 and 2.5 as having moderate improvement, and services with the average scores over 2.5 as having low improvement. 63% 52% 39% 24% 9% 32% 10% 38% 18% 4% 6% 5% Cohort 1 Cohort 2 Cohort 3 A lot Some A little Not at all 2.6 2.0 2.0 1.8 1.7 1.6 1.4 Markets Permits (business, building) Solid waste collection Centralized water supply Social services (health and education) Libraries Assistance citizens get from the LPA administration CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 61 Figure B. 6: Mean Scores for the Level of Increase in Citizen Engagement Note: Based on the survey question “To what extent do you think the following have increased in your LPA as a result of CM assistance?” (1= “A lot”, 2 = “Some”; 3 = “A little”; 4 = “Not at all”). We computed the average (mean) score for each service. We classified engagement with the average score less than 1.5 as having a high increase and services with the average scores between 1.5 and 2.5 as having a moderate increase. Figure B. 7: Increase in citizen engagement with LPA by cohort (% of respondents) Note: Based on the survey question “To what extent do you think the following have increased in your LPA as a result of CM assistance?” (1= “A lot”, 2 = “Some”; 3 = “A little”; 4 = “Not at all”). The graph is based on 344 observations (Nc1=107, Nc2=107, Nc3=130). The difference across cohorts in the distribution of responses to the question is statistically significant at the 10% significance level based on the p-value of the Kruskal Wallis test. 1.8 1.6 1.6 1.3 1.3 Citizens’ participation in the budgetary process Engagement of citizens with your LPA Citizens’ trust in the LPA Citizens’ access to information about your LPAs work The LPA’s responsiveness to citizen needs? 58% 43% 46% 36% 48% 42% 6% 9% 12% 0% 20% 40% 60% 80% 100% Cohort 1 Cohort 2 Cohort 3 A lot Some A little CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 62 Figure B. 8: Increase in citizens’ access to information about the LPA work by the LPA cohort (% of respondents) Note: Based on the survey question “To what extent do you think the following have increased in your LPA as a result of CM assistance?” (1= “A lot”, 2 = “Some”; 3 = “A little”; 4 = “Not at all”). The graph is based on 344 observations (Nc1=107, Nc2=107, Nc3=130). The difference across cohorts in the distribution of responses to the question is statistically significant at the 10% significance level based on the p-value of the Kruskal Wallis test. Figure B.9. Visitor satisfaction with the knowledgeability of the CISC staff. 77% 68% 62% 22% 31% 34% 1% 1% 4% 0% 20% 40% 60% 80% 100% Cohort 1 Cohort 2 Cohort 3 A lot Some A little 4 13 1 0 1 0 2 4 6 8 10 12 14 16 18 Very Satisfied Satisfied Neither Satisfied nor Dissatisfied Dissatisfied Very Dissatisfied In general, how satisfied are you with the knowledgeability of the CISC staff? (n=19) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 63 ANNEX C – DETAILED ANALYSIS FOR EQ 2 A. Strategic Planning Strategic planning usefulness and effectiveness: To a large extent key informants consider that starting CM assistance with strategic planning was useful as it helped the LPAs to identify in a participatory way the real issues of the community. In addition, some respondents mentioned that the strategic planning directly or indirectly helped to identify the priority area for CDP funding or other CM support. Importantly, many stated that the developed strategic plans helped the LPAs to pursue external funding for projects beyond CM, as the inclusion of the priority in the strategic plan is a precondition (or an advantage) for getting the funding. An interviewed LPA official underlined the usefulness of the strategic plan for this purpose: “The strategic development plan, we exploit it practically on every project. Now, there is such a tendency to ask for priorities (to be included in the strategic plan) and it turns out that they are in it, and we cooperate with this plan practically almost always.” [KII, LPA Official] The usefulness of strategic planning is also supported by quantitative data: survey respondents believe that strategic plan workshops and implementation of the plans contributed a lot to an improvement in services in 50 LPAs, moderately in 15 LPAs, and a little or not at all in only one LPA. Figure C. 1: Contribution of Strategic Planning Workshops (MSPW) and implementation of strategic plans to improvement in services (# of LPAs) Some respondents brought up that LPAs had developed strategic plans before CM, either on their own or as part of other projects funded by external donors, that made the CM process duplicative and undermined its utility. A non-USAID donor noted in a KII that strategic planning is only a good entry point if the municipality did not already have a local strategic plan; if they did, the better approach would Low contr. (# LPAs), 1 Med contr. (# LPAs), 15 High contr. (# LPAs), 50 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 64 have been to analyze it and provide assistance based on the existing plan and strategic planning phase could have been skipped to preserve resources. However, other respondents think that CM’s process has added value as it screened the situation, and where strategic plans were in place and not expired, they were only updated as CM tried to fill the gaps and help them further. In addition, compared to other previous processes, strategic plans developed with CM support were considered to be participatory and genuinely encouraged citizen engagement. One local official noted, regarding other previous strategic plans, “(Referring to previous planning experiences) they made the strategy, but only theoretically, they didn’t ask citizens’ opinions… it was a very superficial one… our strategy, that has been made with Comunitatea Mea project, it was created from zero and it really involved citizens. […] I consider it a very effective one because it addressed the real and current problems of the locality.” [KII, LPA Official] Differences in size (population) between LPAs made a big difference in the utility of the support for developing strategic planning. Larger LPAs were more likely to have already gone through this process sufficiently and to have more capacities to develop the strategies on their own, whereas smaller LPAs, approached the process from a different starting point and thus benefited more from the exercise. Citizen participation in the strategic planning process: Overall it seemed that the Municipal Strategic Planning Workshops were a useful approach to gathering citizen input, and citizen feedback was acted upon. However, there was some variability across LPAs. In some FGDs, especially with active citizens, the workshop was known, while in other FGDs the participants said that it was the first time they were hearing about it. In one FGD with active citizens, participants have indicated that it was mostly “Pune Umarul” members that took part in this process. The extent to which citizen feedback was acted upon very much depended on the will of the Mayor and LPA officials as well as the size of their budget. An LPA official explained, “It is clear that the greatest burden falls on the shoulders of the mayors because if you (as a mayor) don't get personally involved, they leave it like ‘it will work as it is.’” [KII, LPA official] It is important to note as well that in later cohorts the workshops took place online instead of in￾person. By this point, social media channels were becoming more central to communication between LPAs and citizens, so citizens might have been more likely to hear about the workshops on Facebook or Viber. However, many respondents indicated that in general (not just for the strategic planning workshop but for all kinds of public hearings) online sessions were less attended or less interactive than in-person ones. A respondent in a focus group with CSOs explained this phenomenon specifically for the strategic planning workshop: “It was very hard from the point of view that it was online, physically it's more productive. But online, someone turns off the camera, and when you don't want to talk alone, you want to see people's reactions and you should at least see the eyes. It was very difficult to get answers online coming from someone other than the local government.” [FGD, CSOs] Perhaps because of the switch to online formats for later cohorts, there were clear trends towards respondents in FGDs from LPAs from earlier cohorts reporting that their feedback was taken into account more than was reported for later cohorts. Feasibility and sustainability of the strategic plans: The lack of sufficient funding in the local budgets for the implementation of the adopted strategic plans was a leitmotif across all the interviews. In the CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 65 telephone survey, respondents from 66 LPAs (out of 66) agreed with the statement that limited financial resources is a challenge for the implementation of MSPW and strategic plans. One LPA official key informant mentioned that only around 20% of their development strategic plan could be funded from the LPA budget; for the rest, they need to look for additional funding from donors, national funds and district subsidies. Another key informant from an LPA told the ET that the strategic plans can “remain on paper” depending on the management of the LPA and local councilors and called for regulations to ensure that financial resources are allocated in line with each LPA’s strategy. An LPA official also mentioned that inflation has worsened this discrepancy between plans and funding sources. This resonates with the opinion expressed by a non-USAID donor: “I have in mind that we still have problems with strategic planning, but strategic planning here in Moldova is mostly done at the theoretical level, not practical because it is not really linked to implementation and funding. […] we as a donor community often try to build that capacity at the theoretical level without providing money, without providing opportunities to practice, but providing a lot of training and a lot of theory.” [KII, Donor] A key informant from GoM stated that from his experience of reviewing the sustainability of previous projects, often the LPAs develop strategies, or, say, elaborate a website because that is a requirement by the donor to get the money for the infrastructure project, but after some time they forget that they have a strategy or a website (some LPAs even could not remember/find the username and password to manage the website). In his opinion, USAID should monitor over time (for instance, three years after the project ends) how these developed strategies (or other tools) are implemented. In his opinion, this would increase the awareness and accountability of the LPAs and would increase the chances that the strategies or other tools are used. He also thinks there is a disconnect between local strategies and national objectives and strategies, such as Moldova 2030 strategy, and that more alignment should be ensured across various policy levels. LPA officials across the board indicated in KIIs that they will continue using and updating strategic plans after CM, and several said directly that they have the capacity to do so. The CATI quantitative data to a large extent also confirms the opinions expressed within KIIs and FGDs that there is interest in the LPAs to continue using the municipal strategic planning workshops (MSPW) and to further implement the strategic plans. Forty-seven LPAs plan to continue using MSPW and further implementation of strategic plans, while 19 say there is interest in using them. Even though there are persistent challenges with limited resources (funding, skilled staff, etc.), as noted above, this kind of planning process seems to be a common requirement for donor funding, therefore, LPAs are externally incentivized to update and use their plans at regular intervals. One LPA official said specifically that the regulations that they have developed for public hearings related to the strategic planning process will live on past the end of the project. At the same time, respondents from sixty-five LPAs indicated in the telephone survey of local officials that they would need at least some technical support to be able to use MSPW again. Also, as shown in Figure C.2 below, in addition to the limited financial resources mentioned above, in the opinion of LPA officials, the limited human resources, the legal and regulatory impediments and the lack of support from the local councils are among other limiting factors that impact the feasibility of developed strategic plans. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 66 Figure C. 2: Challenges in implementing MSPW and strategic plans (# of LPAs) B. CISC According to CM annual reports, five CISCs were launched in mid-December 2020, with an additional seven launched in December 2021. LPAs that had CISCs generally found them to be very useful and effective; when asked which services have been most affected by CM, an LPA official said that due to the CISC all the administrative services have been improved. No respondent spoke negatively of the CISCs themselves (except to say that CISC is not wheelchair accessible, but the issue is being considered and hopefully soon addressed by installing a special lift [FGD, CSOs]). Quite a few specifically praised the CISCs as an effective approach. Four survey respondents named CISCs explicitly as one of the approaches that helped improve services, and all of them said they would continue to use the center. Citizens who reported changes in service provision mainly said that administrative processes have become faster as a result of CM initiatives like CISCs. The citizens and CSOs in the focus groups discussions noted their satisfaction with the opening of CISC’s, mentioning that the process has become more convenient. They especially underlined the fact that there is an opportunity to make a prior appointment, or they can get programmed at the entrance and the ticketing system makes the queueing very organized and rapid. Although a person in an FGD with CSOs mentioned that visitors get a number to be served and there are TV monitors to see whose turn it is to be served next, in none of the 6 observed CISCs (see below) were such systems available. These two quotes from FGDs with CSOs from two towns where CISCs were open with CM support and are already operational illustrate the satisfaction with the CISCs: “I think the time to get a document has decreased. It's getting faster through the current system. [...] It depends of course on how busy the specialist is, but anyway, the waiting time for citizens has shrunk.” [FGD, CSOs] “Because of the Comunitatea Mea project that has been running for a year and a half and has fostered the possibility for the citizen to come with a problem, and go to the town hall and solve it in record time. […] If previously the request was submitted in written form and the deadline was a few days, here it is already resolved more quickly. Sometimes the citizen goes out the same day with his application solved.” [FGD, CSOs] 44 62 63 66 0 10 20 30 40 50 60 70 Lack of support from local council Limited human resources There are legal or regulatory impediments Limited financial resources Number of LPAs CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 67 Another important impact of CM was the provision of computer equipment for CISCs that allowed more services to be managed electronically. Some interviewees noted that the CISCs need to be interoperable with other databases and information systems. CISC operators questionnaire: The interviewed CISC operators mentioned that it takes them between 3 and 15 minutes to resolve a service request received at the CISC. They find that processing service requests entirely online via special software is more convenient and it prevents losing some applications as was the case with paper applications. The operators also reported that citizens find the new modus operandi for accessing administrative services (via CISCs) convenient, and they are pleased. In one LPA a respondent mentioned that in the CISC office in their town hall, there should be a dedicated person to work permanently in the front office, but not involve them all, other workers from the town hall, who do this work by rotation, every day. Observation checklist of operational CISCs: These generally positive comments contrast somewhat with findings from the independent observations of the ET.15These indicate that there is inconsistency in the functionality of CISCs across cohorts. In two of six observed CISCs, each from different cohorts, there were no service desks open with staff members behind them. In one of these locations, the CISC observer encountered initial hostility and resistance from LPA staff until the purpose of the visit was explained. After the purpose of the visit was explained, the desks were immediately staffed and citizens waiting were expediently addressed. In one LPA, there were three service desks, but none were initially staffed upon arrival. The maximum observed staffed service desks across all CISCs was two. In one CISC, there were five service desks, but only one open with staff behind it. Clear instructions were posted in Romanian and Russian in four of six of the CISCs and only one CISC had a visible suggestion box or other mechanisms for visitors to easily provide feedback. Measures to ensure visitor comfort, accessibility, and safety also varied. In three of six CISCs, there was sufficient space (4-6 chairs) for citizens to sit on while they wait. There was no available drinking water in four of six CISCs. There was also no functional restroom available in four CISCs. At three CISCs, anti-COVID-19 measures including disinfectant, social distancing, and masks were not respected. Two CISCs were not wheelchair accessible. Areas of consistency across all CISCs were the presence of a functional printer and computer: all six had a functional computer and five out of six had a functional printer. Additionally, no CISC required citizens to take a number to get served on entry and no payment machine was available at any observed CISC. 15 Six fully operational CISCs were observed across all three cohorts (two from each cohort) between the hours of late morning to late afternoon for a period of two to four hours. Each CISC was visited several times on different days to ensure the observations aligned with the on average "normal" situation at each CISC. The two CISCs observed in Cohort 3 were recently launched in December of 2021. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 68 Figure C. 3: Findings from the observation checklist of operational CISCs CISCs Institutionalization and sustainability: One of the barriers to effective provision of administrative services both through CISCs and through traditional procedures is the fact that LPA staff salaries are low, and thus limit the ability of LPAs to attract qualified candidates. Active citizens explained in one of the focus group discussions: “Nobody wants to work for 5000 lei when in the private sector he/she gets 10,000 lei. […] It is an underfunded domain, let us face it. We know that most LPAs employees are retirees and middle-aged people, who are not very motivated, that is why the quality of services is low. Many of them do not even know information technologies.” [FGD, active citizens]. In addition to low wages and difficulty to attract skilled and motivated staff, the issue seems to be aggravated by the limits on the number of employees (note: in addition to budgetary constraints, the staff number and organization charts have to be endorsed by the state chancellery for every LPA). An active citizen noted that after a discussion with the mayor he realized that the number of LPA staff is insufficient to provide quality services on time, thus the number of employees should be increased. In terms of the sustainability of CISCs, some LPAs said explicitly that they will continue to fund and use CISCs after the end of CM. Some respondents have mentioned that the LPAs have to spend money on administrative services anyway (pay staff salaries, electricity, other utilities, consumables, etc.), and using CISCs doesn't seem to require substantially more funds to run once established. A respondent from USAID also said that CISCs are likely to be sustained and hoped that this approach will be a primary legacy for CM and that one of the most important sources of sustainability for CISCs is funding from other donors. An LPA official mentioned that a CISC will be created in their municipality as part of an EU-funded project. Other donors, such as the World Bank are also planning to provide support, including to the already established CISCs by extending the services provided by them to cover some services provided by the central level that are deconcentrated/delegated at the local level (CUPS – Unified Centers of Public Services). CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 69 CM has developed a solid operational manual for the LPA to use to guide them, including sample regulations, sample documents, and detailed procedures and processes for the organization and operation of the CISC. However, an IREX key informant noticed that there are lots of details to take care of, and also requests for refresher training recently and that it is an ongoing effort. Staff turnover is also an issue because they need to be trained again in how to use the digital systems of CISCs. C. FUNCTIONAL ORGANIZATIONAL CAPACITY ASSESSMENT (FOCAS) All LPAs were required to complete a FOCAS at the start of engagement with CM, which entailed rating 22 different elements within six categories.16 The stated purpose of the assessment was to serve as a baseline for CM activities, and according to the 2021 annual report, the results of three FOCAS guided CM to better customize targeted assistance to the LPAs. Overall, LPA officials were aware of and informed about FOCAS - in the telephone survey, respondents from 64 LPAs, had been involved in the assessment process. In 55 LPAs, respondents knew their LPA’s overall FOCAS score. The tool has a dual purpose: (1) to look across LPAs to measure capacity and progress, and (2) to help individual LPAs to motivate and measure their performance. There is an indication that FOCAS was used regularly for the first purpose, helping CM and USAID measure the effectiveness of the program and make decisions on resource allocation, IREX used the scores to make decisions about which kinds of support they provided for different LPAs. One IREX respondent noted that they are tracking improvements in FOCAS scores from Cohort 1 LPAs and have gradually decreased their levels of assistance to these LPAs as they have shown higher capacities. And the annual “Analysis of FOCAS Results” takes a high-level review of progress, often using aggregate figures to determine overall competence or year-to-year progress. The tool is designed in a way that facilitates that sort of comparison between LPAs or over time, as it is based on aggregated ratings. However, it would need to be used in a different way by individual LPAs in identifying the specific areas where improvement is needed, and it is not clear if distinct approaches were used.17 With respect to use at the LPA level, I LPA officials reported in the survey that they found FOCAS useful generally to help them identify challenges and areas for improvement in their LPA. As seen in Table C.1 below, most telephone survey respondents found the tool easy to understand and motivating to the LPA staff, although two-thirds of LPAs found that the assessment took too much time. 16 The six categories are organizational and strategic planning, service provision, financial management, citizen participation and transparency, project management, and human resources management. 17 Specifically, a variety of different factors are assessed and then aggregated, resulting in ratings that may not reveal quite different results. For example, a single figure for service provisions may mask sharp differences between, say, administrative services and solid waste management, or a rating for organization and strategic planning capacity combining vastly different levels of competence in capital investment planning and local economic development planning. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 70 Table C. 1: Statements about FOCAS and the number of LPAs where respondents agree with those statements STATEMENT AGREEMENT (# LPAS) Those involved in the assessment process found it useful 66 FOCAS is easy to understand 66 FOCAS results help identify needs within the LPA 66 FOCAS results are motivating to LPA staff 66 The assessment process took too much time 43 One LPA Official was very enthusiastic about the potential of FOCAS and was able to even mention the main sections of the tool: “This new FOCAS tool has helped us to assess the current situation, to identify the existing problems and needs, and solutions on how to solve these problems. The fact that it was divided, it had six basic functions: strategic planning, service delivery, financial management, citizen participation and transparency, human resources project management, helped us to understand and see in these six areas, what is the current situation, what we have at the moment, how we are prepared and how we can improve.” [KII, LPA Official] This LPA official respondent went on to describe how areas for improvement were translated into actionable steps that were added to their strategic plans, giving the example of transparency that according to the FOCAS score was lagging behind. They took mitigation measures regarding lack of transparency and in one year time their municipality made remarkable progress in a ranking of transparency made by a national think tank. Several other potential uses for the scores were discussed in key informant interviews. Scores were not used to produce a ranking or to make comparisons between LPAs. LPAs do not have access to the full range of scores or any rankings, although in the survey, respondents from 27 LPAs said they had compared their LPAs scores with the scores obtained by other CM LPAs. An implementing partner respondent stated that “It’s a fine tool and has helped mayors identify areas needing strengthening. [But it is] not a policy or citizen tool.” [KII, Implementing Partner]. Challenges and Sustainability: Several LPA officials stated explicitly that they will continue using the FOCAS tool after the end of the project. However, other types of respondents expressed doubts about its sustainability. There were several challenges that undermine the sustainability of the tool. One is that LPAs think that they would not use the tool regularly if there was no external incentive structure to do so. An LPA official said that while the tool is useful and there is the will to continue using it after the project ends, LPA officials need an external push: “I think it's useful as long as you're somehow guided and when you trust that if you don't achieve something, there's going to be the team behind you who helps you because you always know you achieve something and it seems to you that you are doing well, but when you know intuitively that someone's going to check up on you, you sit down and you evaluate yourself, let's put it that way.” [KII, LPA Official] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 71 One LPA official said that there is no capacity in their LPA to update FOCAS; there is a “high risk for sustainability”. Another LPA official echoed this, saying that they would need more support to continue using the tool. Also, in 43 LPAs the telephone survey respondents think the assessment process took too much time. In KIIs, the interviewed LPAs officials informed the ET that it takes only a few hours (around 2 hours) to apply the tool (to give scores), but it is more difficult to gather all the relevant staff in one room as they are very busy with multiple tasks. They find it more convenient and efficient when they go through the process in a group and discuss each aspect, rather than individually giving scores. Another important obstacle to FOCAS sustainability according to the interviewed key informants is the lack of capacity among LPA staff. Respondents described high fluctuations of LPA staff “as a result of discrepancies in the payroll system of civil servants”. [KII, IREX] Respondents across categories spoke of LPA staff having high workloads and low salaries: “We have one sore point: that we always have a lot of work and not enough people. This is our sore point. Purely professionally, but that's about it.” [KII, LPA Official] “So, it is not difficult to apply. It's quite accessible, it's just that it's harder to mobilize the people responsible for each area to apply this tool. But I tell you, because we have people basically dealing with several activities in the municipality and time is very precious.” [KII, LPA Official] According to the telephone survey results, 65 LPAs were very or somewhat likely to use the FOCAS tool after the CM project ends, and the same number were very or somewhat likely to recommend the FOCAS tool to other LPAs. The two most common reasons for not recommending the FOCAS tool to other LPAs are the process is too time-consuming (17 respondents) and too complicated (12 respondents). Three respondents indicated that the instrument is not useful. Other reasons for not recommending the FOCAS tool listed by respondents themselves include human resources constraints (six respondents) and communication issues (one respondent). FOCAS tool institutionalization: There were some efforts to institutionalize FOCAS at national levels, in particular with the Public Administration Academy, but for the most part this institutionalization process did not happen. IREX respondents said that there were offers made to the GoM and discussions with the State Chancellery, but these had limited impact. At the Public Administration Academy, at least 3 people there know how to use the instrument and one student chose the tool as the topic of their thesis research. A non-USAID donor respondent recommended that someone like CALM, the Public Administration Academy, or some NGO such as IDIS Viitorul support the LPAs in continuing to use FOCAS by reminding them every year to apply the tool, collecting and compiling the scores to create some ranking, or possibly introducing some awards and incentives for LPAs to use it. Based on a Tetra Tech proprietary tool, CM developed a FOCAS user manual that was adapted to the Moldovan context and LPAs' specific needs, but according to the manual and opinions expressed by key informants from IREX, in addition to CM partner municipalities and other LPAs from the country, the manual is open for non-commercial use “by other potential users such as government entities, think tanks, academic groups, researchers and consultants who will be involved in conducting capacity assessments of Moldovan LPAs or of other types of organizations such as NGOs”. [FOCAS User’s manual] However, respondents from non-USAID donors, other projects and government institutions were not aware of the manual availability and where it can be downloaded or acquired. One respondent from a local government focused project expressed the opinion that the lack of a specific CM project dedicated CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 72 webpage where all the developed tools, materials and news would be posted is one of the challenges in making these tools more available and prevents efficient dissemination. D. Communication Strategy The review of the annual reports and the interviews with key informants revealed that the CM work with LPAs on communication strategies implied a series of capacity building events (workshops) and guidance in developing communication plans, in applying various communication tools and specifically in adapting to the new reality and conditions imposed by COVID-19 pandemics. According to the telephone survey, in all 66 LPAs respondents were familiar with CM's assistance to LPAs on communication strategies with citizens. In 37 LPAs, officials felt that the communication campaign or strengthened public outreach had a high contribution to improving citizen engagement; in 28 LPAs, officials felt that it had a moderate contribution and only in1LPA officials felt it had a low contribution. As shown in Table C.2 below, nearly all LPAs have used all of the communication strategies listed in the survey. Table C. 2: Communication strategies and number of LPAs where respondents reported their use/implementation COMMUNICATION STRATEGIES # LPAS Broadcasting local council meetings 63 Developed infographics about the local budget 65 Improved functionality of the webpage 65 Viber communities 66 Developed a Facebook page for your LPA 66 Have bulletin boards to share information with citizens 66 Increased frequency of public hearings 66 Holding more general assemblies / public consultations with citizens 66 Source: Telephone survey of LPA officials Note: Underlying N is between 310 and 334. In addition to the predefined list of communication strategies for the LPA officials to agree or disagree with presented in Table C.2 above, the respondents also have indicated several other communication and citizen engagement approaches. Among them, the respondents have indicated sector meetings (7 LPAs) and neighborhood meetings (in person or online) (9 LPAs), other online consultations/meetings (8 LPAs), posters/magazine/leaflets/brochures/ads (6 LPAs), surveys/Viber poll (6 LPAs), meetings in slums (7 LPAs), focus groups (4 LPAs) and other approaches. Some respondents in FGDs and KIIs, mostly from smaller LPAs, described using methods of communication that pre-dated CM such as going directly to the town hall to speak with the mayor, or CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 73 from the side of the LPA going door to door to speak with citizens. However, especially after the onset of the COVID-19 pandemic, LPAs made much more use of online communication modalities. Most LPAs kept their existing communication modalities and added additional ones which led to a broader range of options; for example, CSO members in a focus group discussion brought up that their LPA has a web page, a Facebook page, a Viber group, petition accumulation points throughout the city, surveys and public hearings. In FGDs, some citizens and CSOs expressed the opinion that the social networks (LPA’s Facebook page specifically) are a more effective and convenient communication/engagement method compared to traditional tools, giving the example of online surveys versus the traditional surveys on paper or door to door. On the other hand, it seems that the LPA’s Facebook page is a more accessible and preferred platform to the classical web pages: “The city hall has a Facebook page and they keep evidence of even the smallest, the smallest things that are done. […] It also has an official website, but it's less visited by people, because it's clear that no one stays to google [name removed] city hall anymore. […] The most effective way is the Facebook page.” [FGD, Active Citizens] LPA officials mentioned in their response to the open-ended (text) question the following results of using various communication strategies: increased citizen engagement (30 LPAs), improved transparency (11 LPAs), and citizens having greater trust in the LPAs/having a better relationship with the government (6 LPAs). Respondents from 4 LPAs also mentioned that as a result of better communication citizens are more financially engaged and are ready to donate money to the LPA for the implementation of community projects (4 LPAs). In 65 LPAs, respondents thought that it was somewhat or very likely that their LPA would continue to update and use a communication strategy in the future after support from CM stops. E. Congress of Local Authorities in Moldova (CALM18) CALM is an important partner in the CM project design and implementation and as such a Memorandum of Understanding was signed between the two organizations. There are two main perspectives of interest for this evaluation: (1) how the project supported and impacted CALM’s organizational development and capacity to fulfill their tasks (such as providing services to their members, advocating for their interests, etc) and (2) how CALM was used by the project to implement CM activities and to reach the stated goals (such as using CALM to disseminate the project results beyond CM partner LPAs). A respondent from IREX spoke specifically about CM project activities to support and build the capacity of CALM. According to key informants, these activities helped CALM improve its communications with members. CM provided space for them to organize meetings and discussions, in particular a series of discussions with members about public administration reform. CM worked together with CALM to set agendas for events and determine who should be invited, and once the pandemic hit, they helped to co￾organize meetings via Zoom. This respondent emphasized that “In our relationship with CALM, we are not the partner that tells them what to do. We have dialogue and determine priorities together.” [KII, IREX] 18 CALM stands for Congresul Authoritatilor Local din Moldova in Romanian. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 74 In this spirit of collaboration, CM has included CALM when organizing discussions with the GoM and other stakeholders around Administrative Territorial Reform. A GoM respondent expressed appreciation that CM facilitated dialogue between them and CALM. A non-USAID donor respondent said that CM had provided financial support to CALM to help them elaborate a vision of local government reform and determine their position and recommendations. However, this respondent said that there should have been technical assistance alongside this financial support; CM’s grant supported CALM staff and experts to do this work instead of bringing in international or local external experts. Despite the efforts of CM to build CALM capacity and bring them into discussions with other stakeholders, one implementing partner respondent asserted that CM had failed to engage enough with CALM. However, a KII respondent said that because CALM has serious issues with financial management and control, their capacity needs to be built before they can be brought in for more engagement. A respondent from a donor and one from GoM described the organizational structure of CALM as disorganized and said that they are difficult to work with. LPA Views of CALM: In the telephone survey, in all 66 LPAs, at least some respondents are familiar with CALM. Also, in all 66 LPAs respondents tend to agree that CALM provides a platform for sharing of knowledge and experience among different LPAs, safeguards the independence of LPAs and contributes to the advancement of decentralization policies. Figure C. 4: Contribution of CALM to the success of the CM project (# of LPAs) In KIIs, while some LPA respondents expressed positive views of CALM, others did not, and no LPA officials interviewed knew about specific collaboration between the CM project and CALM; it seemed that the project’s activities and consultations with CALM were kept separate from its work with LPAs. One LPA Official spoke positively of CALM even though they were not aware of collaboration between CALM and the CM project: 4 2 2 29 29 21 33 35 43 0 5 10 15 20 25 30 35 40 45 50 Provides a platform for sharing of knowledge and experience among different Safeguards the independence of LPAs Contributes to the advancement of decentralization policies Number of LPAs High contr. (# LPAs) Med contr. (# LPAs) Low contr. (# LPAs) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 75 “I don't have information about CALM's collaboration with Comunitatea Mea, but the fact that CALM, it is today a... how to tell you, a force that is there, the only one that is in the local public administration, that still has the ability to defend some interests of local communities. And it's really working in that direction fruitfully and there are results from his work.” [KII, LPA Official] Another LPA official however expressed frustration with CALM: “The voice of the LPAs is not very well heard […] CALM should be the one that unites us and discusses with the central authorities, it should be the voice of the local authorities but in CALM are still the same people and if they are the ones that don't need to be there, they stay there for years and nobody changes them, their quality influences the quality of this dialogue.” [KII, LPA Official] When asked about challenges associated with CALM, CATI respondents from 61 LPAs have indicated that it has limited financial and human capacities, its management needs improvement (57 LPAs) and that it was not engaged enough by the CM project (43 LPAs) or that CALM itself was not interested to get in involved in CM activities (30 LPAs) Figure C. 5: Challenges associated with CALM (# LPAs) In the “other challenges” category that allowed the LPA officials to indicate additional challenges that were not listed, respondents indicated that “CALM raises questions but in the end, it doesn't solve anything”, “to organize more online meetings, I noticed that they work with only a certain group of mayors, it should be more balanced”, “the final result is not seen” and that they “haven’t worked with CALM and the CM did not involve them in the project activities”. In addition to the above challenges, respondents noted that engaging CALM does not solve anything (two LPAs). In addition to the above challenges, respondents noted that engaging CALM does not solve anything (two LPAs) and there was a need for more meetings to be held (two LPAs). Regarding the types of services LPAs receive from CALM, respondents from nearly all LPAs have indicated that they receive training and information (65 LPAs) as well as consultation (64). Respondents from 59 LPAs agreed with the statement that CALM is providing advocacy/lobby services to the LPAs. Respondents in 4 LPAs suggested their LPAs did not receive any services from CALM. 30 43 57 61 0 10 20 30 40 50 60 It was not interested enough to get involved in the CM activities It was not engaged enough by the CM project Its management needs improvement It has limited financial and human resources Number of LPAs CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 76 Figure C. 6: Types of services LPAs receive from CALM (# LPAs) When asked about the improvement in CALM capacity to represent their LPA’s interest in relation to the central government due to CM assistance, respondents from 22 LPAs believe that it has improved a lot, from 41 LPAs - somewhat and from 3 LPAs - a little or not at all. As to the question of whether CALM and the related information tools and networks have been used to communicate results from LPA's CM experience to other LPAs outside the CM project, respondents from 20 LPAs agreed with the statement that it happened a lot and 46 LPAs that it was done somewhat. Figure C. 7: Improvement in CALM capacity to represent your LPA’s and use of CALM to communicate results Potential Future Roles of CALM: Several respondents working at the national level mentioned that CALM could be a partner in institutionalizing CM approaches. A respondent from IREX explained that they are in discussions with CALM about potentially creating an e-learning platform to disseminate CM 4 59 64 65 65 0 10 20 30 40 50 60 70 Not receiving any services Advocacy, lobby Consultations Information Training Number of LPAs 3 46 41 20 22 0 5 10 15 20 25 30 35 40 45 50 CALM and related information tools, networks have been used to communicate results from LPA's CM experience to other LPAs outside CM project Improved capacity of CALM to represent LPA's interest in relation to the central government Number of LPAs A lot (# LPAs) Somewhat (# LPAs) A little/not at all (# LPAs) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 77 tools among mayors across the country. A GoM respondent, speaking about options for institutionalizing public administration courses, called for CALM to be an organizer: “CALM should promote this, that's the duty of CALM. They should come to the Ministry of Finance and say, “We need such courses, let's develop them and we will promote them because our mayors have problems”. [KII17 GoM] An implementing partner respondent called for a platform for donor coordination kept by CALM. F. Community Development Projects CDPs are a key "hard" component of the CM that complements organically the "soft" components related to capacity building, technical assistance or citizen mobilization. According to the latest project documents, 23 communities have already finalized their CDPs, benefitting over 94,000 community members, while 34 other projects are in advanced stages of implementation. In order to ensure the local relevance and ownership of LPA and local stakeholders over CDPs, the CM envisaged a complex mechanism that involves: (i) selection of the project based on the community development strategy and consultations held with the citizens; (ii) co-funding, meaning that LPA had to provide own contribution in order to ensure stronger commitment and ownership over the project; (iii) a prescribed process for planning, engineering, procurements and other relevant procedures ; and, (iv) monitoring the implementation of CDPs by supporting the creation of citizens oversight groups in order to ensure proper accountability and, respectively, quality of CDPs. Despite these complexities, CDPs was one of the most desired components by LPAs, as revealed by many KIIs with LPAs. Also, during KIIs the LPAs acknowledged the importance of citizen participation in selecting the projects. There are two primary ways that citizens were engaged with CDP implementation. One is that they were surveyed and consulted before the CDP was selected to gather their insights and opinions on what kinds of infrastructure projects are the highest priority, and secondly CM organized monitoring groups with citizens to oversee implementation. Citizens in several LPAs spoke of being surveyed both in-person and online, and there were public hearings or consultation meetings that took place for the same purpose of discussing infrastructure priorities for CDPs. The public hearings and consultations overlapped with those for strategic planning—or one could view it as the strategic plans informed CDP selection, so citizen engagement in them also meant that citizens had a say in CDPs. An LPA official explained that their CDP came directly from the action plans developed in their strategic planning process, which involved working groups composed of civil society organizations, LPA officials, and representative organizations for citizens. Still, a number of challenges arose with CDPs. First, most respondents claimed the time-consuming process of approving the project. This is particularly related to requirements to elaborate feasibility studies and specific technical documentation, which was perceived as too much compared to the small scale of CDPs. For example, LPA representative stated during the focus group discussion that "I personally don't understand the sense of so many feasibility studies and I don't know if it is necessary. Once a survey is done, a council decision, a certain number of citizens opting for it, what is the point of doing a study, it wastes time, money for papers." As a result, most CDPs are likely to be finalized in the last year of the CM implementation. Second, the CDPs are subject to rigid regulations induced by the national legislation and USAID/IREX requirements. This is relevant amid raising prices for construction materials and energy. The delays in CDP implementation mentioned above aggravated this problem because the initial budgets CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 78 did not take into account the ongoing rapid inflation. In this regard, two quoted of LPA representatives during focus group discussions are especially relevant: "When I did the project, a ton of asphalt was 1200 lei and it is 1770 now, at the moment a ton of asphalt is 1700-1800 lei. Please tell me, what am I going to do? I have a headache with this project […] I just don't see where I can get the resources to carry out the project, because they have dragged it out so much, because the prices were one way then and otherwise now.” [KII, LPA officials]. “We have a Moldovan law on procurement, it says clearly: ‘If you made a purchase and the price of materials changed, the price for work, then it is the problem of contractors’. […] in terms of the United States of America, perhaps their donors allow it […] this year we will have to adjust all the estimates […] That is three years in order, and now another gas price increase is expected, by the summer there will be another increase in materials. And we end up with these $60,000, what we're talking about, they're melting under our eyes, figuratively speaking.” [KII, LPA officials] The third challenge is related to the incapacity of many LPAs to provide their own contribution to CDPs, because of limited local tax base, putting them in a vicious circle of development. Some respondents stated that instead of asking for their own contribution, the CM should have put more emphasis on empowering LPAs to fundraise and identify the co-financing for CDPs for other donors. Given these reservations regarding CDPs, their sustainability is also equivocal. On the one hand, the CM enforced some important mechanism that bode well for sustainability. One was the assurance of local ownership over the projects through citizen participation in project selection and execution oversight, as well as the own contribution of the LPA. In fact, CM set a new standard for implementing local development/capital projects in terms of transparency, inclusiveness, accountability and technical preparations. This means citizens are more likely to push LPAs to stick to these principles for any other future projects. On the downside, this is based on the assumption that the community stays actively engaged, which could be challenged by the migration process, difficult economic and social conditions and eroding local financial autonomy. Another challenge is related to the fact that CM did not focus on post-execution monitoring because most CDPs are planned to be implemented in the last year of the project. Coupled with limited financial resources, especially amid the ongoing crisis, this fuels two-fold risks: (i) LPAs will not be able to ensure proper maintenance over time; and (ii) it is not clear how the quality of CDPs will evolve over time and if LPAs will be able to enforce the eventual warranty provided by CDPs' implementers. G. Citizen-led Monitoring CM supported the creation of and provided capacity building to Community Oversight Groups (COGs) which monitored the implementation of CDPs, training the COGs' members on project management, technical aspects, and safety requirements to empower them to conduct proper independent oversight of CDPs. Importantly, CM complemented this activity with an awareness raising campaign in order to fuel the citizens' interest in getting involved in such activities, which is an important condition for ensuring sustainability of this action. Other activities supporting citizen monitoring of local government included public hearings and the citizen report cards (CRCs) used for SIAPs. The CRCs provide an important feedback loop to LPAs about the quality of local public services, budgetary process or an infrastructure project. The CM provided this tool only to LPAs implementing the SIAP methodology around CDPs, whereas it might have been more broadly applicable. Notably, CM did not include citizen CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 79 monitoring of procurement processes, which might be of significant value in Moldova because of high corruption risks. In 43 LPAs, officials felt that citizen-led monitoring (of LPA, public services and budget) had a moderate contribution to improving citizen engagement; in 22 LPAs, they felt that it had a high contribution and in only 1 LPA, they felt it had a low contribution. With respect to sustainability, 44 LPAs intend to continue using citizen-led monitoring. H. Citizen Debates on Administrative Territorial Reform The CM activity on promoting administrative territorial reform was primarily focused at the central/policy level. It was undermined by limited political will for this reform at the central level and a narrow understanding about the importance of this reform at the grassroots level. Overall, CM intervention at the grassroot level, by engaging citizens in this process, can be considered insufficient, being focused several public events and an information campaign. In the telephone survey when asked whether the LPA participated in holding citizen debates on administrative-territorial reform, out of 61 LPAs where this question was answered by either mayor or deputy mayor, 19 suggested that LPA participated in holding citizen debates, while 42 said that LPA did not participate. Among 75 officials who said that their LPA participated in holding citizen debates, 72 believed these events were well organized, 71 thought that the debates were useful to citizens, and 70 respondents suggested the debates increased public understanding of decentralization. In 65 LPAs, CM work on decentralization policy was seen as rather useful for future functioning of LPA. I. Revenue Enhancement Development (RED) The CM provided tailored technical support to LPAs on extending the local tax base (e.g. practical strategies for revenue generation, including implementation of new fees and taxes, adjusting bases for existing charges, and facilitating new economic activities) and improving public finance management (e.g. expanded guidance on relevant MoF regulations, hands-on support on local taxes and fees collection management etc.). These activities were highly relevant and even of strategic importance for local development, given the limited and permanently eroding local financial autonomy (local revenues represent only one quarter total public revenues) and ineffective financial management and system of local taxes. The impact of these activities can be assessed using statistical data and feedback from LPAs. Namely, for all cohorts, the total own revenues of LPAs improved significantly compared to the year before joining the project. Thus, beneficiary LPAs outperformed the rest of LPAs in terms of growth of own revenues: • Cohort 1: +47% compared to 34% for the country average • Cohort II: +57% compared to 31% for the country average • Cohort III: +34% compared to 25% for the country average According to the survey data, in 37 LPAs, officials felt that RED had a moderate contribution to an increase in local source revenues; in 24 LPAs, they felt that it had a high contribution and only in 5 LPA, they felt it had a low contribution. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 80 The increase in own revenues of LPAs was uneven and correlated with the dynamics registered by FOCAS scores related to the financial management capacity. As revealed by Figure C.8, the top performer from cohort 1 was Bilicenii Vechi, which also registered a positive improvement in the FOCAS score "Own Revenue Generation and Fundraising" (+1 point in 2020 compared to 2018) and "Budgeting Process and Financial Management'' (+1 point in 2020 compared to 2018), outpacing the average of 0.63 and 0.83 points respectively. Another LPA that performed well was Ialoveni, where FOCAS registered significant progress according to the "Budgeting Process and Financial Management" score (1.67 points in 2020 compared to 2018). From cohort 2, the top performers were Drochia and Cimislia. In the case of Cimislia, there is a notable improvement according to the FOCAS scores "Own Revenue Generation and Fundraising" and "Budgeting Process and Financial Management'' (in both cases, +1 point in 2020 compared to 2019), but in the case of Drochia, surprisingly, the FOCAS does not reflect any improvement in financial management capacity. At the same time, the least performers, such as Ciuciuleni or Crihana Veche, where own revenues increased slower compared to the country average, there is no progress on financial management capacity according to FOCAS. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 81 Figure C. 8: Increase in own revenues of LPAs of cohort 1, compared to country average, (2021 compared to 2018), % -53% 21% 25% 32% 32% 37% 39% 42% 48% 58% 58% 60% 61% 63% 70% 87% 109% 123% 125% 228% 34% -100% -50% 0% 50% 100% 150% 200% 250% LPAs from cohort 1 Country average CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 82 Figure C. 9: Increase in own revenues of LPAs of cohort 2, compared to country average, (2021 compared to 2019), % 10% 14% 21% 23% 24% 27% 37% 37% 39% 40% 44% 47% 47% 47% 50% 50% 55% 57% 58% 60% 62% 77% 77% 88% 105% 106% 31% 0% 20% 40% 60% 80% 100% 120% LPAs from cohort 2 Country average CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 83 Figure C. 10: Increase in own revenues of LPAs of cohort 3, compared to country average, (2021 compared to 2020), % -27% -5% 8% 12% 15% 15% 16% 20% 21% 26% 26% 26% 28% 32% 34% 34% 40% 42% 43% 45% 47% 48% 52% 53% 56% 57% 72% 78% 128% 24% -40% -20% 0% 20% 40% 60% 80% 100% 120% 140% LPAs from cohort 3 Country average CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 84 Therefore, in order to sustain the process of consolidation of local tax base and local financial autonomy, the efforts should continue to strengthen the financial management capacity, with a focus on the budgeting process, local taxes and, especially the asset management system, where LPAs registered relatively low scores. According to the telephone survey data, all 66 LPAs seem to be willing to continue using the RED tool, among them 51 LPAs are likely to continue, and 14 LPAs say there is interest to continue. At the same time, the majority of LPAs (63) need at least some support to be able to use the RED tool, only 2 LPA can continue using RED without outside assistance. Figure C. 11: Challenges in using the RED tool (# LPAs) J. Public Property Asset Management The CM support on public property asset management included the support on Public Property Management system development (e.g. elaboration of PAM, trainings on property valuation etc.) and introduction of innovative tools for property management (e.g. GIS-local platform). Importantly, the CM managed to ensure a good coordination with other donors with similar activities, mainly with the World Bank Land Registration and Property Valuation Project, Norwegian Mapping and Cadaster Authority, the EU Twinning Project, the Agency for Land Relations and Cadaster of Moldova, and the EU funding mechanisms for peer-to-peer learning (TAIEX). The impact of these activities was less obvious compared to RED. According to the telephone survey, in 44 LPAs, officials felt that improved public property management had a moderate contribution to an increase in local source revenues; in 17 LPAs, they felt that it had a high contribution and in 5 LPA, they felt it had a low contribution. A review of budgetary data was not conclusive in showing results. LPAs expressed their interest to continue these approaches/activities. According to the survey, all 66 LPAs seem to be willing to continue using public property management, among them 41 LPAs are likely to continue, and 24 LPAs say there is interest to continue. The majority of LPAs (64) need at least some support to be able to use public property management. 64 62 53 35 51 0 10 20 30 40 50 60 Limited financial resources Limited human resources There are legal or regulatory impediments The local council will not support it Limited demand/interest from local citizens Number of LPAs CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 85 Figure C. 12: Challenges in using the public property management tool (# LPAs) K. Transparent Program Based Budgeting CM provided support to LPAs to ensure greater participation of citizens, civil society, and the private sector in budgeting activities. It included such activities as promotion of GESI responsive budgeting, participatory budgeting, budgetary public hearings and budgetary planning. According to the survey among LPAs, in 48 LPAs, officials felt that Transparent Program Based Budgeting had a moderate contribution to an increase in local source revenues; in 48 LPAs, they felt that it had a high contribution and in 2 LPA, they felt it had a low contribution. In terms of ensuring sustainability, there is room for improvement. All 66 LPAs seem to be willing to continue using Transparent Program Based Budgeting, among them 47 LPAs are likely to continue, and 18 LPAs say there is interest to continue. The majority of LPAs (59) need at least some support to be able to use Transparent Program Based Budgeting, only 6 LPA can continue using RED without outside assistance. Figure C. 13: Challenges in using Transparent Program Based Budgeting (# LPAs) 63 61 55 32 50 0 10 20 30 40 50 60 Limited financial resources Limited human resources There are legal or regulatory impediments The local council will not support it Limited demand/interest from local citizens Number of LPAs 59 63 53 34 52 0 10 20 30 40 50 60 70 Limited financial resources Limited human resources There are legal or regulatory impediments The local council will not support it Limited demand/interest from local citizens Number of LPAs CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 86 ANNEX D – EVALUATION STATEMENT OF WORK USAID/MOLDOVA MID-TERM PERFORMANCE EVALUATION SCOPE OF WORK PURPOSE OF THE EVALUATION The purpose of this action is to conduct a performance evaluation of Comunitatea Mea (CM) Project - a USAID/Moldova flagship local governance program, which supports the Development Objective 1 of USAID Moldova/s Country Development Cooperation Strategy (CDCS): “Strengthened Participatory Democracy”through IR 1.2 “Responsiveness of Targeted Public Institutions Strengthened”. The Mission has selected to conduct the performance evaluation for the following reasons: • In preparation for the next local governance program, the Mission and the Democracy and Governance Office is interested in evaluating the performance of Comunitatea Mea activity, which of the approaches employed by the project proved to be effective and not, and in identifying potential future interventions in the local governance area. • CM is the largest activity in the current DG portfolio and has the widest reach of partner communities. • The evaluation of Comunitatea Mea has been included in the Mission PMP and Evaluation Plan and is part of the Mission wide learning agenda. SUMMARY INFORMATION NAME COMUNITATEA MEA Implementing Mechanism Cooperative Agreement with International Research and Exchange Board (IREX) Total Estimated Ceiling of the Evaluated Project $20,499,961 Life of Project March 5, 2018 – March 4, 2023 Development Objectives DO1: Strengthening Participatory Democracy IR 1.2 Responsiveness of Targeted Public Institutions USAID Office USAID/Moldova BACKGROUND A. DESCRIPTION OF THE PROBLEM AND CONTEXT The fundamental obstacle to democratic development in Moldova is a lack of accountability in the governance system, which creates a negative political environment that excludes citizens from political CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 87 processes; undermines the rule of law; impedes the delivery of basic services; enables corruption to thrive; and renders Moldova vulnerable to external interference. Local government in Moldova is extremely fragmented. The majority of Level 1 local public authorities (LPAs) (meaning villages, towns, and municipalities) struggle with limited funds to cover their mandates, inadequate staff, and obsolete infrastructure that makes service delivery, which LPAs are responsible for, almost impossible without heavy investments from the central government or donor organizations. Comunitatea Mea (CM) is a $20.5 million, five-year activity that aims to strengthen local government to become more effective, transparent, and accountable to citizens. The activity is in its fourth year of implementation. It strives to build local government capacity to meet citizens’ needs. The assumption of this activity is that by increasing the capacity of Local Public Administrations (LPAs), including their ability to work inclusively with citizens and civil society, not only services will be improved, but checks and balances between the central and local governments will also be strengthened, improving the overall system of democratic governance in Moldova. CM serves as USAID/Moldova’s flagship local government program, providing comprehensive assistance to Level 1 communities, including towns, villages, and municipalities, primarily targeting local self￾governing bodies. The program set out at its award to work with up to 100 communities, and currently works in 77, selected through three application cohorts, and representing a range of sizes and geographical locations across Moldova. The principal beneficiaries are the first level of local public administrations that engage citizens to develop their communities and improve public services. The program is also working with central government partners, such as the State Chancellery, Ministry of Finance, Ministry of Agriculture, Regional Development and Environment, Cadastral Agency, e￾Governance Agency, the Congress of Local Authorities of Moldova (CALM), other donor-funded programs to advance decentralization policy reform and increase LPA financial viability. Comunitatea Mea seeks to achieve the following goal, objectives, and results over a five-year period: CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 88 Program Goal: Local Government in Moldova is more effective, transparent, and accountable to citizens Component 1: Improved quality and access to municipal services Component 2: Citizens meaningfully engaged in local governance Component 3: Decentralization policy reforms advanced Component 4: Increased locally-owned revenues and improved financial management practice Results: R1.1: Improved governance in local administrations and local service providers R1.2: Quality of and inclusive access to public services and infrastructure improved for all citizens R1.3: Administrative services and permitting improved in targeted LPAs R1.4: Local administrations more transparent and accountable to citizens R1.5: Greater citizen involvement in planning processes and monitoring service performance R1.6: Strengthened accountability of local government and service delivery using innovative approaches and tools Results: R2.1: LPAs’ decision-making and budgeting processes are more open and transparent R2.2: Citizens are knowledgeable about democratic and transparent government processes R2.3: Citizens are more engaged and active in local government affairs R2.4: Marginalized and vulnerable groups play meaningful roles in local government affairs Results: R3.1: Legislation/ institutional framework adopted to enhance fiscal base of LPAs R3.2: Legislation passed to address the fragmentation of local government units (i.e., administrative territorial reform) R3.3: LPAs and civil society voice in policy reform affecting local governance is strengthened R3.4: CSOs use evidence￾based approaches to monitor, assess impact and report on Government commitment to and progress on decentralization reform Results: R4.1: Increased revenue under the discretion of LPAs R4.2: More transparent LPA budgeting and financial management R4.3: Increased revenues from improved asset management R4.4: Diversified revenue base for LPAs R4.5: At least 20 local governments have increased capacity in assets evaluation and implementation of asset management CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 89 Monitoring Data Comunitatea Mea has developed a robust Monitoring framework, including an outcome focused MEL Plan and extensive data collection methods. The MEL plan synthesizes the main data collection methods into indicators tracking outcomes at the four components of the activity. Data collection methods • FOCAS. CM uses the FOCAS tool as the main planning and monitoring tool for LPA capacity development. All LPAs have gone through a baseline assessment under all sections of the tool (planning, financial management, revenue generation, revenue generation, asset management, service provision). On an annual basis, LPAs conduct repetitive assessments to identify progress in each of the areas. All FOCAS reports are available. • Village population survey. CM conducts a village population survey with a representative sample at least twice within the activity time frame (baseline and progress). The survey collects data on perceptions of village people towards the mayoralty and their interaction. To date, baseline and progress surveys have been conducted for 2 out of three cohorts - 47 LPAs. • Revenue generation. CM is closely tracking the revenues accumulated by LPAs, both transferred from state budget and locally and own generated. Data is available within CM records. EVALUATION QUESTIONS 1. How is Comunitatea Mea making progress toward the achievement of its goals and objectives? To what extent have the activities implemented under Comunitatea Mea achieved their expected results, against each of the four objectives, and to what extent have they been able to collectively contribute to its Goal: “Local Government is more effective, transparent, and accountable to citizens”? CM has been addressing the demand and supply side of the governance. On the demand side CM has invested in community engagement practices, such as creation of active civic groups “pune umarul”, Community Oversight Groups; on the supply side, CM developed the capacity of LPAs to better communicate and engage citizens, engage librarians, organize strategic planning sessions and budget hearings, distribute and collect feedback through questionnaires, etc.. To what extent do citizens have higher expectations/demand better services? Are LPAs more responsive? To what extent is communication between citizens and Governments occuring ? Is the communication generated solely by and while CM is active, or is there an expectation for continuity? 2. To what extent the approaches employed by IREX/Comunitatea Mea were effective in reaching the stated results? CM applied a number of approaches to realize its four objectives. This includes the approach to decentralization reform, use of community development projects, efforts to foster inter-municipal cooperation and others. To what extent have these and other approaches been adopted and shown benefits? Which of the community practices were institutionalized by LPAs, such as FOCAS, SIAPs, CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 90 community surveys, budget hearings, participatory budgeting, Performance and Risk Management Systems? Comunitatea Mea is working with three different Cohorts of LPAs selected competitively, reaching directly 77 LPAs and planning to achieve up to 100 LPAs through Inter-municipal Cooperation (IMCs). How effective is this phased approach? How feasible is the total number of LPAs over the course of the project? How effective is strategic planning in the process of community development that USAID should invest in? Several activities used or proposed technological solutions to improve transparency and efficiency of services. To what extent IT investments promise to become successful and sustainable? 3. Within Objective 4, what impact did Comunitatea Mea have on enhancing locally owned revenues and financial viability of the assisted 77 LPAs? CM is collecting data on LPA revenues, including locally owned ones. Which of the practices implemented with Comunitatea Mea’s assistance led to increased revenues? Does the methodology and assistance to asset management in assisted municipalities led to improvement of asset management? 4.Looking forward and building on the successes and failures of Comunitatea Mea, which areas should USAID support further to build stronger local governments and better local democracy where USAID has a competitive advantage? What are other areas with a potential to generate development outcomes in the Local Governance area that USAID might expand its support to? What is the optimal number of LPAs that USAID should build the capacity of over a project lifetime? Methodology The evaluation must comply with USAID Evaluation requirements as stated in the ADS and the USAID Evaluation Policy. The expected evaluation type is a Performance Evaluation. The evaluation team must use a comprehensive evaluation design and methodology. The evaluation is expected to employ a non-experimental design and use a mixed method approach (e.g., desk review, interviews, focus group discussions, key informant interviews, monitoring indicators, etc.) that will generate the highest quality and most credible evidence on each evaluation question. The evaluation design should entail a more robust methodology than typical performance evaluations. Specifically, the evaluation team is expected to conduct data collection in a meaningful sample of the 77 targeted local governments. This might entail a greater amount of intensive field work for the team as a whole or less￾intensive, structured data collection in a large number of localities. Part of the Methodology, the contractor will describe how each of the methods used, individually or collectively, will contribute to answering the Evaluation questions and the overall purpose of the evaluation. Other data collection methods must be considered, and should be proposed by the contractor. Note. Considering the evolution of the pandemic situation in Moldova and the US, the evaluation team must consider an alternative plan for fieldwork, including employment of local consultants and usage of IT tools and approaches to remote evaluation. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 91 Currently, Moldova is experiencing a low spread of the COVID-19 virus in the country, the daily number of cases are at around 30 - 40 but with a slight increase in the number of imported cases. There are no restrictions on movements within the country, wearing of masks is mandatory only inside. Foreigners arriving in Moldova must present either a PCR test conducted within 72 hours or proof of administration of a COVID-19 vaccine. Team Composition The Implementer will propose the appropriate team composition to be able to fulfill the evaluation tasks. The team should include expertise on local governance and local finance. At least one member of the evaluation team should be a Romanian speaker, and appropriate arrangements made for translation for other team members, including those who may conduct interviews in predominantly Russian-speaking communities. Deliverable(s): 1. Initial kick-off call. Following the receipt of the Tasking, the evaluation team will connect with the Mission to discuss in detail the expectations for the evaluation, clarify the evaluation questions and other important elements before preparing the full scope and budget. 2. Evaluation kick-off call. Within 2 weeks of the evaluation start date, the contractor will arrange for a phone call or video conference with USAID/Moldova and other involved parties. The introductory discussion will cover the workplan, methodology and inputs from each side. 3. Biweekly briefings. The contractor must provide biweekly debriefings to the Activity Manager on the evaluation’s progress to date. 4. Evaluation Work plan and Design: Within three weeks of the kick-off, but not less than two weeks prior to arrival in Chisinau, a draft work plan for the evaluation shall be presented to the Activity Manager. The work plan will include: a) the anticipated schedule and logistical arrangements, including list of potential interviewees, sites to be visited, proposed selection criteria and/or sampling plan; b) evaluation design and methodology and data collection instruments A final Evaluation workplan will be submitted within a week after receiving comments from the Mission. 5. In-Briefing. Within 5 days of arrival in Chisinau the evaluation team will have an in-briefing with the USAID Mission for introductions and to discuss the assignment, initial assumptions, methodology, and work plan. 6. Exit Briefing. The evaluation team is expected to hold an exit briefing to the Mission prior to leaving the country to discuss the status of data collection and preliminary findings and conclusions. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 92 7. Draft Evaluation Report: The draft evaluation report must be submitted to the Activity Manager within five weeks of the evaluation team’s departure from Moldova. The evaluation report structure must correspond to the “USAID Evaluation Report Requirements”, a Mandatory Reference for ADS Chapter 201. The report must use analytical text chapters as well as visual graphics, charts and gauges to facilitate understanding and at-a-glance view of progress. Once the initial draft evaluation report is submitted, USAID Moldova will have 14 business days in which to review and comment on the initial draft, after which point the Activity Manager will submit the consolidated comments to the evaluation team. The evaluation team will submit a revised final draft report 10 business days hence. 8. Recommendation and action plan workshop: A recommendations workshop can be organized between the evaluation team and USAID project team to work on the evaluation recommendations and subsequent action plan that will be initiated by the evaluation team but completed by USAID/Moldova. 9. Final Evaluation Report: The evaluation team will, no later than 10 days after receiving final comments on the draft report, respond/incorporate the comments and resubmit the final report to the Activity Manager. The final evaluation report must be uploaded to the Development Experience Clearinghouse at dec.usaid.gov. 10. Two-page summary: The contractor will produce a user-friendly synthesis or summary of the evaluation in a format that is attractive and understandable for public use. This will be translated into Romanian and made available in English, Russian and Romanian and will be emailed to evaluation participants. 11. Dissemination: The contractor will organize a dissemination meeting with the IP or broader stakeholder participation, depending on the recommendations generated. 12. Utilization follow-up: To incentivize evaluation use, the learning partner will follow-up three months and six-months after the finalization of an action plan to document evaluation utilization. Tentative Dates of performance and timeline: • September/October – field work • November – draft report • January – final report CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 93 ANNEX E – DATA COLLECTION INSTRUMENTS CATI Introduction I am calling from IMAS, and we are working with NORC at the University of Chicago that is conducting a study of the Comunitatea Mea project, financed by USAID. Your mayoralty is one of the beneficiaries of this project and we would like to ask you some questions regarding the assistance you received. The information that you provide will help USAID improve future similar projects in Moldova. Do you have 30 minutes for a discussion now? (If not; Can we agree on a time that you will be available for me to call back?) Confidentiality We want to assure you that the information you provide will not be attributed to you personally. Your privacy will be protected; we will not include your name or any information in our reports that would make it possible to identify you without your consent. Right to Ask Questions and Report Concerns You have the right to ask questions about this assessment and to have those questions answered by us. If you have questions or need confirmation about the evaluation please contact Ms. Elena Petruti at IMAS. Landline: 022 26 00 96; Mobile: 079 227 587; Email: office@imas.md Right to Refuse or Withdraw The decision to participate in this study is entirely up to you. You may refuse to complete the survey, and you have the right to stop taking it at any point. You will not be penalized if you choose to withdraw your participation. There are no known risks or direct benefits for you from participating in this assessment. Nevertheless, your feedback is very important for assessing the effectiveness of Comunitatea Mea project and for the future work of USAID in the field of local development. Consent to Participate in Survey [consent] Do you agree to participate in this survey? 1. Yes 0. No [PROG: If consent=0 exit survey] I. SORTING QUESTIONS [location_LPA] [ENUM: Do not read aloud] LPA location [PROG: Drop down list] [official_type] [ENUM: Do not read aloud] Type of official [PROG: Drop down list] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 94 1. Mayor 2. Deputy Mayor (if LPA > 5000) (citizen engagement, communications) 3. Cadastral Engineer 4. Budget/Finance/Economy specialist, or accountant (whoever is responsible for the budget 5. Specialist on Communal services 6. Secretary of the Local Council (admin, social, gender services; citizen engagement; communication) 7. Other Service specialists II. INTRODUCTORY QUESTIONS [type_assist] [PROG: Mayor or Primary Interviewee Only] Your LPA received many different types of assistance from CM. How was it decided what types of assistance would be given? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Findings during strategic planning 2. Findings from the baseline FOCAS tool 3. Discussions with LPA leadership 4. Followed suggestions made by CM project 5. Our LPA budget 6. The LPA's local development strategy 97. Other, please specify: _______________ 98. Don’t know 99. No response [useful_strategic] [PROG: Mayor or Primary Interviewee Only] How useful has it been for your LPA to start working with CM by focusing on your strategic planning and priorities? 1. Very useful 2. Somewhat useful 3. Not very useful 4. Not useful at all 98 Don’t know 99 No response [component_sort] [PROG: Display to all respondents] In your work at the LPA, in which area or areas do you consider yourself to work: [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Delivering services (such as communal services, infrastructure, administrative services, social services) [PROG: Display Component 1 module] 2. Working directly with citizens, engaging with civil society [PROG: Display Component 2 module] 3. Issues regarding decentralization such as functional responsibility, fiscal base and territorial reform. [PROG: Display Component 3 module] 4. Revenues, financial management, economic development [PROG: Display Component 4 Module] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 95 PROG: If respondent selects 1, display section 4: component 1; If respondent selects 2, display section 5: component 2; If respondent selects 3, display section 6: component 3; If respondent selects 4, display section 3: component 4 SECTION 1: STRATEGIC PLANNING [PROG: DISPLAY TO ALL RESPONDENTS] We’d like to start by asking you some questions about your LPA’s strategic plan. [strategicplan_fam]. Are you familiar with your LPA’s strategic planning process or Strategic Plan? 1. Yes 0. No [SKIP TO: Communications Section] 98. Don’t know [SKIP TO: Communications Section] [strategicplan_partic] Did you yourself participate in the Strategic planning workshop? (MSPW) 1. Yes 0. No 98. Don’t know [strategicplan_feedb] To what extent was feedback from citizens and/or civil society groups included in the plan? 1. A lot 2. Somewhat 3. A little 4. Not at all 98. Don’t know 99. No answer [strategicplan_involv] [PROG: Display if strategicplan_feedb=1, 2, or 3] How were citizen and/or civil society groups involved in the elaboration and the decision making on the approval of the strategic plan? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Inclusion in the working group on the elaboration 2. Focus groups 3. Survey/interviews 4. Publication of the draft strategy on webpage/etc. and receiving written suggestions 5. Public hearings/debates 6. Other 98. Don’t know [strategicplan_vulpop] To what extent do you think that your Strategic Plan takes into account the specific needs of the following types of populations? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 96 Types of Populations 1. A lot 2. Somewhat 3. A little 4. Not at all 97. Not applicable 98. Don’t know 99. No answer Women Different ethnicities People with disabilities Vulnerable children Elderly people LGBTI people Veterans Other special groups, please specify: _______ [strategicplan_when] When was your Strategic Plan approved by the Local Council? 1. Year_______ 2. It was not adopted 3. We do not have a Strategic Plan [PROG: SKIP TO Communications] 98. Don’t know [strategicplan_link] To what extent was the Strategic Plan linked with the local budget? 1. A lot 2. Somewhat 3. A little 4. Not at all 98. Don’t know 99. No answer [strategicplan_impl] To what extent has your LPA implemented your Strategic Plan? 1. Fully 2. Mostly 3. Some 4. A little 5. We have not started 98. Don’t know CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 97 [strategicplan_chall] To the extent of your knowledge, has your LPA faced any of the following challenges in implementing your strategic plan? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. LPA staff are not motivated 2. The LPA does not have sufficient funds to implement the activities 3. The Strategic Plan is not realistic 4. Insufficient LPA staff 5. Insufficient expertise 6. Did not have enough support from the local council 7. Did not have enough support from local citizens 97. Other, please specify: _____________ 98. Don’t know [strategicplan_cont] How likely do you think it is that your LPA will continue to update and use strategic planning in the future, after support from CM stops? 1. Very likely 2. Somewhat likely 3. Somewhat unlikely 4. Not at all likely 98. Don’t know 99. Not applicable [strategicplan_funds] Has your LPA used the strategic plan to seek additional funds outside of your LPA (grants, projects, other assistance)? 1. Yes 0. No [PROG: If no, ask strategicplan_extfun] [strategicplan_succ] If yes, was it successful? 1. Yes 0. No [strategicplan_extfun] Has your LPA sought additional funds outside of your LPA to support implementation of the strategic plan? 1 Yes 0 No 98 Don’t know [strategicplan_useful] In your view, how useful was strategic planning to your LPA? 1. Very useful 2. Somewhat useful CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 98 3. Not very useful 4. Not at all useful 98 Don’t know SECTION 2: Communications [PROG: Display to all respondents] [comms_familiar] Are you familiar with the work that CM has done with your LPA on developing a communication strategy with your citizens? 1. Yes 0. No [PROG: Skip to Other_Officials] [comms_initiated] Which of the following communication strategies have you initiated within your LPA? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Viber communities 2. Developed infographics about the local budget 3. Developed a Facebook page for your LPA 4. Have bulletin boards to share information with citizens 5. Improved functionality of the webpage 6. Increased frequency of public hearings 7. Broadcasting local council meetings 8. Holding more general assemblies/public consultations with citizens 9. Other, please specify____________ 10. You have not developed a communication strategy for local citizens [PROG: Skip to Other_officials] 98. Don’t know 99. No response [comms_result] In your opinion what has been the result of using this communication strategy? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Citizens have more information about the LPA 2. Citizens understand better the local budget 3. Citizens are more engaged in the community 4. There has been no result 5. Other, please specify______________ 98 Don’t know 99 No response [comms_likely] How likely do you think it is that your LPA will continue to update and use a communication strategy in the future, after support from CM stops? 1. Very likely 2. Somewhat likely CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 99 3. Somewhat unlikely 4. Not at all likely 98. Don’t know 99. Not applicable [other_officials] We have additional questions based on the type of assistance provided by CM to your LPA. Please confirm that you were involved in this work. [ENUM: Confirm the fields that are written in the tracking sheet are correct for this respondent] [ENUM: Ask this part to Mayors ONLY] Please also confirm the name and contact information of other LPA officials who have interacted with CM on each of the following topics [ENUMERATOR: please confirm that the name and contact information provided by the mayor is the same as what is listed in the tracking sheet] Type of Assistance Provided by CM Yes No 1. Strategic planning 2. Communication 3. CALM 4. FOCAS 5. Communal services (roads, street lighting, solid waste collection, etc.) and Community Development plan and small infrastructure project 6. Citizen participation 7. Decentralization Policy 8. Local Own Source Revenues (including asset management) PROG: If respondent selects 6, display section 4: component 1; If respondent selects 7, display section 5: component 2; If respondent selects 8, display section 6: component 3; If respondent selects 6, display section 3: component 4 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 100 SECTION 3: Other questions A. Other LPAs [PROG: Display to Deputy Mayor only] [otherlpas] In your work on activities with CM, have you ever interacted with other LPAs involved in CM? 1. Yes 0. No [PROG: SKIP TO otherlpas_meet] 98. Don’t know [PROG: SKIP TO calm_fam] [otherlpas_why] If yes, was this to: [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. To receive training or other technical assistance together 2. To share your experiences 3. To provide help to each other 4. To implement Inter Municipal Cooperation initiatives together 97. Other, please specify: ________ 98. Don’t know [otherlpas_how] How did you interact with the other LPAs? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. In small meetings with 3 or fewer other LPAs 2. In large meetings with more than 3 other LPAs 3. Shared materials (guides or training material) 4. On-going personal contact 97. Other, please specify: ________ 98. Don’t know [otherlpas_useful] On the whole, how useful would you consider these interactions to be in helping you understand strategies to improve services, citizen engagement or OSR? 1. Very useful 2. Somewhat useful 3. Not very useful 4. Not at all useful 98. Don’t know 99. No reply [otherlpas_meet] [PROG: Display if otherlpas=0] Would you be interested in meeting counterparts from other CM LPAs? 1. Yes 0. No 98. Don’t know CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 101 B. CALM [calm_fam] Are you familiar with CALM? 1. Yes 0. No [PROG: Skip to otherdonors] [calm_agree] To what extent to you agree with each of the following statements about CALM? (agree strongly, agree, disagree, strongly disagree) 1. Provides a platform for sharing of knowledge and experience among different LPAs 2. Contributes to the advancement of decentralization policies 3. Safeguards the independence of LPAs 98. Don’t know 99. No answer [calm_success] To what extent did CALM contribute to the success of the CM project? 1. Greatly 2. Somewhat 3. A little 4. Not at all 98 Don’t know 99 No reply [calm_challenge] [PROG: Multiselect] Did CALM face any of the following changes in helping achieve the objectives of the CM Project? [ENUM: Respondent can mention more than one] 1. It was not engaged enough by the CM project 2. It was not interested enough to get involved in the CM activities 3. It has limited financial and human resources 4. Its management needs improvement 97. Other, please specify: _______ 98 Don’t know 99 No reply [calm_services] What types of services does your LPA receive from CALM? [ENUM: Respondent can mention more than one] [PROG: Multiselect]: 1. Information 2. Training 3. Consultations 4. Advocacy, lobby 5. It is not receiving any services [PROG: Exclusive] 98. Don’t know 99. No reply CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 102 [calm_capacity] To the best of your knowledge, do you think the capacity of CALM to represent your (LPAs) interest in relation to the central government has improved as a result of CM assistance to them? 1. A lot 2. Somewhat 3. A little 4. Not at all 98. Don’t know 99. No answer [calm_comm] To your knowledge, was CALM and their information tools, networks used to communicate results from your CM experience to other LPAs outside CM project: 1. A lot 2. Somewhat 3. A little 4. Not at all 98. Don’t know 99. No answer C. Other Donors [PROG: Display to Deputy Mayor only] [other_donor] Has your LPA received assistance from any other program or international donor during the last five years? 1. Yes 0. No [other_donor who] [PROG: Display if other_donor=1] Which other programs or international donors has your LPA received assistance from? [ENUM: Respondent can mention more than one] ____________________ [otherdonors] In your opinion has work with CM helped you leverage and obtain additional funding from another donor? 1. Yes 0. No 98. Don’t know D. FOCAS [PROG: Display to all respondents] [FOCAS_familiar] Are you familiar with the Functional Organizational Capacity Assessment (FOCAS) that your LPA completed? 1. Yes 0. No [PROG: Skip to end of block] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 103 [FOCAS_involved] Were you involved in the assessment process itself? 1. Yes 0. No [FOCAS_agree] To what extent do you agree with each of the following statements? (strongly agree, agree, disagree, strongly disagree) 1. Those involved in the assessment process found it useful (strongly agree, agree, disagree, strongly disagree, 98. Don’t know, 99. No answer) 2. The assessment process took too much time (strongly agree, agree, disagree, strongly disagree, 98. Don’t know, 99. No answer) 3. FOCAS is easy to understand (strongly agree, agree, disagree, strongly disagree, 98. Don’t know, 99. No answer) 4. FOCAS results helps identify needs within the LPA (strongly agree, agree, disagree, strongly disagree, 98. Don’t know, 99. No answer) 5. FOCAS results are motivating to LPA staff (strongly agree, agree, disagree, strongly disagree, 98. Don’t know, 99. No answer) [FOCAS_score] Do you know your FOCAS overall score? 1. Yes 0. No [PROG: Skip to FOCAS_Sustainability] [FOCAS_compare] Have you compared your own score with the scores obtained by other CM LPAs? 1. Yes 0. No 98. Don’t know 99. No answer [FOCAS_sustainability] In your opinion, how likely is your LPA to use the FOCAS tool after the CM project ends? 1. Very likely 2. Somewhat likely 3. Somewhat unlikely CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 104 4. Very unlikely 98. Don’t know 99. No answer [FOCAS_recommend] How likely are you to recommend the FOCAS tool to other LPAs? 1. Very likely 2. Somewhat likely 3. Somewhat unlikely 4. Very unlikely 98. Don’t know 99. No answer [FOCAS_recwhynot] [PROG: Display if focas_recommend=3 or 4] Why not? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. The process is too complicated 2. The process is too time-consuming 3. The instrument is not useful 4. Other, please specify: _______ 98. Don’t know 99. No answer SECTION 4: COMPONENT 1. IMPROVED QUALITY AND ACCESS OF SERVICES [PROG: DISPLAY IF OTHER_OFFICIALS=5] [comp1_servicequal] Have you seen any improvement in the quality of the following services due to the work of CM? [PROG: Questions about quality and access should be asked in tandem, i.e., for each service, ask first about quality and then (if relevant) about access.] Quality of Service 1. A lot of improvement in quality 2. Some improvement in quality 3. A little improvement in quality 4. No improvement in quality 98 Don’t know 99 Not applicable Street cleanliness Street & Sidewalk condition Street lighting Solid waste collection Libraries CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 105 Quality of Service 1. A lot of improvement in quality 2. Some improvement in quality 3. A little improvement in quality 4. No improvement in quality 98 Don’t know 99 Not applicable Permits (business, building) Social services (health and education) Centralized water supply Wastewater Markets Parks & sports facilities Schools / kindergartens Transportation infrastructure Cemetery Subsidies received from the LPA CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 106 [comp1_serviceaccess] Have you seen any improvement in the access to each of the following services due to the work of CM? Access to Service 1. A lot of improvement in access 2. Some improvement in access 3. A little improvement in access 4. No improvement in access 98. I don’t have enough information to answer Solid waste collection Libraries Permits (business, building) Social services (health and education) Centralized water supply Markets Assistance citizens get from the LPA administration [PROG: If comp1_serviceaccess=4 or 98, skip to comp1_asstother] [comp1intro] To what extent, if any, would you say the following CM approaches, tools, or practices contributed to the improvement in services in general. We have several questions about the type of assistance provided by CM to your municipality. [PROG: In the following table, respondents will be first asked the entirety of column A. Following that, they will be asked questions B, C, D, and E for each of the approaches where the answer was 1, 2, or 3.] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 107 Component 1: Improved services [comp1_a] Contributed to an improvement in services 1. A great deal 2. Some 3. A little 4. No contributio n at all 98. I don’t know this tool [PROG: Those who answer 4 or 5 will not answer questions in columns B-E] [comp1_b] To what extent is there interest in your LPA continue to use this tool/approach? 1. Yes, we will continue 2. There is great interest 3. There is some interest 4. No interest 98. Don’t know. [PROG: Do not ask comp1_d and comp1_e if comp1_b=No interest] [comp1_d] To what extent would your LPA need technical support to be able to use this tool again? 1. We could do it with no outside assistance 2. We could do it with limited technical support 3. We could do it with a lot of technical support 98.Don’t know [comp1_e] Other than technical support, what would be the main challenges in using this tool again? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Limited financial resources 2. Limited human resources 3. There are legal or regulatory impediments 4. Lack of support from local council 5. Limited demand/interest from local citizens 98.Don’t know 99.Not applicable 1. Strategic Planning Workshops (MSPW) and further implementatio n of strategic plans [Comp1_otherapp]: In your opinion is there any other CM approach used by your LPA that helped improve services that you would like to mention? 1. Yes, please specify: ______________ 0. 98. Don’t know [PROG: Skip to comp1_asstcont] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 108 Component 1: Improved services [comp1_a_oth erapp] Contributed to an improvement in services 1. A great deal 2. Some 3. A little 4. No contributio n at all 98. I don’t know this tool [PROG: Those who answer 4 or 5 will not answer questions in columns B-E] [comp1_b_oth erapp] To what extent is there interest in your LPA continue to use this tool/approach? 1. Yes, we will continue 2. There is great interest 3. There is some interest 4. No interest 98. Don’t know. [PROG: Do not ask comp1_d_other app and comp1_e_other app if comp1_b_other app =No interest] [comp1_d_otherap p] To what extent would your LPA need technical support to be able to use this tool again? 1. We could do it with no outside assistance 2. We could do it with limited technical support 3. We could do it with a lot of technical support 98. Don’t know [comp1_e_otherapp] Other than technical support, what would be the main challenges in using this tool again? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Limited financial resources 2. Limited human resources 3. There are legal or regulatory impediments 4. Lack of support from local council 5. Limited demand/interest from local citizens 98. Don’t know 99. Not applicable [PROG: Display Piped text from Comp1_oth erapp] [comp1_asstcont] Is there assistance from CM other than the above approaches that contributed to improvements in services? 1. Yes, please specify: ____________________ 0. No 98. Don’t know [comp1_asstother] Is there other assistance that would help improve services that you would like CM or another future program to provide? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 109 1. Yes, please specify: ____________________ 0. No 98. Don’t know SECTION 5: COMPONENT 2: CITIZENS MEANINGFULLY ENGAGED IN LOCAL GOVERNANCE [PROG: DISPLAY IF OTHER_OFFICIALS=6] [comp2_engage] To what extent do you think the following have increased in your LPA as a result of CM assistance? 1. A lot 2. Some 3. A little 4. Not at all 98. Don’t know a. Engagement of citizens with your LPA b. Citizens’ access to information about your LPAs work c. The LPA’s responsiveness to citizen needs? d. Citizens’ trust in the LPA e. Citizens’ participation in the budgetary process [PROG: If comp2_engage=4 or 98, skip to comp2_asstother] [comp2_intro] To what extent would you say the following activities contributed to the improvement in citizen engagement in local governance. We have several questions about the type of assistance provided by CM to your municipality. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 110 [PROG: In the following table, respondents will be first asked the entirety of column A. Following that, they will be asked questions B, C, D, and E for each of the approaches where the answer was 1, 2, or 3.] Component 2: Citizens meaningfully engaged in local governance [comp2_a] Contributed to an improvement in citizen engagement 1. A great deal 2. Some 3. A little 4. No contribution at all 98. I don’t know this tool [PROG: Those who answer 4 or 5 will not answer questions in columns B-E] [comp2_b] To what extent is there interest in your LPA for these approaches/tool s to continue to be used? 1. Yes, we will continue 2. There is great interest 3. There is some interest 4. No interest 98. Don’t know [PROG: Do not ask comp2_d and comp2_e if comp2_b=No interest] [comp2_d] To what extent would your LPA need technical support to be able to make use of this tool again? 1. We could do it with no outside assistance 2. We could do it with limited technical support 3. We could do it with a lot of technical support 98. Don’t know [comp2_e] Other than technical support, what would be the main challenges within the LPA in using this tool again? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Limited financial resources 2. Limited human resources 3. There are legal or regulatory impediments 4. Lack of support from local council 5. Limited demand/interest from local citizens 98. Don’t know 99. Not applicable 1. Strengthened LPA public outreach, communication , transparency (including: a) grassroots for democratic development” methodology (GDD) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 111 b) “Development of LPA’s communication strategy” workshops c) Training librarians to conduct surveys) 3. Citizen led monitoring of LPA, public service, budget [Comp2_otherapp] In your opinion is there any other CM approach used by your LPA that helped increase citizen engagement that you would like to mention? 1. Yes, please specify: ______ 0. No [PROG: Skip to comp2_asstcont] 98. Don’t know [PROG: Skip to comp2_asstcont] Component 2: Citizens meaningfully engaged in local governance [comp2_a_ot herapp] Contributed to an improvement in citizen engagement 1 A great deal 2. Some 3. A little 4. No contribution at all 98. I don’t know this tool [PROG: Those who answer 4 or 5 will not answer questions in columns B-E] [comp2_b_ otherapp] To what extent is there interest in your LPA for these approaches/tool s to continue to be used? 1. Yes, we will continue 2. There is great interest 3. There is some interest 4. No interest 99.Don’t know [comp2_d otherapp] To what extent would your LPA need technical support to be able to make use of this tool again? 1. We could do it with no outside assistance 2. We could do it with limited technical support 3. We could do it with a lot of technical support 98.Don’t know [comp2_e_otherapp] Other than technical support, what would be the main challenges within the LPA in using this tool again? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1 Limited financial resources 2. Limited human resources 3. There are legal or regulatory impediments 4. Lack of support from local council 5. Limited demand/interest from local citizens 98. Don’t know 99. Not applicable CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 112 [PROG: Do not ask comp2_d_othe rapp and comp2_e_other app if comp2_b_othe rapp =No interest] PROG: Display Piped text from Comp2_othe rapp] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 113 [comp2_asstcont] Is there assistance from CM other than the above approaches that contributed to the increase in citizen engagement? [ENUM: Respondent can mention more than one] 1. Yes, specify: ____________________ 0. No 98. Don’t know [comp2_asstother] Is there other assistance that would increase citizen engagement that you would like CM or another future program to provide? 1. Yes, specify: ____________________ 0. No 98. Don’t know [lpa_feedback] [PROG: Display if respondent worked on Component 1 or 2] What is your LPA doing to collect feedback on its service quality from your citizens? 1. Office hours 2. Suggestion box 3. Citizen survey, 4. Feedback at CISCs 5. Other, please specify ________ 6. Nothing 98 Don’t know [lpa_extentfeedb] [PROG: Display if lpa_feedback=1, 2, 3, 4, 5] To what extent have you used this feedback in your job? 1. A great deal 2. Some 3. A little 4. Not at all 98 Don’t know CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 114 SECTION 6: COMPONENT 3: DECENTRALIZATION POLICIES [PROG: DISPLAY IF OTHER_OFFICIALS=7] [comp3_aware] We would now like to ask you some questions about your awareness of CM activities in areas related to decentralization policy, and the extent of their impact on your LPA. CM Decentralization Activities on: [comp3_a] Are you aware of this work? 1. Yes 0. No 98. Don’t know [comp3_b] Has CM ever consulted with you on this topic? 1. Yes 0. No 98. Don’t remember [comp3_c] Do you believe this work has already had a positive impact on your LPA? 1. Yes 0. No 98. Don’t know [comp3_d] If not, to what extent is it likely to have a positive impact on your LPA? 1. Very likely 2. Somewhat likely 3. Not very likely 4. Not at all likely 98. Don’t know Measures to enhance the fiscal base of LPAs Administrative Territorial reform Methodology for waste management tariffs Review / improvement of functionals assignments (for example, of communal services, urban planning, social services, education, public order) Other CM activities that you know of, please specify: ______ [comp3_debates] Has your LPA participated in holding citizen debates on administrative-territorial reform? 1. Yes 2. No 98. Don’t know CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 115 [comp3_agree] If yes, to what extent do you agree with each of the following statements? 1. These events were well organized (strongly agree, agree, disagree, strongly disagree) 2. The debates were useful to citizens (strongly agree, agree, disagree, strongly disagree) 3. The debates increased public understanding of decentralization (strongly agree, agree, disagree, strongly disagree) [comp3_useful] From what you know, do you think CM work on decentralization policy has been useful for future functioning of your LPA? 1. Very helpful 2. Somewhat 3. Not very 4. Not at all helpful 98. I don’t know. SECTION 7: COMPONENT 4: LOCAL OWN SOURCE REVENUES [PROG: DISPLAY IF OTHER_OFFICIALS=8] [comp4_sourcerev] To what extent would you say has working with CM helped your LPA increase own source revenues? 1. A lot 2. Some 3. A little 4. Not at all 98. Don’t know [comp4_increaserev] [PROG: Display if comp4_sourcerev=1, 2, or 3] In your view, which of the following contributed to that increase? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Introduced new taxes 2. New tax rates 3. New fees 4. Adjusted tax base 5. Increase in tax collection 6. Increase in fee collection 7. Improved asset management 8. Other, please specify. 98. Don’t know CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 116 [comp4_imp] To what extent would you say the following tools and practices contributed to the increase in local own source revenues. We have several questions about the type of assistance provided by CM to your municipality. [PROG: In the following table, respondents will be first asked the entirety of column A. Following that, they will be asked questions B, C, D, and E for each of the approaches where the answer was 1, 2, or 3.] Component 4: Tools and Practices [comp4_a] Contributed to an increase in local revenues 1. A great deal 2. Some 3. A little 4. No contribution at all 5. I don’t know this tool [PROG: Those who answer 4 or 5 will not answer questions in columns B-E] [comp4_b] To what extent is there interest in your LPA to continue to use this tool / practice? 1. Yes, we will continue 2. There is great interest 3. There is some interest 4. No interest 5. Don’t know. [PROG: Do not ask comp4_d and comp4_e if comp4_b=No interest] [comp4_d] To what extent would your LPA need technical support to be able to use this tool again? 1. We could do it with no outside assistance 2. We could do it with limited technical support 3. We could do it with a lot of technical support 4. Don’t know [comp4_e] Other than technical support, what would be the main challenges in using this tool again? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Limited financial resources 2. Limited human resources 3. There are legal or regulatory impediments 4. The local council will not support it 5. Limited demand/interest from local citizens 98. Don’t know 99. Not applicable 1.Revenue Enhancement Development (RED) (exploring options to increase revenues) o Includes detailed budget analysis, recommendations, guidance, workshops CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 117 3. Improved public property management policies / systems, innovative tools (asset management, land and buildings owned by LPAs) 7. Transparent, program-based budgeting (infographics, gender-based budgeting, participatory budgeting) [Comp4_otherapp]: In your opinion is there any other CM approach used by your LPA that helped increase revenues that you would like to mention? 1. Yes, please specify: _____ 0. No [PROG: Skip to comp4_assetmgmt] 98. Don’t know [PROG: Skip to comp4_assetmgmt] Component 4: Tools and Practices [comp4_a_ot herapp] Contributed to an increase in local revenues 1. A great deal 2. Some 3. A little 4. No contributio n at all 98. I don’t know this tool [PROG: Those who answer 4 or 5 will not answer questions in columns B-E] [comp4_b_othe rapp] To what extent is there interest in your LPA to continue to use this tool / practice? 1. Yes, we will continue 2. There is great interest 3. There is some interest 4. No interest 98. Don’t know. [PROG: Do not ask comp4_d_othera pp and comp4_e_othera pp if comp4_b_othera pp =No interest] [comp4_d_othera pp] To what extent would your LPA need technical support to be able to use this tool again? 1. We could do it with no outside assistance 2. We could do it with limited technical support 3. We could do it with a lot of technical support 98. Don’t know [comp4_e_otherap p] Other than technical support, what would be the main challenges in using this tool again? [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Limited financial resources 2 Limited human resources 3. There are legal or regulatory impediments 4. The local council will not support it 5. Limited demand/interest from local citizens 98. Don’t know 99. Not applicable CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 118 PROG: Display Piped text from Comp4_otherapp] [comp4_assetmgmt] Did your LPA receive assistance in asset management? 1. Yes 0. No [PROG: Skip to comp4_areasimprov] 98. Don’t know [PROG: Skip to comp4_asstcont] [comp4_improv] [PROG: Display if comp4_assetmgmt=1] If yes, to what extent do you believe that assistance led to improvements in asset management? 1. Major improvement 2. Some improvement 3. Little improvement 4. No improvement 98. Don’t know [comp4_areasimprov] [PROG: Display if comp4_ improv=2] Please indicate which aspects of asset management improved [ENUM: Respondent can mention more than one] [PROG: Multiselect] 1. Improved local inventories 2. Adopted a public asset management plan 3. New or improved registers 4. Increased revenue from asset management 5. Use of GIS-Local platform 6. Other, please specify: ______________ 98. Don’t know [comp4_asstcont] Is there assistance from CM other than the above approaches that contributed to the increase in local source revenues? [ENUM: Respondent can mention more than one] 1. Yes, specify: ____________________ 0. No 98. Don’t know [comp4_asstother] Is there other assistance that would help increase revenues that you would like CM or another future program to provide? [ENUM: Respondent can mention more than one] 1. Yes, specify: ____________________ 0. No 98. Don’t know [closingtext] This concludes our survey. Thank you very much for your time. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 119 INTRODUCTION AND CONSENT STATEMENT – KEY INFORMANT INTERVIEWS KEY INFORMANT INTERVIEWS Hello my name is__________ (interviewer name) and my colleague here is____________ (notetaker name). We are working with NORC at the University of Chicago on behalf of USAID Moldova. We are conducting an evaluation for USAID to understand what has worked well and what has not worked well in the Comunitatea Mea project that aims to strengthen local government. The project is implemented by IREX. This interview will require 60-90 minutes of your time. Your participation in this study is voluntary. If you are unable to answer a question, you may skip it or even stop the interview at any time; there will be no repercussions for this. However, your feedback will be very useful and help USAID design future programs in Moldova. Your responses will be kept confidential and anonymous. The information you provide will be used in summary form only and will not identify you as a participant of this interview. If you have any questions, you may ask them now or later, even after the interview has started. If you wish to ask questions later, you may contact the Team Lead Ritu Nayyar-Stone at NORC or me. I will share contact information with you now. [SHARE RITU’S EMAIL (Nayyarstone-ritu@norc.org) AND YOUR OWN PHONE NUMBER/EMAIL] Do you have any questions for me at this time? [ANSWER QUESTIONS] Do you agree to participate in this interview? Yes No [END INTERVIEW] Thank you. I would like to record this interview with your permission so that our team can make sure we are capturing your words accurately. The recording will not be shared with anyone outside of our research team and it will be destroyed at the end of our study. Would this be okay? Yes No [DO NOT RECORD; TAKE NOTES ONLY] I will now start the recording and will ask you one more time if you agree to participate so that your consent is recorded. [START RECORDING TO GET VERBAL CONSENT] Do you agree to participate in this interview? Yes No KII INTERVIEW GUIDE – LPA OFFICIALS LPA for In-person KII Overall/ General Approaches. Component 1 Approaches Used Component 2 Approaches Used Component 4 Approaches Used Ungheni (Cohort 1) FOCAS, SPWG Municipal Strategic Planning Workshop; improve public Strengthen LPA outreach; Communication strategy workshop; GIS-Local platform; Revenue enhancement (RED); Borrowing best practices*; CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 120 LPA for In-person KII Overall/ General Approaches. Component 1 Approaches Used Component 2 Approaches Used Component 4 Approaches Used transportation infrastructure; IMCs; CIP; Multi-year Financial Planning Development Tomai (Cohort 1) FOCAS, SPWG Road & sidewalk rehabilitation; SIAP Community oversight groups (COGs); Citizen Report Cards; Revenue enhancement (RED); Budget calendar development Jora De Mijloc (Cohort 2) FOCAS, SPWG; Grassroots Democratic Development Methodology; MSPW; Roof on kindergarten repair; Strengthen LPA outreach; transparency and citizen engagement tools; Decentralization Corners; local economic development, support to librarians; Local tourism development action plan; RED Ceadir Lunga (Cohort 2) FOCAS, SPWG; Grassroots Democratic Development Methodology; MSPW; Park reconstruction; Training to develop CISCs Strengthen LPA outreach; Civic Forums- Pune Umarul Investment offers developed; RED; Public Property Management (PPM) training; Pelivan (Cohort 3) FOCAS, SPWG MSPW; Road repair; administrative service improvements; Pune Umarul; Strengthen LPA outreach; transparency and citizen engagement tools Gender budgeting; RED; financial management; Asset management; local economic development Soroca (Cohort 3) FOCAS, SPWG MSPW; Park rehabilitation; Identified to receive CISC (early December 2021); CIP Strengthen LPA outreach; transparency and citizen engagement tools; technical support to librarians; Gender budgeting; RED; financial management; Asset management; local economic development Chisinau CIP GIS-Local platform; MYFP; LTFM; Public property management system development Note: * The participants learned to use different financial instruments for liabilities and to develop annual budget forecasts of liquidity with a monthly distribution CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 121 General .To start, please tell us a little about your involvement with CM. [Ask this question to ALL LPA officials interviewed] Comunitatea Mea Overall Efficacy [Ask these questions to ALL LPA officials interviewed] This first set of questions will focus on CM’s overall activities. 3.1 Yours is a Cohort X LPA to work with the CM project. In your opinion, did this affect your interaction with CM? Do you see any room for improvement? EQ1/2.1.3 In your opinion has governance improved in your LPA due to work with CM? Please elaborate. EQ1/2.2.4 In your opinion which of the several technical approaches being used by CM have been most useful in achieving results within your LPA? . Are any of these technical approaches being institutionalized by your LPA? Which ones? a. Are there any challenges to institutionalization? 4.4 In your opinion, what instruments/activities provided by CM will continue within your LPA following the end of the project? Will you be able to continue their implementation? What would you need in order to continue implementing it? EQ1/2.1.1 In your opinion, were administrative services improved in your LPA due to CM? If yes, how? If no, why not? . Are there challenges to sustaining administrative improvement once the CM project ends? Please elaborate. EQ1/2.1.5 Has assistance provided by CM affected citizens' access to services or the quality of services? . What services have been affected the most? a. What approaches have been used and which has been the most effective? b. Did the quality and access to services change for vulnerable, ethnic/linguistic and disabled population? Please elaborate. c. Will service improvement approaches continue to be used after CM ends? i.e. have they been institutionalized within your LPA? d. Have there been any changes in citizen's assess to service or the quality of services in your LPA recently? If yes, what has changed? EQ1/2.2.5 Has your LPA adopted any new practices during COVID-19 with CM support that you would like to continue going forward? Please elaborate. Coordination and Cooperation [Ask these questions to the Mayor or Deputy Mayor only] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 122 4.2 Was there coordination and cooperation with other donors and donor projects during CM implementation? . Was there any successful cases of complementarity/synchronization with other projects that maximized the project impact in your LPA? a. Is there scope for more cooperation and coordination? 3.2 Have beneficiary LPAs shared their experience and lessons learned with other LPAs from the CM project? Did you learn from other CM beneficiary LPAs? . If yes, how was this done? a. if no, do you think this will be useful? How often do you think it would be useful to meet and share lessons learned? 3.4 As a Cohort 1/Cohort 2 LPA what follow-up or monitoring did CM undertake with your LPA after project activities were implemented? 4.1 In your opinion, . What else can USAID do to help LPAs achieve good local governance that was not addressed so far or has been addressed in a limited manner a. Which are the areas that USAID has sufficiently supported and can phase out of b. which other areas with a potential to generate development outcomes should USAID explore and support to strengthen LGs in Moldova? c. Are there specific activities that were not adequately funded by USAID? If yes, which ones? XX If the CM project would continue for another 5 years, would you still be interested in participating in the project as beneficiary? If yes, in which areas would you require assistance the most? CM Technical Areas Person to Speak with within the LPA 1. Citizen Engagement 0. Strategic Planning 0. FOCAS Tool 0. CISC 0. Community Development (Infrastructure) Project 0. Budgetary Processes / Financial Management 0. Own Source Revenues 0. Decentralization Reform CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 123 Please let us know who within your LPA has engaged with CM on the following technical areas so that we can ask questions to the right group of people: XX is there anything else you’d like to share with me today? [Ask Mayor before finishing the interview. This question is repeated again at the end for other LPA officials that are interviewed] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 124 KII INTERVIEW GUIDE – IMPLEMENTING PARTNERS General .Please tell us a little about how familiar you are with the CM project and your engagement with it. Comunitatea Mea Overall Efficacy This first set of questions will focus on CM’s overall activities. EQ1/2.2.4 In your opinion which of the several technical approaches being used by CM have been most useful in achieving results within targeted LPAs? . Are any of these technical approaches being institutionalized by the LPAs? Which ones? a. Are there any challenges to institutionalization? EQ1/2.1.3 One of CM’s objectives is to contribute to improved governance in local administrations and service providers. Do you feel this has been achieved in target LPAs that have received CM assistance? . Which demand side approaches have been most effective and why? (Ask about: Pune umarul, citizen report cards, work with civil society, etc.) a. Which supply side approaches have been most effective? (Ask about: communication campaigns, budget hearings, strategic plans etc.) b. In your opinion which approaches are sustainable and will likely continue after CM ends? 4.4 In your opinion, what approaches used by CM will continue to be used by LPAs following the end of the project? EQ1/2.1.5 Has assistance provided by CM affected citizens' access to services or the quality of services? . What services have been affected the most? a. What approaches have been used and which has been the most effective? b. Did the quality and access to services change for vulnerable, ethnic/linguistic and disabled population? Please elaborate. c. Will service improvement approaches continue to be used after CM ends? i.e. have they been institutionalized within the LPAs? Citizen Engagement EQ1/2.2.3 In your opinion do LPAs that are receiving assistance from CM have lower or higher levels of citizen engagement in government? Please elaborate with concrete examples. . Do marginalized and vulnerable groups have opportunities to engage with government? If yes, how? If no, why not? a. Will any of these citizen engagement approaches get institutionalized in the LPA? If yes, which ones? If no, why not? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 125 b. Are there any opportunities for citizens to engage with CM target LPAs and give input or feedback on local priorities or satisfaction with services? If yes, please elaborate. Key Tools We are now going to discuss some of the key tools used by CM. EQ1/2.1 In your opinion, how useful is the Functional Organizational Capacity Assessment (FOCAS) tool to identify capacity gaps at the local level? . Were there any challenges in implementing the FOCAS within each LPA? a. Do you feel that the FOCAS tool will be used by LPAs after the CM project ends? If yes, what are the benefits you foresee and are there procedures in place to ensure the tool is institutionalized? If not, what are the limiting factors? EQ1/2.4.2 Which of the approaches used by CM to improve LPA own source revenues have been most effective? . In your opinion which approach is most likely to continue after the CM program ends? EQ1/2.4.3 Has the cooperation with CM helped LPAs increase their revenues from improved asset management? . What have been the challenges in implementing the framework of Public Real Property Asset Management? a. Where are most LPAs in achieving this framework? b. In your opinion will LPAs continue with asset management after CM ends? EQ1/2.4.4 How are the internal financial management controls enforced by beneficiary LPAs? . What was the contribution of CM with the institutionalization of this instrument? Decentralization Reform EQ1/2.3.2 What assistance has CM provided on the development of a legislation/institutional framework aimed at enhancing fiscal, administrative and political decentralization in Moldova? . What have been the challenges? (Work so far has focused on PEA, waste mgmt tariffs, research on Inter Municipal Cooperation, local taxes analysis, online meeting of local councils, Local Governance Fellowship Program etc.) EQ1/2.3.3 Has CM been effective in building the capacity of civil society to have a voice in decentralization policy reform? Please elaborate. . if not, what have been the main challenges? a. What tools/approaches worked or did not work, and why? 1. How do you think the results achieved in individual beneficiary LPAs would be affected by the implementation of administrative territorial reform? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 126 Coordination and Cooperation 4.2 Was there coordination and cooperation with other donors and donor projects during CM implementation? . Was there any successful cases of complementarity/synchronization with other projects that maximized the project impact? a. Is there scope for more cooperation and coordination? 1. In your opinion, . What else can USAID do to help LPAs achieve good local governance that was not addressed so far or has been addressed in a limited manner a. Which are the areas that USAID has sufficiently supported and can phase out of b. Which other areas with a potential to generate development outcomes should USAID explore and support to strengthen LGs in Moldova? c. Are there specific activities that were not adequately funded by USAID? If yes, which ones? XX Is there anything else you would like to share? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 127 KII INTERVIEW GUIDE – GOVERNMENT OF MOLDOVA General .Please tell us a little about how familiar you are with the CM project and your engagement with it. CM Overall Efficacy This first set of questions will focus on CM’s overall activities. EQ1/2.1.3 One of CMs objectives is to contribute to improved governance in local administrations and service providers. Do you feel this has been achieved in target LPAs that have received CM assistance? a. Which demand side approaches have been most effective and why? (Ask about: Pune umarul, citizen report cards, work with civil society, etc.) b. Which supply side approaches have been most effective? (Ask about: communication campaigns, budget hearings, strategic plans etc.) c. In your opinion which approaches are sustainable and will likely continue after CM ends? EQ1/2.2.4 In your opinion which of the several technical approaches being used by CM have been most useful in achieving results within targeted LPAs? a. Do you know if any of these technical approaches are being institutionalized by the LPAs? Which ones? b. Are there any challenges to institutionalization? 4.4 In your opinion, what approaches used by CM will continue to be used by LPAs following the end of the project? EQ1/2.1.5 To your knowledge, has assistance provided by CM affected citizens' access to services or the quality of services? a. What services have been affected the most? b. What approaches have been used and which has been the most effective? c. Did the quality and access to services change for vulnerable, ethnic/linguistic and disabled population? Please elaborate. d. Will service improvement approaches continue to be used after CM ends? i.e. have they been institutionalized within the LPA? EQ1/2.2.3 In your opinion do LPAs that are receiving assistance from CM have lower or higher levels of citizen engagement in government? Please elaborate with concrete examples. a. Do marginalized and vulnerable groups have opportunities to engage with government? If yes, how? If no, why not? b. Will any of these citizen engagement approaches get institutionalized in the LPA? If yes, which ones? If no, why not? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 128 Financial Management EQ1/2.2.1 In your opinion is the budgetary process in LPAs receiving assistance from CM more or less open and transparent compared to other LPAs in Moldova? Please elaborate. a. Have there been any challenges in implementing participatory budgeting? [Note only 2 LPAs have implemented participatory budgeting (civic budgeting)] EQ1/2.4.2 Which of the approaches used by CM to improve LPA own source revenues have been most effective? a. In your opinion which approach is most likely to continue after the CM program ends? Does the government plan to promote specific laws, policies or reforms that would strengthen the local tax bases and financial autonomy based on CM support/instruments/results? Decentralization Reform EQ1/2.3.1 Did CM provide the GoM any technical assistance in preparing and passing legislation to address the fragmentation of local government units (i.e., administrative￾territorial reform)? a. Did you have any discussions/collaboration with the CM project on territorial reform? If yes, what is your opinion about those discussions/collaborations? Did you find them useful/constructive? Did it lead to any results or follow-up measures? b. What have been the biggest challenges in working with the CM on issues dealing with administrative-territorial reform? c. Were approaches modified to deal with the challenges? EQ1/2.3.2 What assistance has CM provided on the development of a legislation/institutional framework aimed at enhancing fiscal, administrative and political decentralization in Moldova? a. What have been the challenges? (Work so far has focused on PEA, waste mgmt tariffs, research on Inter-Municipal Cooperation, local taxes analysis, online meeting of local councils, Local Governance Fellowship Program etc.) b. Which of the regulatory/legislative initiatives that CM has worked on so far with GoM have been the most impactful or least effective? EQ1/2.3.3 Has CM been effective in building the capacity of civil society to have a voice in decentralization policy reform? Please elaborate. a. If not, what have been the main challenges? b. What tools/approaches worked or did not work, and why? EQ1/2.4.2 Which of the approaches used by CM to improve LPA own-source revenues have been most effective? [Ask about asset management here at one of the approaches] a. In your opinion which approach is most likely to continue after the CM program ends? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 129 EQ1/2.4.4 How are the internal financial management controls enforced by beneficiary LPAs? a. What was the contribution of CM with the institutionalization of this instrument? 4.3 How do you think the results achieved in individual beneficiary LPAs would be affected by the implementation of administrative-territorial reform? Cooperation and Coordination 4.2 Was there coordination and cooperation with other donors and donor projects during CM implementation? a. Was there any successful cases of complementarity/synchronization with other projects that maximized the project impact? b. Is there scope for more cooperation and coordination? 4.1 In your opinion, a. What else can USAID do to help LPAs achieve good local governance that was not addressed so far or has been addressed in a limited manner b. Which are the areas that USAID has sufficiently supported and can phase out of c. Which other areas with a potential to generate development outcomes should USAID explore and support to strengthen LGs in Moldova? d. Are there specific activities that were not adequately funded by USAID? If yes, which ones? XX Is there anything else you would like to share? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 130 KII INTERVIEW GUIDE – LOCAL GOVERNANCE ASSOCIATIONS General . Please tell us a little about how familiar you are with the CM project and your engagement with it.. Comunitatea Mea Overall Efficacy This first set of questions will focus on CM’s overall activities. 3.1 Do you think CM was able to work effectively across all LPAs in cohort 1, 2, and 3? Do you see any room for improvement? EQ1/2.1.3 One of the CM’s objective is to contribute to improved governance in the local administrations and in service providers. Do you feel this has been achieved in target LPAs that have received CM assistance? a. Which demand side approaches have been most effective and why? (Ask about: Pune umarul, citizen report cards, work with civil society, etc.) b. Which supply side approaches have been most effective? (Ask about: communication campaigns, budget hearings, strategic plans etc.) c. In your opinion which approaches are sustainable and will likely continue after CM ends? EQ1/2.2.4 In your opinion which of the several technical approaches being used by CM have been most useful in achieving results within targeted LPAs? a. Are any of these technical approaches being institutionalized by the LPAs? Which ones? b. Are there any challenges to institutionalization? 4.4 In your opinion, what approaches used by CM will continue to be used by the LPAs following the end of the project? Compared to other donor projects, what makes CM different? In your opinion, what are its relative strengths and weaknesses? EQ1/2.1.5 Has assistance provided by CM affected citizens' access to services or the quality of services? a. What services have been affected the most? b. What approaches have been used and which has been the most effective? c. Did the quality and access to services change for vulnerable, ethnic/linguistic and disabled population? Please elaborate. d. Will service improvement approaches continue to be used after CM ends? i.e. have they been institutionalized within the LPA? EQ1/2.5 How effective or ineffective was the support provided by CM to target LPAs to improve their communication with citizens? a. What have been the concrete results, and have they been measured? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 131 EQ1/2.2.3 In your opinion do LPAs that are receiving assistance from CM have lower or higher levels of citizen engagement in government? Please elaborate with concrete examples. a. Do marginalized and vulnerable groups have opportunities to engage with government? If yes, how? If no, why not? b. Will any of these citizen engagement approaches get institutionalized in the LPAs? If yes, which ones? If no, why not? c. Are there any opportunities for citizens to engage with their LPA and give input or feedback on local priorities or satisfaction with services? If yes, please elaborate. Key Tools EQ1/2.1.4 How effective and useful has it been to start assistance to the CM target LPAs by focusing on strategic planning and priorities? a. In your opinion will strategic plans be regularly updated and implemented once CM ends? b. To what extent do strategic plans take into account citizen needs in the community? To which extent do the plans take into account gender issues, or the specific needs of different ethnic/linguistic or disabled groups? c. Do LPAs have sufficient funds to implement the elaborated strategic plan? How feasible are the priorities stipulated in the plans? EQ1/2.1 In your opinion, how useful is the Functional Organizational Capacity Assessment (FOCAS) tool to identify capacity gaps at the local level? a. To your knowledge, were there any challenges in implementing the FOCAS within each LPA? b. Do you feel that the FOCAS tool will be used by LPAs after the CM project ends? If yes, what are the benefits you foresee and are there procedures in place to ensure the tool is institutionalized? If not, what are the limiting factors? c. Do you think FOCAS is the right instrument to self-measure the capacities of LPAs and would it be useful to be scaled up at a national level? If so, how? EQ1/2.1.6 What Community Development projects were implemented with the support of CM? What were the main tools used in this regard? a. How did citizens engage in these projects? b. Have LPAs allocated operation and maintenance costs for these infrastructure projects going forward? c. How useful is the Community Development Strategy to prioritize infrastructure projects in beneficiary LPAs? Would this approach be useful for other LPAs outside the CM project? d. Do you know any experience of CM target LPA’s with Capital Improvement Plan? How useful is this tool in planning and securing investment projects? e. Will LPAs continue using these CDPs once CM ends? Budgetary Processes/ Financial Management EQ1/2.2.1 In your opinion is the budgetary process in LPAs receiving assistance from CM more or less open and transparent compared to other LPAs in Moldova? Please elaborate. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 132 a. Have there been any challenges in implementing participatory budgeting? [Note only 2 LPAs have implemented participatory budgeting (civic budgeting)] EQ1/2.2.2 Are citizens consultations and oversight aligned to provide input into the budgetary calendar? Who sets the dates for these consultations? Which of the approaches used by CM to improve LPA own source revenues have been most effective? a. In your opinion which approach is most likely to continue after the CM program ends? EQ1/2.4.3 Has the cooperation with CM helped LPAs increase their revenues from improved asset management? a. What have been the challenges in implementing the framework of Public Real Property Asset Management? b. Where are most LPAs in achieving this framework? c. In your opinion will LPAs continue with asset management after CM ends? EQ1/2.4.4 How are the internal financial management controls enforced by beneficiary LPAs? a. What was the contribution of CM with the institutionalization of this instrument? b. Do you think the LPAs plan to use/enforce the internal financial management controls in the future? Decentralization Reform EQ1/2.3.2 What assistance has CM provided to the GoM on the development of a legislation/institutional framework aimed at enhancing fiscal, administrative and political decentralization in Moldova? a. What have been the challenges? (Work so far has focused on PEA, waste mgmt tariffs, research on Inter Municipal Cooperation, local taxes analysis, online meeting of local councils, Local Governance Fellowship Program etc.) EQ1/2.3.3 Has CM been effective in building the capacity of civil society to have a voice in decentralization policy reform? Please elaborate. a. if not, what have been the main challenges? b. What tools/approaches worked or did not work, and why? EQ1/2.3.5 In your opinion how comprehensive are the sentinel indicators developed by CM to monitor the decentralization reform? a. Was CALM consulted in the development of the 8 sentinel indicators? b. To your knowledge how will these indicators be used? EQ1/2.3.4 Was CALM's capacity to advocate and represent member LPAs on decentralization reform affected by cooperation with CM? Please elaborate. . What activities were especially effective? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 133 a. What could be improved? Were there any challenges? EQ4 How do you think the results achieved in individual beneficiary LPAs would be affected by the implementation of administrative territorial reform? Coordination and Cooperation 4.2 Was there coordination and cooperation with other donors and donor projects during CM implementation? a. Was there any successful cases of complementarity/synchronization with other projects that maximized the project impact? b. Is there scope for more cooperation and coordination? 3.2 Have beneficiary LPAs shared their experience and lessons learned with each other? a. If yes, how was this done? b. if no, do you think this will be useful? 3.6 Has CM disseminated the results of the project beyond the targeted beneficiary LPAs? a. If yes, did CALM play any role in this? 4. In your opinion, a. What else can USAID do to help LPAs achieve good local governance that was not addressed so far or has been addressed in a limited manner b. Which are the areas that USAID has sufficiently supported and can phase out of c. which other areas with a potential to generate development outcomes should USAID explore and support to strengthen LGs in Moldova? d. Are there specific activities that were not adequately funded by USAID? If yes, which ones? XX is there anything else you’d like to share with me today? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 134 KII INTERVIEW GUIDE – DONORS Please tell us a little about how familiar you are with the CM project and your engagement with it. Overall CM Efficacy This first set of questions will focus on CM’s overall activities. EQ1/2.1.3 One of the CM objectives is to contribute to improved governance in local administration and service providers. Do you feel this has been achieved in target LPAs that have received CM assistance? a. Which demand side approaches have been most effective and why? (Ask about: Pune umarul, citizen report cards, work with civil society, etc.) b. Which supply side approaches have been most effective? (Ask about: communication campaigns, budget hearings, strategic plans etc.) c. In your opinion which approaches are sustainable and will likely continue after CM ends? Key Tools We are now going to discuss some of the key tools used by CM. EQ1/2.2.4 In your opinion which of the several technical approaches being used by CM have been most useful in achieving results within targeted LPAs? a. Are any of these technical approaches being institutionalized by the LPAs? Which ones? b. Are there any challenges to institutionalization? EQ1/2.1.5 To your knowledge, has assistance provided by CM affected citizens' access to services or the quality of services? a. What services have been affected the most? b. What approaches have been used and which has been the most effective? c. Did the quality and access to services change for vulnerable, ethnic/linguistic and disabled population? Please elaborate. EQ1/2.2.3 In your opinion do LPAs that are receiving assistance from CM have lower or higher levels of citizen engagement in government? Please elaborate with concrete examples. a. Do marginalized and vulnerable groups have opportunities to engage with government? If yes, how? If no, why not? b. Will any of these citizen engagement approaches get institutionalized in the LPA? If yes, which ones? If no, why not? EQ1/2.4.2 Which of the approaches used by CM to improve LPA own source revenues have been most effective? a. In your opinion which approach is most likely to continue after the CM program ends? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 135 EQ1/2.2.1 In your opinion do beneficiary LPAs have more transparent and participatory budgeting processes compared to other LPAs as a result of the CM project? is the budgetary process in LPAs receiving assistance from CM more or less open and transparent compared to other LPAs in Moldova? Please elaborate. Decentralization Reform EQ1/2.3.2 Are you aware about the assistance that CM has provided on the development of a legislation/institutional framework aimed at enhancing fiscal, administrative and political decentralization in Moldova? [if yes, follow with detailed questions, if not skip to the next question] a. In your opinion, what approaches and types of assistance provided by CM have been most effective? b. And what approaches have not worked well? c. What have been the challenges? (Work so far has focused on PEA, waste mgmt tariffs, research on Inter-Municipal Cooperation, local taxes analysis, online meeting of local councils, Local Governance Fellowship Program etc.) EQ1/2.3.3 Has CM been effective in building the capacity of civil society to have a voice in decentralization policy reform? Please elaborate. a. If not, what have been the main challenges? b. What tools/approaches worked or did not work, and why? Coordination and Cooperation 4.2 Was there coordination and cooperation with other donors and donor projects during CM implementation? a. Was there any successful cases of complementarity/synchronization with other projects that maximized the project impact? b. Is there scope for more cooperation and coordination? 4.1 In your opinion, a. What else can USAID do to help LPAs achieve good local governance that was not addressed so far or has been addressed in a limited manner b. Which are the areas that USAID has sufficiently supported and can phase out of c. which other areas with a potential to generate development outcomes should USAID explore and support to strengthen LGs in Moldova? d. Are there specific activities that were not adequately funded by USAID? If yes, which ones? 4.4 In your opinion, what approaches used by CM will continue to be used by the LPAs following the end of the project? Compared to other donor projects, what makes CM different? In your opinion, what are its relative strengths and weaknesses? a. When your organization selects an LPA to work with do you take into account if the LPA was a beneficiary of the CM project? b. Would you use the FOCAS tool to evaluate potential beneficiary LPAs for your future projects? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 136 c. Would you use any of the CM implemented approaches in your own projects? XX Is there anything else you would like to share? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 137 FGD GUIDE – CSO GROUPS AND ACTIVE AND NON ACTIVE CITIZENS INTRODUCTION AND CONSENT Thank you very much for meeting with us today. My name is ___________ (moderator name) and this is my colleague __________ (assistant name) who will be taking notes throughout our discussion. We work for IMAS, an independent research firm here in Moldova. [Introduce any other observers from the evaluation team] We are undertaking an evaluation for USAID to understand what has worked well and what has not worked well in the Comunitatea Mea project that aims to make local government more effective, transparent, and accountable to citizens. The project is implemented by IREX. IMAS is non-political research firm, and we are not related to any political parties. We are completely neutral on all the issues we’ll be talking about; we’re just here to learn about your experiences. Our discussion should take approximately 90 minutes. Your participation does not involve any risks other than what you would encounter in daily life. Your participation in this study is voluntary. You can choose not to answer a question or leave the discussion at any time without providing a reason. There will be no repercussions for this. However, your participation will be very useful and help USAID design future programs in Moldova. Your privacy will be protected. We are using first names only today and we will not include your name or any identifiable information in any of our reports. Please note that we cannot guarantee full confidentiality because of the group setting, as we cannot ensure that other participants will not disclose any information shared here. However, we ask that what we discuss during our group talk remains here with us. We will be recording this session so we can write an accurate report of what was said. Once we make sure everything was captured correctly, it will be destroyed. It will never be shared with anyone outside of the research team. If you have any questions, you may ask them now or later, even after the discussion has started. If you wish to ask questions later, you may contact Elena Petruti at elena.petruti@imas.md or (Elena’s telephone). [Start the recording to get verbal consent] Do you agree to participate in this discussion today and have the session recorded? Yes No [If participants say “yes,” continue. Dismiss individuals who say “no.” If the entire group does not consent to recording, stop recording and proceed with notetaking only.] GROUND RULES • You don’t have to wait to be called on to talk, please jump in when you have something to say. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 138 • Please talk one at a time. • Our goal is equal “airtime” – so that everybody talks about the same amount. • Say what you believe, even if it’s not what everyone thinks. There are no wrong answers, just different opinions, and we want to hear them all. • Be respectful of others’ opinions • Put your telephones on silent, step out to take phone calls SELF INTRODUCTIONS Before we start, let’s do a quick round of introductions so we all get to know each other. Please introduce yourself to the group and tell us: • First name • A hobby/interest [Ask a volunteer to start and go around the circle, moderator finishes] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 139 A. Quality of and Access to Services Today we will be talking about your local public authority or LPA. By LPA, we mean the mayor, council, and other people elected to serve you. When we ask you about the LPA, we want you think about the entire level of government, not any specific politician. The LPA does many things, including providing public services to citizens. By public services, we mean things like street lighting, garbage collection, local roads, water supply, and libraries. As a citizen, you have the right to these services, regardless of who is in charge politically. In your community, what public services are provided by your LPA? In general, how satisfied or dissatisfied are you with the types of services provided by your LPA? (a) What services should be provided that are not currently provided? What services should be provided more? (b) In your opinion, are there certain groups of citizens that get fewer services than others? Please explain. How satisfied or dissatisfied are you with the quality of public services provided by your LPA? (a) Which services are the highest quality? Please explain (b) Which services are the lowest quality? Please explain. EQ1/2.1.5 Have you noticed any changes in the types of services provided to citizens or the quality of those services over the past [Insert # from Table 1] year(s)? [Ensure both types of services and quality are discussed; as needed, remind participants we are not interested in discussing politicians] (a) If yes, what has changed? What do you think caused those changes? EQ1/2.1.2 Have you noticed any changes in the way your LPA provides administrative services and issues permits over the past [Insert # from Table 1] year(s)? [If needed, provide examples: authorization for construction, business license, certificates of birth/death, marriage procedures, property issues.] (a) If yes, what has changed about how your LPA provides these services? What do you think caused those changes? (b) If no, please share any challenges you may be facing with administrative services in your LPA. B. Citizen Engagement Now I’d like to ask you some questions about your experiences engaging with your LPA. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 140 EQ1/2.2.3 What opportunities are there for you to engage with your LPA to give input or feedback on community priorities? What opportunities are there for you to give input or feedback regarding your satisfaction with public services? EQ1/2.4.5 [Only ask in Straseni, Cimislia, Drochia:] Have you ever used the CISC (Citizen Information Service Center) in your LPA? (a) If yes, what was your experience? How could it be improved? (b) If no, have you seen or heard anything about the CISC? Please explain. EQ1/2.1.6 Have you been directly involved in monitoring the Community Development Project (funded by Comunitatea Mea) in your LPA? [If needed, describe monitoring: providing oversight on quality and whether things are completed on time] (a) If yes, what was your experience? How could it be improved? (b) If no, what have you heard from others who are doing the monitoring? What is your opinion regarding involving citizens in monitoring the Community Development Project (funded by Comunitatea Mea)? Is a good/bad idea? Is it working? EQ1/2.1.4 Did you participate in your LPA’s Municipal Strategic Planning workshop? Please explain your involvement. [If needed, describe the workshop: a meeting where citizens can be involved in planning community priorities for the next 3-5 years] To what extent do you think that feedback from citizens and community initiative groups was incorporated into the LPA’s strategic plan? EQ1/2.2.1 How satisfied or dissatisfied are you with the budgetary process in your LPA? This includes how budgets are set and how money is spent in the community. Please explain. Have you ever participated in the local budgeting process in your community? Please explain your involvement. (a) If yes, do you feel that your suggestions and comments were taken into consideration? Have you noticed any changes in your LPA’s budgeting process over the past [Insert # from Table 1] year(s)? If so, how? How else would you like to see citizens involved in working with the LPA? Please share your ideas for more engagement. C. Decentralization Our next topic is decentralization reform. By decentralization, we mean the authority given by the central government to LPAs to make decisions and spend resources that affect citizens’ lives. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 141 EQ1/2.3.4 [Only ask citizens:] What do you think about decentralization reform? (a) Have your opinions changed over the past [Insert # from Table 1] year(s)? If yes, in what way? [Only ask CSO/community groups:] Has the perception of citizens towards decentralization reform changed over the past [Insert # from Table 1] year(s)? If yes, in what way? D. Conclusion Finally, I have one last question. We’ve talked about public services and ways citizens are involved in working with the LPA. Now we want to hear from you:What else is important to do to make local government work better for you? What should be the priorities? Please explain. (a) In your opinion, what could donors do to support to these priorities? Thank you for your participation. CISC USER SURVEY NORC at the University of Chicago Comunitatea Mea Performance Evaluation User/CISC Exit Survey DRAFT November 29, 2021 District: ____________________________ LPA: ____________________________ Address: ________________________________________________________________________ GPS (lat) ____________________________ (lon) ____________________________ Interviewer: ____________________________ Supervisor: _____________________________ Interview date: ____________________________ Interview start time: ____________________________ Interview end time: ____________________________ SECTION 0: INTRODUCTION 1. Do you work in this institution? 1. Yes (End of interview) 2. No 2. Did you come to submit a request, receive any type of service, or receive any information in this CISC? 1. Yes 2. No (End of interview) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 142 “Hello, my name is ______________. I am conducting a survey for a study financed by USAID. The study would like to know about your experiences receiving information or services from your Primaria. The results of this survey are very important for understanding how to improve these services.” ORAL CONSENT “We would like to invite you to participate in an interview we will be conducting today. Your opinion is very important. We will ask you very general questions about your experiences using the CISC. This would take approximately 10 minutes of your time. Your participation in this study is completely voluntary and does not entail any risk. If you do not want to answer a question you are not required to. You can withdraw your consent and end your participating in this study at any time. Your name will not appear in any report that is written about this study. Any information we collect from you during the interview will be kept in strict confidentiality. Your participation would be much appreciated.” 1. Do you agree to participate in this study? 1. [ ] Yes 2. [ ] No (Thank them and say goodbye; end of the interview) SECTION I: SERVICES RECEIVED BY THE USER 1. [RCTE_RAZ] What was the reason for this visit? [DO NOT READ; MARK ALL THAT APPLY] 1. Submit a petition 2. Request for firewood (heating subsidy) 3. Land ownership 4. Update on business activity 5. Request for daycare services 6. Request for social aid 7. Building permit 8. Property issues 9. Others, specify: ____________ 98. Don’t know 99. No response 2. [RESOL_1] Is coming to the CISC today your first attempt at resolving this issue? 1. Yes [GO TO Section II] 2. No 3. [RESOL_2] When did you first attempt to resolve this issue? 1. Less than 7 days 2. Between 7 and 14 days (1-2 weeks) 3. Between 14 and 21 days (2-3 weeks) CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 143 4. Between 21 and 28 days (3-4 weeks) 5. More than 4 weeks but less than 2 months ago 6. More than 2 months ago 98 Don’t Know 4. [RESOL_3] What else did you try before your visit to the CISC today? [Allow for multiple selections] 1. Searched the webpages of the Primaria [GO TO Section II] 2. Went to the Primaria to try and resolve the issue [GO TO Section II] 3. Called an official [GO TO Section II] 4. Visited this CISC previously [GO TO next question] 5. Other, specify: _____________ 5. [RET] Why have you returned to this CISC? 1. To receive an update on the progress of my previous request 2. To request additional or enhanced services 3. To file a complaint or provide feedback regarding the service I received 4. Because I was directed here by the Primaria 5. Other, specify: ____________ 99. No Response 6. In the past, have you gone to the Primaria (as opposed to the CISC) for a similar issue? 1. Yes (if Yes, complete section II and section III.) 2. No (only complete section II and IV) 98. Don’t know. SECTION II: IMPACT OF SERVICES A 7. [TREATMENT] Now I am going to ask you about the treatment you received. By treatment I am referring to the courtesy of the people who helped you and the willingness of the staff to help. How satisfied are you with the treatment that you received during your visit to the CISC? [SHOW TABLET OPTIONS] 1. Very dissatisfied (go to question 8) 2. Dissatisfied (go to question 8) 3. Neither satisfied nor dissatisfied (skip to question 9) 4. Satisfied (skip to question 9) 5. Very satisfied (skip to question 9) [PROGRAMMER: 97, 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-5] 97. It is still too soon to have an opinion 98. Don’t know 99. No response CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 144 8. Please explain why you are very dissatisfied or dissatisfied with the treatment you received. _________________ 9. [QUAL] In general, how satisfied are you with the knowledgeability of the CISC staff? [SHOW TABLET OPTIONS] 1. Very dissatisfied 2. Dissatisfied 3. Neither satisfied nor dissatisfied 4. Satisfied 5. Very satisfied [PROGRAMMER: 97, 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-5] 97. It is still too soon to have an opinion 98. Don’t know 99. No response A 10. [WAIT] How long did you have to wait to be served? 1. Approximately ________ minutes [PROGRAMMER: 97, 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTION 1] 98. Don’t know 99. No response 11. [GOAL] Now I would like to ask you about the outcome of your visit. Did you achieve the goal of your visit? 1. Yes 2. Still in process [GO TO QUESTION 11] 3. No [GO TO QUESTION 11] [PROGRAMMER: 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-3] 98. Don’t know 99. No response 12. [GOAL_2] Why did you not achieve your goal? 1. The staff were not present 2. The staff did not know how to help me 3. The staff refused to assist me CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 145 4. The problem has to be solved by another entity 5. They asked me for a bribe that I did not want to pay 6. I did not have money to cover the cost of the service 7. I need to submit additional documents/information 8. I need to wait for a response from the relevant authority on my request 9. Other, specify: _______________________ [PROGRAMMER: 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-3] 98. Don’t know 99. No response 13. [EFFORT] Considering the efforts that you made to receive services from the CISC, was it worthwhile to receive these services? 1. Yes 2. No [PROGRAMMER: 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-3] 98. Don’t know 99. No response SECTION III: COMPARE WITH PREVIOUS SERVICE Now I am going to ask you to compare your experience today at the CISC with the experience you had in the past on similar issues at the Primaria. 14. [TREATMENT] Now I am going to ask you about the treatment you received. By treatment I am referring to the courtesy of the people who helped you and the willingness of the staff to help. [SHOW TABLET OPTIONS]. Compared to your past experience, was the treatment your received today: 1. Much better 2. Somewhat better 3. Neither better nor worse 4. Somewhat worse 5. Much worse [PROGRAMMER: 97, 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-5] 98. Don’t know 99. No response 15. [QUAL] Compared to your last visit at the Primaria, were the CISC staff more knowledgeable?: [SHOW TABLET OPTIONS] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 146 1. Much more knowledgeable 2. Somewhat more knowledgeable 3. Neither more nor less knowledgeable 4. Less knowledgeable 5. Much less knowledgeable [PROGRAMMER: 97, 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-5] 98. Don’t know 99. No response 16. [WAIT] Compared to your last visit at the Primaria, did today’s visit take longer or less long? 1. Longer 2. About the same 3. Less time 4. I don’t remember [PROGRAMMER: 97, 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTION 1] 98. Don’t know 99. No response 17. [GOAL] Now I would like to ask you about the outcome of your last visit to the Primaria, not the CISC. Did you achieve the goal of your visit? 1. Yes 2. Still in process [GO TO QUESTION 19] 3. No [GO TO QUESTION 18] [PROGRAMMER: 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-3] 98. Don’t know 99. No response 18. [GOAL_2] Why did you not achieve your goal on your last visit to the Primaria? 10. The staff were not present 11. The staff did not know how to help me 12. The staff refused to assist me 13. The problem had to be solved by another entity 14. They asked me for a bribe that I did not want to pay 15. I did not have money to cover the cost of the service 16. I need to submit additional documents/information 17. I need to wait for a response from the relevant authority on my request CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 147 18. Other, please specify: _______________________ [PROGRAMMER: 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-3] 98. Don’t know 99. No response 19. [EFFORT] Overall, how would you compare your experience at the CISC compared to your previous visit? 1. Much better 2. Better 3. Same 4. Worse 5. Much worse [PROGRAMMER: 98, 99 SHOULD NOT APPEAR ON TABLET; THEY SHOULD ONLY APPEAR IF THE RESPONDENT DID NOT SELECT OPTIONS 1-3] 98. Don’t know 99. No response SECTION IV: CHARACTERISTICS OF SERVICE USERS 20. [AGE] How old are you? ________ 21. [GENDER] Gender [OBSERVE] 1. Male 2. Female 22. [DISAB] [OBSERVE] Please select for if the respondent has the following observable characteristics: 1. Wheelchair 2. Elderly 3. Hearing aid 4. Visual impairment 5. Other, Specify: ________ 6. None END OF THE INTERVIEW CISC OPERATOR QUESTIONNAIRE 1. On average, how much time does it take you to resolve a service request received at the CISC? 2. What are your thoughts regarding processing service requests entirely online via a software compared to via a submitted paper application? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 148 3. Have you had any issues regarding using the new software? 4. Have you received any feedback/suggestions from citizens using the CISC regarding the service quality? 5. Are there other important changes that come to your mind to make the administrative service process more efficient at the CISC? CISC OBSERVATION CHECKLIST NORC at the University of Chicago Comunitatea Mea Performance Evaluation CISC Observation Checklist Final December 7, 2021 Question Observation Notes How many service desks are open with staff behind them? (#) How many service desks are there in total? (#) How many staff are at the CISC when you arrive? (#) Does there seem to be a functional printer? Yes No Does each service windows have a functional computer? Yes No Are citizens required to take a number to get served on entry? Yes No Is there sufficient space for citizens (at least 4 to 6 chairs) to sit while they wait their turn? Yes No Is there any drinking fountain/water available? Yes No Is there a functional restroom available? Yes No Are there clear instructions to citizens posted in plain view in Romanian and Russian? Either on a bulletin board, table or via posters? (Please indicate mode of information under notes) a) Information about accessing local government or central government services on the internet b) Hours of operation / work schedule of the CISC c) Other: _____________ Yes No Are the anti-COVID-19 measures respected (wearing masks, social distancing, availability of disinfectant)? Yes No Is there a payment machine available? Is there a suggestion box or any other clearly visible option for visitors to easily share feedback about the quality of services at the CISC? Yes No Is the building wheelchair accessible? Yes No CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 149 QUESTION BANK The following questions include all questions that were included in individual instruments. These were also used to interview USAID and IREX staff members. EQ # PROTOCOL QUESTIONS EQ1/2.1 In your opinion, how useful is the Functional Organizational Capacity Assessment (FOCAS) tool to identify capacity gaps at the local level? (a) Were there any challenges in implementing the FOCAS within each LPA? (b) Are FOCAS scores used in adaptive management and learning? How? [Ask CM staff, USAID and LPA officials only] (c) How much time does it take and how many people are involved in your LPA to complete the FOCAS tool? [Ask LPA officials only] (d) Do you compare your own score with the scores obtained by other CM LPAs? [Ask LPA officials only] (e) Do you feel that the FOCAS tool will be used by LPAs after the CM project ends? If yes, what are the benefits you foresee and are there procedures in place to ensure the tool is institutionalized? If not, what are the limiting factors? (f) Do you think FOCAS is the right instrument to self-measure the capacities of LPAs and would it be useful to be scaled up at a national level? If so, how? [Ask Donors, GoM and CALM only] EQ1/2.3 What criteria were used to determine the application of specific approaches for each LPA beyond the FOCAS tool? (e.g. CISC, CIPs, IMCSs, CRCs etc.)? (a) Why were some approaches not used for all project beneficiaries? (b) Among the approaches your LPA tried which were less useful? (c) Were some of the first approaches used the best to begin with? (d) Please share with us or list the most useful innovative approaches that you would recommend for implementation in other LPAs and scaled up to the national level. EQ1/2.5 How effective or ineffective has the communication initiative with citizens been in the LPA? (a) What have been the concrete results, and have they been measured? EQ1/2.6 In your opinion has your LPA improved its communication with its citizens over the last 4 years? (a) If no, how could this be improved? (b) If yes, what has changed? (c) If yes, what do you now know about your LPA that you did not know before? EQ1/2.1.1 In your opinion, were administrative services improved in targeted LPAs due to CM? If yes, how? If no, why not? (a) Are there challenges to sustaining administrative improvement once the CM project ends? Please elaborate. EQ1/2.1.2 Has your LPA changed the way it provides administrative services and issues permits as a result of its cooperation with CM? If yes, what changes have been undertaken? If no please share any challenges you may be facing with administrative services in your LPA [FDGs with CSO/Civil groups] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 150 EQ # PROTOCOL QUESTIONS EQ1/2.1.3 How has CM defined "improved governance" in local administrations? Do you feel this has been achieved in target LPAs that have received CM assistance? In your opinion has governance improved in your LPA due to work with CM? Please elaborate. [Use this lead in question only for LPA instrument] (a) Which demand side approaches have been most effective and why? (Ask about: Pune umarul, citizen report cards, work with civil society, etc.) (b) Which supply side approaches have been most effective? (Ask about: communication campaigns, budget hearings, strategic plans etc.) (c) In your opinion which approaches are sustainable and will likely continue after CM ends? EQ1/2.1.4 How effective and useful has it been to start assistance to the LPA by focusing on strategic planning and priorities? (a) In your opinion will strategic plans be regularly updated and implemented once CM ends? (b) To what extent do strategic plans take into account citizen needs in the community? To which extent do the plans take into account gender issues, or the specific needs of different ethnic/linguistic or disabled groups? (c) Did you participate in your LPAs Municipal Strategic Planning workshop? Do you think that feedback from citizens and community initiative groups was incorporated into the LPAs strategic plan? [Ask in FGD with citizens and civic groups] (d) Does your LPAs have sufficient funds to implement your strategic plan? How feasible are the priorities stipulated in the plan? EQ1/2.1.5 Has assistance provided by CM affected citizens' access to services or the quality of services? (a) What services have been affected the most? (b) What approaches have been used and which has been the most effective? (c) Did the quality and access to services change for vulnerable, ethnic/linguistic and disabled population? Please elaborate. (d) Will service improvement approaches continue to be used after CM ends? i.e. have they been institutionalized within the LPA? (e) Have there been any changes in citizen's assess to service or the quality of services in your LPA recently? If yes, what has changed? EQ1/2.1.6 What Community Development projects were implemented with the support of CM? What were the main tools used in this regard? (a) How did citizens' engage in these projects? (b) Have LPAs allocated operation and maintenance costs for these infrastructure projects going forward? (c) How useful is the Community Development Plan to prioritize infrastructure projects in your LPA? Would this approach be useful for other LPAs? (d) Please share any experience your LPA has had with a Capital Improvement Plan (e) Will LPAs continue using these CDPs once CM ends? (f)What has been your experience in monitoring the Community Development Project in your LPA? If you have not been directly involved, what have you heard from others who are doing the monitoring? Or what is your opinion regarding involving citizens in this process? [For citizens and CSOs only] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 151 EQ # PROTOCOL QUESTIONS EQ1/2.2.1 In your opinion is the budgetary process in LPAs receiving assistance from CM more or less open and transparent compared to other LPAs in Moldova? Please elaborate. (a) Have there been any challenges in implementing participatory budgeting? [Note only 2 LPAs have implemented participatory budgeting (civic budgeting)] (b) Has the quality of comments and inputs into the budgetary process from citizens changed in the last few years? What role if any did the CM project play regarding this? [LPA officials only] (c) As a citizen what are your thoughts about the budgetary process followed by your LPA? Are you satisfied or dissatisfied with the process? [FGD with citizens and civic groups only] (d) Have you ever participated in the local budgeting process in your community? If yes, do you feel that your suggestions and comments were taken into consideration? Has this process changed over the last 3-4 years? [FGD with Citizens and CSOs only] EQ1/2.2.2 Are citizens consultations and oversight aligned to provide input into the budgetary calendar? Who sets the dates for these consultations? EQ1/2.2.3 In your opinion do LPAs that are receiving assistance from CM have lower or higher levels of citizen engagement in government? Please elaborate with concrete examples. (a) Do marginalized and vulnerable groups have opportunities to engage with government? If yes, how? If no, why not? (b) Will any of these citizen engagement approaches get institutionalized in the LPA? If yes, which ones? If no, why not? (c) Are there any opportunities for you to engage with your LPA and given input or feedback on local priorities or satisfaction with services? If yes, please elaborate. EQ1/2.2.4 In your opinion which of the several technical approaches being used by CM have been most useful in achieving results within targeted LPAs? (a) Are any of these technical approaches being institutionalized by the LPAs? Which ones? (b) Are there any challenges to institutionalization? EQ1/2.2.5 Has your LPA adopted any new practices during COVID-19 that you would like to continue going forward? Please elaborate. EQ1/2.3.1 Did CM provide the GoM any technical assistance in preparing and passing legislation to address the fragmentation of local government units (i.e., administrative territorial reform)? (a) What have been the biggest challenges in working with the GoM on issues dealing with administrative territorial reform? (b) Were approaches modified to deal with the challenges? (c) Did CM involve other stakeholders or projects to advocate for this reform? Please elaborate. EQ1/2.3.2 What assistance has CM provided on the development of a legislation/institutional framework aimed at enhancing fiscal, administrative and political decentralization in Moldova? (a) What have been the challenges? (Work so far has focused on PEA, waste mgmt tariffs, research on Inter Municipal Cooperation, local taxes analysis, online meeting of local councils, Local Governance Fellowship Program etc.) (a)Which of the regulatory/legislative initiatives that CM has worked on so far with GoM have been the most impactful or least effective? [GoM KII only] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 152 EQ # PROTOCOL QUESTIONS EQ1/2.3.3 Has CM been effective in building the capacity of civil society to have a voice in decentralization policy reform? Please elaborate. (a) if not, what have been the main challenges? (b) What tools/approaches worked or did not work, and why? EQ1/2.3.4 Was CALM's capacity to advocate and represent member LPAs on decentralization reform affected by cooperation with CM? Please elaborate. (a) What activities were especially effective? (b) As a smaller/larger LPA do you think CALM adequately advocates for you to the national government? (c) Has the perception of citizens towards decentralization reform changed over the last 3 years? If yes, in what way? EQ1/2.3.5 Are CSOs in Moldova using sentinel indicators developed by CM to monitor the decentralization reform? (a) Were CSO consulted in the development of the 8 sentinel indicators? (b) Who is responsible for collecting data for these sentinel indicators going forward? EQ1/2.4.1 Has working with CM helped you increase and diversify your own source revenues within your LPA? (a) How did the CM project help you in this area? (b) Which approaches has your LPA used in increasing OSR? (LTFM, LED, RED, Internal Control, etc.) (c) Has working with CM helped you allocate your budget more efficiently? If yes, which approaches helped with this? Were there any approaches that did not work as expected? EQ1/2.4.2 Which of the approaches used by CM to improve LPA own source revenues have been most effective? (a) In your opinion which approach is most likely to continue after the CM program ends? EQ1/2.4.3 Has the cooperation with CM helped LPAs increase their revenues from improved asset management? (a) What have been the challenges in implementing the framework of Public Real Property Asset Management? (b) Where are most LPAs in achieving this framework? (c) In your opinion will LPAs continue with asset management after CM ends? EQ1/2.4.4 How are the internal financial management controls enforced by beneficiary LPAs? (a) What was the contribution of CM with the institutionalization of this instrument? (b) How do you plan to use/enforce the internal financial management controls in the future? [Ask LPA officials in KII] CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 153 EQ # PROTOCOL QUESTIONS EQ1/2.4.5 Has your LPA measured citizens' satisfaction with the CISCs? If yes, what did you learn? If no, why not? (a) Are you planning to collect citizens' feedback in the future? Are the costs for this activity covered by the LPA? (b) Does your LPA have sufficient operations and maintence resources to keep the CISCs open the future? (c) Have you ever used the CISC in your LPA? If yes, what was your experience? 3.1 Do you think CM was able to work effectively across all LPAs in cohort 1, 2, and 3? Do you see any room for improvement? 3.2 Have beneficiary LPAs shared their experience and lessons learned with each other? (a) If yes, how was this done? (b) if no, do you think this will be useful? 3.3 Before starting work with a new cohort were lessons learned from the previous cohort incorporated into implementation? (a) if yes, how was this done? (b) If no, why not? What were the challenges? (c) How did COVID-19 affect work with cohort 3 LPAs? 3.4 What follow-up or monitoring did CM undertaken with previous cohorts after project activities were implemented? 3.5 Were intermunicipal cooperation projects (IMC) accepted and encouraged? Where? (a) If IMC applications where not allowed, why? (b) How did IMC results compare to results seen in single applicant LPAs? (c) What were the strengths and challenges of this approach? 3.6 Has CM disseminated the results of the project beyond the targeted beneficiary LPAs? 3.7 Is CMs organizational structure and HR sufficient or insufficient to implement the project design? (a) Would you do anything differently? 4.1 In your opinion, (a) What else can USAID do to help LPAs achieve good local governance that was not addressed so far or has been addressed in a limited manner (b) Which are the areas that USAID has sufficiently supported and can phase out of (c) which other areas with a potential to generate development outcomes should USAID explore and support to strengthen LGs in Moldova? (d) Are there specific activities that were not adequately funded? If yes, which ones? 4.2 Was there coordination and cooperation with other donors and donor projects during CM implementation? (a) Was there any successful cases of complementarity/synchronization with other projects that maximized the project impact? (b) Is there scope for more cooperation and coordination? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 154 EQ # PROTOCOL QUESTIONS 4.3 How do you think the results achieved in individual beneficiary LPAs would be affected by the implementation of administrative territorial reform? 4.4 In your opinion, what part(s) of CM will continue following the end of the project? CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 155 ANNEX F – SOURCES OF INFORMATION DOCUMENTS REVIEWED The following table presents document reviewed in detail that were determined to be the most useful and informative for answering evaluation questions. Other documents provided by IREX were reviewed, but were used for providing context for instrument development, not for answering evaluation questions. DOCUMENT TYPE DOCUMENT NAME Annual Report FY 2018 Annual Report "CM_Annual Report_FY2018" Annual Report FY 2019 Annual Report "CM_72011718CA00002_Annual Report_FY2019_Final.pdf" Annual Report FY 2020 Annual Report "CM_72011718CA00002_Annual Report_FY2020_FINAL_SUBMISSION" Annual Report FY 2021 Annual Report Quarterly Report Quarterly Report Apr 2021-June 2021 Quarterly Report Quarterly Report Oct 2021-Dec 2021 Presentation Annex #32 FOCAS Analysis Presentation Indicators Annex_35_CM_FY20_Performance Indicators Data CM Component 4 Revenue Generation data MEL Plan CM MEL Plan REVISED Nov 2021 Excel Data FOCAS Data and Comparative Analysis_Cohort 1 Excel Data FOCAS Data and Comparative Analysis_Cohort 2 Excel Data FOCAS Data and Comparative Analysis_Cohort 3 Excel Data Population Survey- Cohort 1 Follow-up_Comparative Analysis 2019- 2021_IREX Excel Data Cohort 2 Baseline_Survey Results Analysis_IREX Excel Data Cohort 3 Baseline_Survey Results Analysis_IREX Annex Annex #34_List of Current CM Sub-Grants CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 156 DOCUMENT TYPE DOCUMENT NAME Annex Annex #01_CM_CDP_Infographic_October 2021 Annex Annex #02_Links to CDS Annex Annex #04_CM IMC Guidelines Annex Annex #18_Budget and revenue comparative analysis. All partner LPAs Annex Annex #19_Report on LED support to 5 LPAs from Cohort II Annex Annex #28_Revenue from asset management Annex Annex #33_GESI Assessment Summary Annex Annex #09_Draft law on the administrative-territorial organization of Moldova Annex Annex #10_Draft law on the administrative-territorial reform Annex Annex #11_Draft law on legal amendments Annex Annex #27_List of Approved Acts on IMCS development Annex Annex #14_LPA Functional Assessment Annex Annex #13_Draft Methodology on Waste Management Tariffs Annex Annex #15_LPA Costing Study Report Annex Annex #16_Policy recommendations on the system of local taxes Annex Annex #17_Communication Campaign Concept Annex Annex #20_Investment Offer of Donduseni LPA Annex Annex #23_List of LPAs public financial data published on social networks and websites Annex Annex #24_List of LPA Partners References on Component 4 activities Annex Annex #25_List of approved Budget Calendars Annex Annex #22_Guidelines on GESI responsive budgeting integration in budget cycles CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 157 DOCUMENT TYPE DOCUMENT NAME Annex Annex #26_Recommendations on Institutional development and Reorganization of the Public Procurement Process in Chisinau LPA Annex Annex #29_List of approved Acts on Asset Management Excel Data Annex #35_Budget and Pipeline Expenditure Analysis Annex Annex #12_Framework regulation on special fees Annex Annex # 32_Communications _ Media Mentions Folder with several annexes Annex #03_CIP Policy Frameworks Folder with several annexes Annex #06_ CISC_Operations Manuals Folder with several annexes Annex #07_Social Programs_Budgets Folder with several annexes Annex #08_SIAP User Guides Folder with several annexes Annex #21_Concepts of Local Tourism Development for 2022-2024 Folder with several annexes Annex #30_Asset Management Plans Annex Annex #31_Registry of Public Assets KEY INFORMANTS No. NAME GENDER19 STAKEHOLDER TYPE POSITION KII01 Daniel Serban M CM Staff Chief of Party, IREX KII02 Cristine Grecu F CM Staff Deputy Chief of Party, IREX KII03 Igor Mironiuc M CM Staff Component 2 Lead, IREX KII04 Stela Alexei F CM Staff Component 4 Lead, IREX KII05 Andrei Cantemir M CM Staff Component 1 Lead, IREX KII06 Ludmila Ungureanu F CM Staff Component 3 Lead, IREX KII07 Igor Spivacenco M USAID Project Management Specialist, Monitoring and Evaluation 19 “M” indicates a male respondent, and “F” indicates a female respondent. In total, there were 30 male and 29 female respondents. CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 158 No. NAME GENDER19 STAKEHOLDER TYPE POSITION KII08 Diana Cazacu F USAID Project Management Specialist KII09 Victoria Gellis F USAID Office Director KII10 Diana Toma F Implementing partners Ma Implic Project, SKAT Moldova KII11 Viorel Pana, Dorel Noroc, Natalia Sclearuc and Ludmila Popa 2M + 2F GoM Ministry of Finance KII12 Andrian Ermurachi M GoM State Chancellery KII13 Lucia Casap and Constantin Rusu 1M + 1F Donors World Bank KII14 Alexandru Pelivan M Implementing partners UNDP KII15 Victoria Cujba F GoM State Chancellery KII16 Valentina Plesca F Implementing partners GiZ KII17 Dumitru Budianschi M GoM Ministry of Finance KII18 Zaporojan Veaceslav M LPA Official Mayor KII19 Ludmila Goncear F LPA Official Secretary of local council KII20 Marina Didenco F LPA Official Accountant KII21 Cociorva Serghei M LPA Official Cadastral Engineer KII22 Radu Danii M Donors SDC KII23 Ion Manole M Implementing partners Promolex KII24 Maria Ovdii F GoM Agency for Land Relations and Cadaster KII25 Onorina Soric F Donors Former project officer at CoE KII26 Marina Scutaru F LPA Official Contact person (focal point), involved in project implementation for the LPA KII27 Svetlana Bolocan F GoM Ministry of Environment CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 159 No. NAME GENDER19 STAKEHOLDER TYPE POSITION KII28 Dorin Andros M GoM Former state secretary of the Ministry for Agriculture, Regional Development and Environment KII29 Tomasz Horbowski M Implementing partners Solidarity Fund Leader Program KII30 Zinaida Adam F Donors UNDP central office/former MIDL project manager KII31 Alexandru Ambros M LPA Official Mayor KII32 Pavliuc Tatiana F LPA Official Chief accountant KII33 Ina Olearciuc F LPA Official Public Relations Specialist KII34 Tatiana Zabulica F LPA Official Senior specialist KII35 Marcel Busan M LPA Official Secretary of local council KII36 Calmatui Silvia M LPA Official Chief accountant KII37 Lilia Pilipetchi F LPA Official Mayor KII38 Ghenadie Minascurta M LPA Official Deputy Mayor KII39 Chas Cadwell M IREX Subcontractor Urban Institute KII40 Liliana Tincu F LPA Official A former civil servant in Ungheni Municipality, actually local coordinator in EU4Moldova project KII41 Denis Ternovschi M LPA Official Deputy Mayor KII42 Svetlana Ciobanu F LPA Official Head of Public Institution of Ungheni Municipality dealing with strategic planning, fundraising (applying for projects) KII43 Fiodor Topciu M LPA Official Mayor KII44 Fiodor Enac M LPA Official Deputy Mayor KII45 Anna Constandoglo F LPA Official Chief accountant KII46 Natalia Constandoglo F LPA Official Specialist on investments KII47 Mihail Stamov M LPA Official Deputy Mayor KII48 Anatolii Topal M LPA Official Mayor CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 160 No. NAME GENDER19 STAKEHOLDER TYPE POSITION KII49 Nicolai Dișli M LPA Official Focal point KII50 Maros Krama M Donors SlovakAid KII51 Serghei Labliuc M LPA Official Mayor KII52 Galina Cebotari F LPA Official Chief accountant KII53 Ion Voica M LPA Official Cadastral Engineer KII54 Raisa Cotorobai F LPA Official Head of internal audit department KII55 Timur Tsusuk F IREX Subcontractor Tetra Tech home office project manager PARTICPATING LPAS – CATI Cohort 1: 1 Slobozia Mare 2 Onişcani 3 Ursoaia 4 Slobozia-Dusca 5 Gribova 6 Obreja Veche 7 Costești 8 Ialoveni 9 Borogani 10 Ciorești 11 Bilicenii Vechi 12 Feștelița 13 Talmaza 14 Lozova 15 Strășeni 16 Valea Perjei 17 Comrat Cohort 2: 18 Varnița 19 Colibași 20 Crihana Veche 21 Taraclia 22 Budești 23 Cimișlia 24 Selemet 25 Drochia CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 161 26 Coșnița 27 Doroțcaia 28 Edineț 29 Sărata Veche 30 Ciuciuleni 31 Răzeni 32 Ruseștii Noi 33 Lalova 34 Mihăileni 35 Ștefan Vodă 36 Pănășești 37 Pîrliţa 38 Sculeni 39 UTAG Cohort 3: 40 Cobusca Veche 41 Geamăna 42 Hîrbovăț 43 Larga 44 Călărași 45 Sipoteni 46 Căinari 47 Bubuieci 48 Sîngera 49 Gura Galbenei 50 Izbiște 51 Dondușeni 52 Pelinia 53 Vărvăreuca 54 Ciuciulea 55 Cărpineni 56 Mingir 57 Tigheci 58 Peresecina 59 Dumbrăvița 60 Sîngerei 61 Sîngereii Noi 62 Sireți 63 Zubrești 64 Inești 65 Verejeni 66 Copceac CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 162 ANNEX G – DISCLOSURE OF CONFLICTS OF INTEREST Name Ritu Nayyar-Stone Title Team Lead Organization NORC Evaluation Position? ☒ Team Leader ☐ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / AID-OAA-M-13-00013, Tasking N055 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Comunitatea Mea, IREX Agreement No. 72011718CA00002 I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 4/8/2022 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 163 Name Katharine Mark Title Local Governance Evaluator Organization NORC at the University of Chicago Evaluation Position? Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / AID-OAA-M-13- 00013, Tasking N055 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Comunitatea Mea, IREX Agreement No. 72011718CA00002 I have real or potential conflicts of interest to disclose. Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date April 14, 2022 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 164 Name Ion Beschieru Title Decentralization and Local Service Delivery Expert Organization NORC Evaluation Position? Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / AID-OAA-M-13- 00013, Tasking N055 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Comunitatea Mea, IREX Agreement No. 72011718CA00002 I have real or potential conflicts of interest to disclose. □ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 10/04/2022 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 165 Name Adrian Lupusor Title Local Government Finance Expert Organization NORC and Expert-Grup Evaluation Position? ☐ Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / AID-OAA-M-13-00013, Tasking N055 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Comunitatea Mea, IREX Agreement No. 72011718CA00002 I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date May 2, 2022 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 166 Name Anna Solovyeva Title Data analyst Organization NORC Evaluation Position? ☐ Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / AID-OAA-M-13-00013, Tasking N055 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Comunitatea Mea, IREX Agreement No. 72011718CA00002 I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 4/8/2022 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 167 Name Jessica Wallach Title Qualitative Specialist Organization NORC at the University of Chicago Evaluation Position? Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M /AID-OAA-M-13- 00013,Tasking N055 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Comunitatea Mea, IREX Agreement No. 72011718CA00002 I have real or potential conflicts of interest to disclose. □ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1.Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2.Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3.Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4.Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5.Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6.Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 04/11/2022 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 168 Name Samantha Downey Title Quantitative Specialist Organization NORC at the University of Chicago Evaluation Position? Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / AID-OAA-M-13- 00013, Tasking N055 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Comunitatea Mea, IREX Agreement No. 72011718CA00002 I have real or potential conflicts of interest to disclose. Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 4/18/22 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 169 Name Samantha Austin Title Evaluation Support Organization NORC Evaluation Position? Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / AID-OAA-M-13- 00013, Tasking N055 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Comunitatea Mea, IREX Agreement No. 72011718CA00002 I have real or potential conflicts of interest to disclose. Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 4/11/2022 CONTRACT NO. GS-10F-0033M / ORDER NO. 7200AA18M00016 / DRG-LER II TASKING N055 USAID.GOV COMUNITATEA MEA MID-TERM PERFORMANCE EVALUATION | 170 Name Tulay Akoglu Title Qualitative Support Organization NORC at the University of Chicago Evaluation Position? ☐ Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) GS-10F-0033M / AID-OAA-M-13-00013, Tasking N055 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Comunitatea Mea, IREX Agreement No. 72011718CA00002 I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 4/18/2022 U.S. Agency for International Development/Moldova 57/1, Banulescu-Bodoni Street ASITO Building, 5th Floor Chisinau, Moldova 2005