GEORGIA GOOD GOVERNANCE INITIATIVE ACTIVITY Mid-Term Performance Evaluation Report NANA PIRVELI FOR USAID This document was produced for review by the United States Agency for International Development, Democracy, Human Rights and Governance Center under the Learning, Evaluation and Research Activity II (LER II) contract: GS10F0218U/7200AA18M00017. Prepared by: The Cloudburst Group 8400 Corporate Drive, Suite 550 Landover, MD 20785-2238 Tel: 301-918-440 USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 1 CONTENTS ACRONYMS 2 EXECUTIVE SUMMARY 1 EVALUATION PURPOSE AND QUESTIONS 5 PROJECT BACKGROUND AND CONTEXT 6 EVALUATION METHODS AND LIMITATIONS 7 EQ 1. DESIGN—SCOPE AND SCALE 9 EQ 2. CONTRIBUTION AND EFFICIENCY 15 EQ 3. CAPACITY-BUILDING 23 EQ 4. DONOR COORDINATION 27 ANNEX A. KII LIST 30 ANNEX B. DESK REPORT 34 USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 2 ACRONYMS CoP Community of Practice CSB Civil Service Bureau CSO Civil society organization EQ Evaluation question ET Evaluation team GESI Gender equality and social inclusion GGI Good Governance Initiative GIA Gender Impact Assessment GIZ Deutsche Gesellschaft für Internationale Zasammmenarbeit GoG Government of Georgia IDFI Institute for Development of Freedom of Information IPU Inter-Parliamentary Union IRC Innovations and Reforms Center IRI International Republican Institute M&E Monitoring and Evaluation MP Members of Parliament MRDI Ministry of Regional Development and Infrastructure NAP National Action Plan NDI National Democratic Institute OGP Open Government Partnership PEFA Public Expenditure and Financial Accountability PDO Public Defender’s Office PFM Public Financial Management RIA Regulatory Impact Assessments SAO State Audit Office SoW Scope of work TIG Transparency International Georgia TOC Theory of change ToT Training of trainers UNDP United Nations Development Programme USAID United States Agency for International Development WVI World Vision International USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 1 EXECUTIVE SUMMARY BACKGROUND AND CONTEXT This document presents the midterm evaluation of the Georgia Good Governance Initiative (GGI), a project funded by the United States Agency for International Development (USAID) Mission in Georgia and implemented by Tetra Tech. The performance period is from 2015 to 2022. The overall objective of the evaluation was to 1) examine the appropriateness of the activity’s design of the development context, 2) assess its efficiency and effectiveness in advancing key reforms and select interventions, and 3) provide recommendations on reform topics and technical approaches that USAID should continue to support in the future. The GGI was designed to support democratic governance reforms introduced by a new government that assumed power in 2012. The new government was a coalition of six parties led by the Georgia Dream Party. The political commitments of the coalition in the party included public sector reform, decentralization, improvement of political and civic rights, universal access to health services, gender equality and social inclusion (GESI) of minority groups, and continued advances on Georgia’s Euro-Atlantic trajectory. Soon after assuming power, the Government of Georgia (GoG) drafted a 2020 National Development Strategy and, in 2013, a new constitution entered into force that involved moving from the presidential system of government to a mixed parliamentarian system and adopting the two-year National Action Plan (NAP). It is within this context that the GGI aimed to enhance the transparency, accountability, and effectiveness of the GoG through strengthening the demand and supply side of governance. The GGI focused on the following: improved administrative and financial management systems of public institutions at all levels; increased openness of government through the improvement of mechanisms to ensure transparency and accountability; strengthened policy development and lawmaking processes through engaging with government agencies and Parliament for evidence-based decision-making; and enhanced institutional oversight of government involving not only Parliament but also other state checks and balances institutions, such as the State Audit Office (SAO). EVALUATION PURPOSE As per the scope of work (SoW), the present evaluation aims to assess the performance of the GGI in the last six years of activity implementation. Concretely, the study looks into design and scope, efficiency and effectiveness, capacity-building, and levels of donor coordination. The analysis also includes the provision of key recommendations for future programming in each of the areas assessed. METHODOLOGY The methodology used in the present evaluation combines the use of quantitative and qualitative information and data from secondary sources of information (e.g., annual reports, capacity needs assessment, etc.) gathered through in-depth desk review with qualitative primary data collected through 60 semi-structured interviews. The information collected was then processed according to the indicators set per each of the evaluation questions (EQs), triangulated, and analyzed for draft reporting. FINDINGS EQ 1—Design and Scope. GGI was appropriately designed given the development context. The Evaluation Team (ET) found that the decision to focus on the supply side (government USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 2 institutions) with a flexible and adaptable implementation approach resulted in the timely delivery of support that was mostly demand-driven. This approach was welcomed by most government counterparts who gained a sense of ownership over the project’s objectives. The ET did not find that inclusion of stakeholders from different branches of government resulted in any unintended consequences; however, the ET did note that the supply side (government) received more funding and technical support while the demand side (civil society) received less support, thereby reducing their engagement in policy development and oversight in the reform areas targeted by the activity. Regarding the theory of change (TOC), the ET found that it remains valid. This evaluation found that at least 26 different legal or public reforms have been enacted with direct contributions from GGI and its civil society organization (CSO) partner efforts. Nevertheless, the TOC presented some challenges specifically linking the results to the initial activity design. EQ 1 Recommendations include: ● Perform an in-depth analysis of political impediments to achieving results, allowing greater focus on efforts in areas where progress is feasible. ● Refine sub-results or add more specific and easier to measure sub-results in line with the indicators measured in the Monitoring and Evaluation (M&E) plan (e.g., Complexity-Aware Monitoring). ● Further incorporate CSOs into the new project design through the co-creation process and continue efforts to co-design and be adaptive with GoG. ● Carry out GESI analysis at the design level of the project and develop a GESI strategy to ensure GESI mainstreaming across all project activities, M&E, and reporting into future project designs. EQ 2—Efficiency and Effectiveness. GGI’s tailored technical assistance contributed to the government’s efforts to deliver on some of its reform agenda in civil service reform, open governance, local government decentralization reform, and public oversight. For instance, GGI provided training and expert advice to the Civil Service Bureau (CSB) to implement civil service reform, provided consultants to assess the capacity of service delivery competencies and municipal statistics development skills in municipalities; and funded inter-municipal learning and cooperation. GGI’s support also stimulated debate on decentralization policy among key stakeholders. The major contribution of the project in achieving the expected results was the implementation of the Open Government Partnership (OGP). GGI was instrumental in supporting the implementation of the OGP Action Plan 2015–2020. Direct support to local government allowed them to launch inclusive public budgeting processes to demonstrate greater levels of responsiveness to citizens. In these and other ways, GGI was effective in achieving results anticipated by the contract. Regarding the extent to which the level of effort was commensurate with the results, the ET found the project was considered by interviewees to be efficient in mobilizing resources and delivering activities, as it could accommodate government requests and deliver them on short notice. These activities were considered by the interviewed organizations necessary to push for the reforms. In the absence of GGI, these activities may have been undertaken by the government or by another development partner. But, most likely, they would not have taken place. Nonetheless, it is difficult to assess the level of effort compared to the results, due to results being described in a broad and general manner that is difficult to be measured. The general description of the diversity of reforms supported also made it difficult to measure the efficacy investments in each of the areas. EQ 2 Recommendations include: USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 3 ● Reduce the scope of institutional partners (e.g., fewer institutions targeted, or areas of demand-driven support—the support provided to OGP is a good example of focus) while keeping a demand-driven approach (e.g., support to decentralization or oversight institutions). This would reduce the atomization of activities and increase the possibility to leverage efforts and resources for better results. ● Always integrate a component of the supply and demand side of governance and their actors within the focus chosen (e.g., decentralization or specific sector). ● Support for oversight institutions should extend beyond SAO and target other actors such as the ombudsman and other organizations channeling citizen complaints about the state’s action. EQ 3—Capacity-Building. Capacity-building remains at the core of GGI as more than half of the activities were devoted to developing the skills and knowledge of GoG staff. The GGI project carried out a capacity needs assessment at the beginning of the project, after which specific trainings were planned annually in collaboration with the beneficiary institutions. The majority of capacity development activities addressed the needs of the GoG’s central institutions—administration of the GoG/CSB, the Parliament, and the municipalities—and much less so for line ministries and other oversight institutions. This confirms the project’s deliberate prioritization of particular stakeholders to achieve the expected results. The impact of the investment in oversight institutions (parliament, SAO, PDO, and the State Inspector’s Service) was difficult to discern. This is because the project is still being implemented and outcome level results were not identified in the documentation or in the key informant interviews. Nonetheless, the stakeholders interviewed expressed their appreciation of the USAID’s interventions. However, the ET observed that the result areas that received more training (e.g., Result 1) have not necessarily delivered more results or showed more impact in terms of reforms. In terms of approach and tools, GGI made use of a wide range of capacity-building tools such as coaching, traditional training workshops, “learning by doing”/on-the-job training, building in￾house training capacity of partners agencies (i.e., parliament, public audit institute, and SAO), experience sharing, and fellowship programs. These were highly appreciated by stakeholders because they supported capacity development and incentivized beneficiaries to participate in their planning, which fostered ownership. The majority of the trainings were conducted by Georgians using the Georgian language and cultural references, which made the engagement of targeted institutions more relevant and effective. The project also built new activities upon previously supported activities, where strong commitment already existed among beneficiaries. The reputation of USAID, coupled with the GGI team’s extraordinary professionalism, was also considered a huge factor when exerting pressure over authorities to fulfill their commitments. In other cases, GGI supported the fulfillment of the GoG’s international commitments, such as in the area of OGP and anti-corruption. GGI ensured the institutionalization of new products and services. For instance, all municipalities where GGI provided support for participatory budgeting were allocated annual funding to ensure certain continuity in the short-/medium-term with the aim to embed the practices within the local institutions. Yet, other contextual factors played against the sustainability of capacity-building efforts. High turnover of staff and insufficient budgets are acknowledged constraints within Georgian institutions that undermine the sustainability of any capacity development activity. EQ 3 Recommendations include: ● Integrate a capacity-building approach in the design of the activity that responds to the needs of the context and the specific institutions targeted. It is also advised that the design of the activity include measures of the sustainability of capacity-building activities, learning modules, and, if possible, the exit strategy so that capacity-building activities are embedded within the beneficiary institutions. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 4 ● Establish a system to measure participants’ learning performance and quality of the capacity-building activities to be applied consistently. The ET found that reporting was mainly focused on outputs in line with indicators. Learning on capacity-building practices and knowledge was rather limited, contributing further to the atomization of impact. ● Introduce collaborative, learning, and adapting techniques together with Thinking and Working Politically through an applied political economy approach to implementation should be prioritized to ensure the project continues to seize political “windows of opportunity” and pivot rapidly. EQ 4—Donor Coordination. GGI effectively coordinated with other donors active in central and local governance to synergize its investments. A good example was when GGI, the United Nations Development Programme (UNDP), and the Inter-Parliamentary Union (IPU) supported Parliament’s efforts to institutionalize a gender self-assessment toolkit. As a result, the toolkit is now on Parliament’s website and it will serve as a guide for future strategic and institutional development planning. Another example is when GGI—along with the International Republican Institute (IRI), National Democratic Institute (NDI), World Vision International (WVI), and UNDP—supported the Regional Policy and Local Self Government Committee initiative in an “Institutional Dialogue.” This general forum served as a platform for communication between Parliament and central and local governments to discuss their communication strategies and action plans. The GoG’s governance-related donor coordination mechanisms are mainly related to OGP coordination meetings, for which GGI served as an informal and auxiliary secretariat to the government. However, for non-OGP coordination, respondents indicated that the level of government-driven donor coordination appears to be inefficient and is often politicized. This contributed to some overlapping of donor activities during the first years of project implementation. For example, one respondent shared a case when GGI wanted to do the evaluation and assessment of MRDI’s participatory practices, but MRDI stated that Deutsche Gesellschaft für Internationale Zasammmenarbeit (GIZ) was scheduled to conduct this activity, which was not accurate. EQ 4 Recommendations include: ● Conduct a brief mapping exercise to understand where the activity will need to ensure donor coordination for future projects/activities. ● Make more concerted efforts to connect with key donors and GoG institutions to “check in” on coordination efforts and learn if the activity could increase coordination efforts in the short or long term. ● Invest in strengthening the GoG donor coordination mechanism by exploring incentives that may encourage coordination or barriers that may prevent it. EVALUATION PURPOSE AND QUESTIONS 1.1 EVALUATION PURPOSE The present document presents a performance assessment of the midterm evaluation of the GGI, a project funded by the USAID Mission in Georgia and implemented by TetraTech. The project was initiated in 2015 and was initially expected to end in 2020. In 2019, the project was extended for 18 additional months, from January 2020 to July 2021. A third extension up to April 2022 was approved to face the consequences and manage governance challenges caused by Covid-19 outbreak. According to the SoW (see Annex I), the overall objective of the evaluation was to 1) examine the appropriateness of the activity’s design of the development context, 2) assess its efficiency USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 5 and effectiveness in advancing key reforms and select interventions, and 3) provide recommendations on reform topics and technical approaches that USAID should continue to support in the future. The evaluation was conducted by a team of four experts: Gemma Piñol Puig (team leader), Marc Cassidy (governance expert), and Natia Nadiradze and Eteri Kamarauli (local experts). This report includes a description of the project background, a presentation of the evaluation approach and data collection tools, and an assessment of the performance of the GGI project. 1.2 EVALUATION QUESTIONS The SoW included 14 specific EQs to assess the design, efficiency, effectiveness, and sustainability of the project, as well as its capacity-building approach. There was no specific question related to GESI. Nevertheless, the team of experts considers that GESI aspects are integrated into specific EQs under EQ 1. Concretely, the EQs are: EQ 1 (Design—Scope and Scale): 1.1 To what extent was the activity appropriately designed to achieve its expected results given the development context? 1.2 To what extent did the inclusion of beneficiaries with competing interests—the executive (national and local), legislature, oversight bodies, and civil society—help or hinder the design? 1.3 To what extent was the budget allocation for grants to civil society sufficient to ensure their fulsome engagement in policy development and oversight in the reform areas targeted by the activity? 1.4 Are there any gaps in the TOC or pieces that could have been dropped? EQ 2 (Contribution and Efficiency): 2.1 How effective has the partner been in achieving results anticipated by the contract? 2.2 Considering the major reform areas targeted by the activity—civil service reform, open governance, local government decentralization reform, and public oversight (including Parliamentary and state audit strengthening, and citizen engagement)—to what extent has GGI contributed to progress in these areas? 2.3 To what extent was the level of effort commensurate with the result? 2.4 Are their specific areas recommended for future programming? EQ 3 (Capacity-Building): 3.1 To what extent has GGI identified and applied the most effective capacity-building approaches/tools to deliver and sustain results in the key reform areas? 3.2 Which capacity-building approaches/tools have resulted in sustainable capacity increases for key actors and/or processes, which have not, and why? 3.3 Were there common contextual factors that affected the sustainability of capacity-building results? EQ 4 (Donor Coordination): USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 6 4.1 To what extent has GGI effectively coordinated with other donors active in central and local governance to enhance its performance? 4.2 Were there any specific cases of donor coordination/cooperation that directly assisted GGI in delivering results? If so, what approaches were used? 4.3 Were there other ways/approaches for donor cooperation that could have brought better results and would be recommended for future programming? PROJECT BACKGROUND AND CONTEXT The year 2012 marked an important change in the political landscape of Georgia. The long￾time hegemony of the United National Movement party was replaced by a coalition of six parties led by the Georgia Dream Party. This change in the Georgian government was seen as a great opportunity to introduce policy reforms by national and international actors. The new executive showed the capacity to impose a vision for the future of the country and a willingness to carry out important reforms in terms of democratic governance. The political commitments of the coalition in the party included public sector reform, decentralization, improvement of political and civic rights, universal access to health services, GESI for minority groups, and continued advances on Georgia’s Euro-Atlantic trajectory. The executive quickly drafted a 2020 National Development Strategy, and a new constitution entered into force in 2013 that involved moving from the presidential system of government to a mixed parliamentarian system and adopting the two-year NAP. It is within this context that the GGI project was designed to support these important reforms. The project was created as part of the Transparent and Accountable Governance project to “strengthen the overall transparency, accountability, and effectiveness of governance in Georgia.” Concretely, GGI is intended to do this by strengthening the demand and supply side of governance. GGI focused on the following: ● Result 1: Improved administrative system and financial management systems of public institutions at all levels. This includes the public administration system, local and national public financial management (PFM) systems, and budgeting processes. ● Result 2: Increased openness of government through the improvement of mechanisms to ensure transparency and accountability, including the participation of citizens as part of the checks and balances of government actions. ● Result 3: Strengthened policy development and lawmaking process, engaging with government agencies and Parliament for evidence-based decisions. ● Result 4: Enhanced institutional oversight of government involving not only Parliament but also other state checks and balances institutions, such as the SAO and the ombudsman. The TOC of the GGI project is underpinned by the USAID Strategy on Democracy, Human Rights, and Governance (2013). There, democratic governance is key to ensuring sustainable social and economic development and requires the existence of accountable, legitimate, and effective state institutions. This is based on the assumption that democratic or good governance occurs when the state has the capacity, legitimacy, and authority to deliver public services, ensure the rule of law, and act on behalf of the public good. Accountable governments also tend to be more inclusive and equitable. Thus, the project targeted all the institutions that play a key role on the supply side of accountability, such as the central services in charge of administrative and financial management systems and oversight institutions (mainly Parliament and the SAO). The project was designed as part of the Transparent and Accountable Governance project, a holistic systems-oriented approach aimed to enhance transparent and accountable state institutions through supporting both demand and supply USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 7 side of accountability. Gender and social inclusion are considered cross-cutting issues to be mainstreamed across the project implementation. The project had an initial implementation period of five years but was extended twice to address emerging and governance issues. The first extension took place in December 2019 in the run-up to the 2020 elections. At this time, it was considered important to extend the project to ensure the sustainability of the project achievements, the continuity of the priority, and ongoing reforms in the political agenda in the Parliament constituted in the aftermath of the elections. In November 2020, GGI was extended a second time to face the challenges posed by the COVID-19 pandemic in terms of democratic governance. The immediate impact of the pandemic was in the slowdown of project implementation and even the cancellation of some activities. While the GoG quickly provided an adequate, effective, and efficient response to the health emergency, including access to basic social and educational services for the most vulnerable populations, it also adopted several restrictions of movement and public gatherings and made unpopular decisions, putting under threat a relatively open civic space, civil and political rights, and the integrity and transparency of public institutions. This created the necessity to continue supporting democratic governance in a situation of “democratic emergency.” Hence, the GGI project was extended for an additional nine months and will end in April 2022. EVALUATION METHODS AND LIMITATIONS 3.1 EVALUATION METHODS The midterm evaluation was prepared principally using contribution analysis based on an examination of mainly qualitative primary and secondary data. The secondary data were collected through an in-depth desk review. Several working documents were designed to review each of the documents included in the literature survey as per the indicators provided in the evaluation matrix. The information collected through these grids was then systematized and processed according to each EQ. The pre-preliminary findings of the desk review were presented in a desk report and submitted to USAID. The desk review also served to rebuild the result chain of the project based on factual contributions of the project to specific national and local legal and policy reforms. This has been done both by tracking all the project activities within the frame they took place and by mapping the specific reforms to which they were intended to contribute. This reconstruction of the resulting change helped to address efficiency- and effectiveness-related EQs. The primary data were collected through semi-structured interviews. A total of 68 people from 50 organizations were interviewed: 18 staff from 14 central and local government administration; 9 from Parliament, including staff and members of Parliament (MPs); 7 from oversight institutions; 21 people from 13 CSOs; 9 from development partners, and 8 from TetraTech and USAID.1 Further, the level of engagement of beneficiaries in this evaluation is proportionate to the actors that have mostly benefited by the project, being the GoG, closely followed by CSOs and Parliament. Therefore, the collection of primary data can be considered balanced and inclusive. 1 See Annex 1 KII List USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 8 The information collected in the semi-structured interviews was reported in field grids, processed per evaluation matrix indicators, and then analyzed per the EQ. Once the desk and field research were finalized, then the information collected was jointly analyzed by contrasting the findings and identifying the common trends. 3.2 LIMITATIONS The team of experts faced three main constraints due to access to certain information and documents (e.g., needs analysis), more fundamental issues related to the project’s internal M&E and GESI, and the fact that the project was still ongoing at the time of the present evaluation. In addition, due to COVID-19-related travel restrictions, the international team members were unable to travel to Georgia, which required greater reliance on remote interviews. The team was able to conduct remote interviews and Georgia-based team members conducted in-person interviews where possible. GGI WILL END IN APRIL 2022 The midterm performance evaluation took place while the project was ongoing. This means that the ET was not able to capture the impact of all the activities delivered by the GGI project, especially those related to the challenges posed by COVID-19, the purpose for which the project was extended in November 2020. QUALITY OF DOCUMENTS AND INFORMATION A thorough review of the documents provided revealed that certain data and information constraints limited what could be studied and to which level of specificity. The annual reports provided information related to the delivery process of the activities in great detail. On some occasions, it was observed that this information was repetitive across the years, making it difficult to understand when an achievement happened. These reports also included an M&E framework providing information about the achievements in a given year against the targets. However, the way this was structured with the text made it difficult to relate the text to the table. Furthermore, the project did not include a baseline, but it was defined based on the existing information which was not the same for all the existing indicators. The annual targets were adjusted per the PMEP, although the final targets remained the same. This made it difficult to track progress. Finally, the annual reports and the rest of the documents analyzed 23% 22% 15% 12% 15% 13% Level of engagement per organization Central and local public administration CSOs Parliament Oversight Institutions Development Partners USAID and TT USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 9 provided little information related to GESI and, when it did exist, it was poor and insufficient to assess GESI mainstreaming in the GGI project. EQ 1. DESIGN—SCOPE AND SCALE This EQ aims to assess the quality and adequacy of the design of the action and the scope. This involves assessing whether the action was designed taking into account context issues and whether it involved the appropriate and necessary stakeholders to identify their needs and capacities. It also requests to look into the more technical aspects of the project and evaluate the quality of the TOC and result chain, including a description of indicators and results. FINDINGS 1.1 TO WHAT EXTENT WAS THE ACTIVITY APPROPRIATELY DESIGNED TO ACHIEVE ITS EXPECTED RESULTS GIVEN THE DEVELOPMENT CONTEXT? GGI was designed to strengthen Georgia’s national and subnational governance system, by focusing on the central and local governments and—to a lesser degree—Parliament, other state oversight institutions, and non-state oversight institutions (CSOs). The evaluators found the program was designed appropriately for the context, and that the demand-driven design allowed it to address and contribute to key ongoing PAR and parliamentary reforms. The GGI activity provided demand-driven assistance to selected municipalities. Respondents from municipalities shared that they were directly involved in the design of activities for their respective organizations. One respondent stated that in their municipality, Batumi, both municipality and GGI staff initiated activities regarding service delivery strategy and action plans. The activities also received support from the Innovations and Reforms Center (IRC) to help Batumi municipality with open data processing, visualization, and presentation of the selected services. Kutaisi Municipality and GGI staff met once a year to discuss the municipalities’ needs and develop activities. Possibly, because of these design efforts by GGI in partnership with the municipalities, the activity achieved outcomes that fall under Results 1, 2, and 3. These outcomes included: Batumi Public Service Delivery Strategy and Action Plan; Building Integrity and Transparency Strategy and Action Plans for Tbilisi, Telavi, Lagodekhi, Senaki, Zugdidi, and Ozurgeti; and Rules and Regulation on the Use and Development of the Kutaisi City Municipality Territory. GGI has supported the MRDI with communication/web support, trainings on research methodology, convening local governments to address water policy management issues, and many other issues. Respondents from MRDI said that “every intervention was initially planned with the department and designed together.” As a result of this co-design effort, several outcomes and results have been achieved during the activity. GGI provided technical assistance for incorporating key commitments of MRDI’s Building Integrity and Transparency Action Plan into the National Anti-Corruption Action Plan, the development of the National Decentralization and Good Governance Strategy, external website redesign support related to improving communication with the public, and providing training in research methods for MRDI staff. The GGI activity has provided significant support to Georgia’s CSB to implement civil service reform and several other projects over the life of the activity. Respondents within the CSB said the Bureau was directly and actively involved in the design of activities from the beginning. A procedure was developed where CSB shared their needs with GGI once a year when GGI was drafting its action plan. Additionally, GGI was able to answer ‘ad hoc requests’ proposed by the CSB with GGI. One respondent mentioned the project is mostly supporting CSB to implement those activities. The respondents went on to further say that all the relevant actors participated in design efforts effectively. As a result of these co-designed and demand-driven USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 10 activities, GGI also supported the CSB in conducting a personal data protection audit in select areas, creating training modules for CSB staff, holding a training of trainers (ToT) event for managers on functional analysis in public organizations, and performing functional analyses of legal entities of public law related to the Public Administration Reform Roadmap. All CSB staff interviewed for this evaluation said that expert support on research methodologies from GGI was helpful. As a result, they are better able to conduct research independently and present findings on their own. For example, one of the three CSB staff interviewed said that the skills she learned are used in everyday actions and the quality of her work has increased. Respondents in the Parliament of Georgia said Parliament, including specific committees when needed, was directly involved in the design of the activities. Parliament engaged in planning and design with GGI by sharing its needs. There are several outcomes that could be connected to co-design efforts by GGI and the GoG. One such outcome is related to Regulatory Impact Assessments (RIA), GGI conducted a series of capacity-building activities for Parliament to address this gap and help prepare its partner committees and other relevant parliamentary units. These committees included: Sectoral Economy and Economic Policy, Healthcare and Social Issues, and Environment and Natural Resources. The objective of GGI’s RIA-related work with Parliament was to build parliamentary staff and elected members' capacity in RIA methodology. GESI, mainly of youth and minority groups, were considered cross-cutting issues in the SoW. No evidence was found of gender and social inclusion analysis at the beginning of the project. Through the present assessment, it was difficult to track GESI mainstreaming throughout the project. Most of the GESI activities delivered by the GGI are gender-sensitive activities such as the inclusion of gender perspective in housing policy or carrying out gender-sensitive analyses at the Parliament level. However, these are not sufficient to consider that GESI was mainstreamed across the whole project. No evidence of GESI measures applied to project implementation was found. For example, gender budgeting could have been included in participatory budgeting, some GESI measures could have been included in the trainings, and data could have been provided in a disaggregated manner when possible and relevant (e.g., number of participants in trainings or participatory mechanism) in a more consistent manner that it was provided. In fact, GGI reporting could be considered gender blind as GESI activities are vaguely reported and information was not provided in a GESI-sensitive manner. 1.1 TO WHAT EXTENT DID THE INCLUSION OF BENEFICIARIES WITH COMPETING INTERESTS—THE EXECUTIVE (NATIONAL AND LOCAL), LEGISLATURE, OVERSIGHT BODIES, AND CIVIL SOCIETY—HELP OR HINDER THE DESIGN? GGI was designed based on consultations with a large number of stakeholders in two phases. The first phase involved more general consultations as part of the appraisal process. There, all stakeholders were involved, including main project beneficiaries (GoG, municipalities, Parliament and SAO), CSOs, and development partners. But when it came to the specific design process of the project, the second phase, mostly the government was consulted. Parliament, local authorities, and other oversight institutions were consulted to a lesser extent compared to the central government in the specific design of the project. Several respondents’ perception was that all relevant state institutions were involved and overall efforts by those participants were relevant and positive. There was limited consultation in this second phase with CSOs and other donors. CSO respondents either said they could not speak to the design process, further illustrating their lack of involvement in this second and more specific phase, or shared that they were not a part of the design process. This was because the GGI project was initially conceived as an activity to support the supply side of governance complementary to another activity aimed at supporting the demand side of accountability under the same project, and, thus, targeting more specifically CSOs. In the second phase, development partners were not as much USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 11 engaged as in the first phase, which probably could not help to avoid some of the overlapping issues encountered during the first months of the activity implementation. The enabling and disabling factors that affected the project design process included the numerous recipient organizations, dynamics of competing interests among the different beneficiaries and different interest between the government departments, high turnover of the staff, and a six-year implementation period without a mid-term review. These factors make it difficult for the ET to determine the overall impact on good governance. GGI’s demand-driven approach to technical assistance considered the needs of the GoG stakeholders and adapted to changes in the Georgian political landscape. In fact, the project was designed in a way that could align to ongoing and/or new processes leading to a highly flexible and adaptable approach to support democratic governance processes. GGI hired several local experts to help address specific needs for support requested from the GoG. These were mainly ad hoc requests not foreseen in the original project design but considered relevant and adequate to be addressed through the GGI project, illustrating the responsive and adaptive nature of the activity. For example, GGI was asked by Parliament and the Gender Equity Council to support Gender Impact Assessment (GIA) efforts to help mainstream gender into the early stages of policymaking. GGI successfully fulfilled this request, which included capacity-building, piloting, presentation and discussion, and discussion of lessons learned with the target GoG institution. Other donors that supported this effort include NDI and UNDP’s Gender Program. 1.2 TO WHAT EXTENT WAS THE BUDGET ALLOCATION FOR GRANTS TO CIVIL SOCIETY SUFFICIENT TO ENSURE THEIR FULSOME ENGAGEMENT IN POLICY DEVELOPMENT AND OVERSIGHT IN THE REFORM AREAS TARGETED BY THE ACTIVITY? As mentioned above, the level of engagement with CSOs was rather weak at a design level and continued to remain weak during the implementation of the GGI. A total of 15 grants over the last six years totaling just over $1 million USD were granted to local CSOs. There were nine grantees with multiple grantees receiving more than one grant. Many of the CSO respondents discussed their involvement with GGI in terms of their grant awards and SoW and less in terms of co-design and collaboration. However, these CSOs also shared their positive experiences working with GGI as a “comfortable” or “excellent” partner. Several of the CSOs implemented projects aimed at engaging in policy development or oversight. Based on the reports, it appears CSOs received grants or sub-contracts due to the limited resources available for grants. Regarding CSO engagement in policy development and oversight, several CSOs led or supported activities addressing these issues. With GGI funding, Transparency International Georgia supported efforts by GGI in 2017 and 2018 to enhance Parliament’s transparency and accountability through targeted impact analysis and increased civil participation related to the Open Parliament Partnership, and the creation of and release of legal analysis reports that allowed interested stakeholders to learn about drafting law contents. This could have enabled more participation in the lawmaking process. Ten legislative analyses were published to deliver an analytical overview and recommendations for proposed legislation related to human rights, legal issues, defense, budget and finance, sector economy, healthcare, and the environment. The CSO also authored several reports related to Parliamentary functions and its action plan. GGI also funded the development of MRDI’s Transparency and Anti-Corruption Framework, which was included in the OGP NAP. Through a fixed amount grant, the Institute for Development of Freedom of Information (IDFI) conducted independent research and interviewed stakeholders to develop a situational analysis on existing mechanisms of anti￾corruption and potential threats of corruption at the MRDI and its subordinate institutions. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 12 Based on the situational analysis, IDFI—through close engagement with the MRDI and GGI— developed a draft anti-corruption strategy, action plan, and list of performance indicators to be used to measure the implementation of selected activities. As mentioned previously, the IRC supported oversight efforts at the municipal level through their grant from GGI. These activities further illustrate oversight efforts at different levels of government related to Result 4. These CSO-led projects were strongly devoted to supporting the government and were meant to reinforce the supply side of governance but not strengthen CSOs’ or citizens’ demand-side capacity for democratic governance as the initial objective of engaging the CSOs in the GGI (as this was included in another activity under the same project.) In fact, very few grants were designed to enhance civic participation in decision-making and/or social accountability initiatives, and no grants for CSOs or for other oversight institutions such as the ombudsman were targeted to support the demand side of accountability, as it was not envisioned in the initial activity design. This limitation would be confirmed by the analysis of the CSOs projects that focused on providing capacity-building services to the government (supply side of accountability) rather than actors to enhance CSOs and the general public as drivers of democratic governance (demand side of accountability). However, once incorporated the activity issued several grants aimed at civic participation. Finally, as mentioned previously, the total funding for CSOs was approximately $1 million USD. This amount divided among 15 grants makes it a challenge to map specific outcomes and track impact related to pushing for demand-driven governance reforms and bottom-up accountability. Projects were of very small size to deliver specific tools or activities complementary to those provided by the GGI and not sufficiently large and robust to track changes in terms of the supply side of governance. Furthermore, most of these grants were aimed at delivering activities to reinforce the demand side of governance, as service providers of the government, and not to enhance the supply side of governance by linking CSOs’ roles in engaging citizens in using the information made available through the GGI project. 1.3 ARE THERE ANY GAPS IN THE TOC, OR ARE THERE PIECES THAT COULD HAVE BEEN DROPPED? IF administrative and financial management of public institutions is improved; IF openness of government is increased; IF policy development and lawmaking processes are strengthened; and IF institutional oversight of government is strengthened, THEN transparency, accountability, and effectiveness of governance in Georgia will be strengthened.—GGI TOC Overall, the TOC still seems valid. This evaluation found that at least 26 different legal or public reforms have been enacted with direct contributions from GGI and its CSO partner efforts. These results, or contribution to results, illustrate that the TOC is still appropriate. When asked, respondents did not feel any aspects should be dropped and only shared what other activities should be included or scaled up as the activity continues. Activities that respondents suggested for future programming include helping to establish a new platform for cooperation, strengthening core CSO funding, and providing further support on decentralization efforts. Nevertheless, the TOC presented some challenges in terms of result chain and measurement as discussed above. The ET faced challenges specifically linking the results to the initial activity design. Descriptions of expected results are broad and general. This, combined with a simple results framework with results broadly defined, made it difficult to establish a consistent and coherent result chain. The ET recognizes that it can be a challenge to separate results that aim to strengthen a complex system. Despite the dearth of linear causal evidence, the ET noted that the project’s activities contributed directly or indirectly to the different results as it was very difficult to associate specific activities to specific results (e.g., evidence-based research capacities serve for policy-making as well as oversight). The association of activities to the different results was also caused by the fact that the areas of intervention are close; USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 13 overall, those related to R2, R3, and R4 and beneficiaries are even overlapping (e.g., R3 and R4, where the Parliament is targeted for lawmaking and oversight). Each of the sub-results could be a result area, leaving a kind of “missing middle” between the results achieved by the outputs and the impact results established by the GGI project. Measurement challenges were also noticed in the M&E framework that mixed the monitoring of delivery of outputs (e.g., “Number of laws…”; “Number of draft legislative, and policy documents…”; “Number of more effective/improved oversight tools…”) with monitoring of outcomes (e.g. “Number of GoG agencies that demonstrate improved efficiency…” or “Degree to which government implements the OGP action plan”). However, monitoring of results in the short/medium term (outcomes) was not done in a consistent manner and across all the results. Only R1 and R2 included some indicators measuring performance in the short/medium term such as ‘2.1 Number of people participating in new or improved government mechanisms for citizen consultation (Output/Custom)’ (short term) or ‘1.1 Percent of GOG agencies that demonstrate improved efficiency at the organizational and human capacity level’ (medium term). Additionally, the fact that all the indicators were quantitative made it very difficult to assess the quality of the results obtained. FIGURE 1: GGI THEORY OF CHANGE In terms of the result chain, the ET faced challenges specifically linking the results to the initial activity design. Descriptions of expected results are broad and general. This, combined with a simple results framework with results broadly defined, made it difficult to establish a consistent and coherent result chain. The ET recognizes that it can be a challenge to separate results that aim to strengthen a complex system. Despite the dearth of linear causal evidence, the ET noted that the project’s activities contributed directly or indirectly to the different results, as it was very difficult to associate specific activities to specific results (e.g., evidence-based research capacities serve for policy-making as well as oversight). The association of activities to the different results was also caused by the fact that the areas of intervention are similar; USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 14 overall, the activities related to R2, R3, and R4 and beneficiaries are even overlapping (e.g., R3 and R4, where the Parliament is targeted for lawmaking and oversight). RECOMMENDATIONS ● The ET recommends USAID and Tetra Tech carefully analyze the factors shared by respondents regarding impediments to achieving results. USAID’s guidance on Thinking and Working Politically Through Applied Political Economy Analysis2 can be used by project staff to systemically analyze the drivers and barriers for change. By dropping activities that lack political support from the government, GGI might be able to focus more of its efforts in areas where progress is feasible. Impediments to achieving results are mentioned in the following sections. ● Refine sub-results or add more specific sub-results to be more direct and easier to measure in line with the indicators measured in the M&E plan. This may include the use of progress markers and journals—expect to see, like to see, love to see, etc.— or other policy influencing M&E methods, like USAID’s Complexity-Aware Monitoring, that could better illustrate incremental progress, challenges to achieving results, and contribution in complex systems. ● Incorporate CSOs more into the new project design through the co-creation process of activities and emphasizing citizen-driven governance needs. ● Continue efforts to co-design and be adaptive with GoG to support their identified needs within the scope of the activity. ● Carry out GESI analysis at the design level of the project and develop a GESI strategy to ensure GESI mainstreaming across all project activities, M&E, and reporting into future project designs. EQ 2. CONTRIBUTION AND EFFICIENCY Contribution and efficiency EQs are intended to assess the project implementation approach, including the capacity of the implementing partner to achieve the expected results of the project. The questions also explore areas that would require additional support in the near future. FINDINGS 2.1 HOW EFFECTIVE HAS THE PARTNER BEEN IN ACHIEVING THE RESULTS ANTICIPATED BY THE CONTRACT? The GGI project was implemented by TetraTech, an organization with long-standing experience in the field governance sector and, therefore, in the areas targeted by the project. The GGI team is composed mainly of local staff with support from Tetra Tech’s Home Office and consultants. The ET observed that the project was implemented based on a “demand￾driven” approach that consisted of providing support based on the needs of partner organizations that were identified in the short and medium terms. This approach was highly appreciated by the interviewees because of the flexible nature of the assistance. The project was implemented using a mix of a wide range of activities such as training, technical assistance for studies, reviews, awareness-raising campaigns, coordination, facilitation of government and citizen interactions and dialogues, and the provision of grant funding when necessary. 2 USAID TWP/PEA 2018. https://usaidlearninglab.org/library/thinking-and-working-politically-twp￾through-applied-political-economy-analysis-pea-guide USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 15 Respondents mentioned that the GGI approach was considered an added value of the GGI project compared to the projects supported by other development partners. Most other donor projects tend to be implemented using a more programmatic, linear, and rigid approach that does not address immediate bottlenecks of policy reforms that are often related to processes and that tends to extend the time and effort to endorse reforms. The approach was also seen as adapting to the specific political issues and changes within the targeted institutions. Some of the people interviewed acknowledged that sometimes it was not easy to engage with the institutions targeted under Result Area 1 and Result Area 3 (e.g., departments dealing with human resources, financial management, or parliamentary committees) due to the lack of ownership over these processes. The demand-driven approach allowed GGI to penetrate those institutions and facilitate their engagement despite political issues. In these cases, GGI was able to adapt to the context by engaging on an informal basis, including the delivery of activities in informal settings and in non-politically sensitive areas or themes. The participants gained a greater understanding and ownership, leading to important changes in key public administrative services such as administrative and human resource systems. GGI’s support to Parliament bolstered their confidence to demand a minister to testify in front of the MPs and report on the progress or challenges of a given policy. This, of course, required significant effort and was highly time-consuming. GGI staff engaged with beneficiaries and dealt with reform processes for both developing activity plans every year and responding to unplanned demands. But at the same time, GGI’s flexibility combined with the capacity to work formally and informally led to the delivery of planned and ad hoc activities that contributed to the project results in different levels and areas. For the administrative and financial management of public institutions, the major improvements were highlighted in terms of increased capacity to produce financial information at the local level through carrying out Public Finance Management (PFM) and PEFA assessments and to increase the transparency of the budgets of the local authorities through introducing participatory budgeting. In terms of capacities, skills, and the knowledge of government, the staff was improved, leading to progress in their professional careers. The GGI also supported enhancing the institutional public management performance by assisting in a review of the public civil service legal framework and in the setting up information technologies system for improving the functional civil service and human resource management systems. In terms of strengthened policy development, lawmaking, and enhanced institutional oversight of government, the most important achievement in these areas were the capacities developed within the Parliament, mainly at the individual and process levels and the adoption of the RIA of legislative acts by the Parliament (CSOs Meter). The committees received external technical expertise that strengthened committee members’ understanding of how to promote their interests in accordance with the rules of procedures. A Community of Practice (CoP) of national RIA practitioners was created by the GoG. GGI collaborated with several other donors/projects, including USAID’s Governing for Growth, on trainings and workshops. A GGI consultant supported Kutaisi City Hall to conduct an RIA of social programs funded through the municipal budget. GGI also supported a series of capacity-building activities for Parliament to address skill and competency gaps in RIA. As a result, parliamentary committees created RIA working groups and GGI consultants mentored these working groups to prepare RIAs on several draft laws. The adoption of RIA, including the institutionalization of monitoring requires that the government presents a RIA when initiating the specific draft laws, which ensures that the law is based on the analysis of Ex-Ante assessment of the possible impact of the law and, thus, supports a more transparent and informed decision-making process. The law on RIA further connected GGI co-design and support toward Result 4. Besides the RIA, the GGI also provided evidence-based studies for decision-making processes to the government, such as a study that supported the department in defining the USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 16 policy for the high-mountain regions and the technical expertise for policy drafting. At the local level, decision-making processes were also supported through, for example, the provision of support to design spatial planning-related policies and to carry out participatory budgeting and other participatory processes that could favor opening up spaces for citizen participation and oversight. In terms of evidence-based decision making, GGI also worked with partner parliamentary committees to identify specific pieces of legislation requiring expertise that could not be provided by committee staff or through stakeholder consultations and identified and provided such expertise through short-term experts. One respondent stated, “At the beginning of each year, the staff of the Healthcare and Social Issues committee met to identify the needs and required actions and, later, the necessities were communicated with GGI. After the discussion and agreement on the work in selected areas, GGI was involved through expert support or technical assistance.” One example of this support was RIA technical assistance, including capacity-building and coaching related to the draft Labor Safety law. Finally, GGI supported partner committees in organizing consultations on key draft legislation discussed in Parliament, including roundtable discussions with key stakeholders and public discussions of draft legislation in the capital and regions of Georgia. For example, GGI experts supported the Environment and Natural Resources Committee related to the Green Lines legislation, which aims to protect green spaces in self-governing cities. This law has yet to be formally adopted at the time of the present evaluation. Nevertheless, the major contribution of the project in achieving the expected results was the implementation of the OGP. The GGI was instrumental in supporting the implementation of the OGP Action Plan 2015–2020. In fact, it was the only project providing support to OGP at implementation and at the coordination level, which facilitated the coordination and consultation meetings among all the OGP stakeholders. Within this context, the GGI supported the involvement of the municipalities that led to increasing the implementation of OGP commitments in five municipalities in the OGP Action Plan 2016–2017 to 11 municipalities that were included in the OGP Action Plan 2018–2019. The OGP Action Plan also led to the participation of CSOs in the OGP process and opened up different spaces for dialogue with the GoG. Besides the OGP, the project delivered a series of outputs (e.g., the creation of websites and public information of new policies) that have enhanced the availability of information necessary for state and non-state oversight actors and citizens to hold the government accountable. Despite the project's contributions to the achievement of the expected results, the GGI approach presented certain trade-offs in terms of effectiveness, due in part to its continuous adaptation to changing context and government priorities. The demand-driven approach served to reach as many beneficiaries as possible and allowed project alignment with the ongoing national policy process and the political agenda, but a downside of the approach was that some activities occurred on a rather ad hoc basis in many sectors, from civil service reform to health, urban development, and water projects. By following a policy design process that was not framed in a broader change process (e.g., water sector reform), which likely led to the atomized achievement of the expected results. This could be mainly attributed to changing priorities of government, overall, in the last three years of project implementation characterized by political instability, elections and Covid-19 global pandemic. Complementarity among the different activities within the sector was within the approach of the implementing partner but it did not occur in all cases. For example, an activity aimed at providing an evidence base for a policy (e.g., a study) was not often linked to a learning process (e.g., training and involvement in the research) to enhance the capacities of the beneficiary on how to carry out this research or at least be involved in the research process to understand the requirements for the policy research. This was the case for the support provided to the CSB, which received support from a consultant to assist them in applying USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 17 research methods along with training on research methods for the CSB staff. However, this trend was not observed across all the project implementations. Building on previous activities was often constrained by the large amount and often disparate priorities of the government. Based on the findings from the interviews and desk research, this may have been a missed opportunity to leverage efforts and resources to achieve higher impact. Finally, the quality of the sub results did not help to accurately assess this EQ. The definition of the sub-area results remained too broad. Each of these sub-area results could be a result area. The indicators included in the M&E plan could help to improve the definition of these sub-area results and understand the specific unit to be measured. However, the indicators used are to measure the GGI impact at a broader level or at the output level, creating a “missing middle” in measuring these (intermediate) sub results, as discussed in EQ 1. 2.2 CONSIDERING THE MAJOR REFORM AREAS TARGETED BY THE ACTIVITY—CIVIL SERVICE REFORM, OPEN GOVERNANCE, LOCAL GOVERNMENT DECENTRALIZATION REFORM, AND PUBLIC OVERSIGHT (INCLUDING PARLIAMENTARY AND STATE AUDIT STRENGTHENING AND CITIZEN ENGAGEMENT)—TO WHAT EXTENT HAS THE GGI CONTRIBUTED TO PROGRESS IN THESE AREAS? Based on the analysis of information collected in the M&E framework and through the reconstruction of the project result chain, applying a contribution analysis of the project outputs to the actual reforms implemented in the last six years, it can be concluded that the project's main contributions were in the area of open governance (R2). The GGI played a key role in supporting and scaling up the implementation of the OGP Action Plans designed in these five years. In fact, the GGI could be seen as providing technical assistance to the OGP implementation and, therefore, as a major contributor to the achievement of the results at the local and national levels as reported year by year by the Independent Reporting Mechanism of the OGP Action Plans.3 The relevance and impact of the GGI for the OGP Action Plans can be confirmed by the information provided by the M&E data, where the level of implementation of the OGP action plan increases as the GGI is implemented across the years. Before the intervention, the level of implementation of the OGP was about 25 percent, and it increased up to 48 percent of the implementation in 2019. Although the levels of implementation of the OGP Action Plan were not steady during the project implementation period, the percentages were always higher during project intervention than before. TABLE 1: GGI PERFORMANCE MONITORING TABLE WITH ANNUAL PROGRESS RESULTS PERFORMANCE INDICATOR UNIT BASE￾LINE RESULT Y1 RESULTY2 RESULTY3 RESULTY4 RESULTY5 2.5. Degree to which GoG implements the OGP action plan (Outcome/Context) Percent (%) based on progress 25% 30% 66% 30% 64% 48% SOURCE: GGI ANNUAL REPORT 2020 Following OGP implementation, the GGI was key in “strengthening [the] policy development and lawmaking process” as it supported the drafting process of a large number of laws and policies,4 including the Anti-corruption Strategy (2017–2020) and Action Plan, the anti￾corruption policy, and the Building Integrity and Transparency Strategy and Action Plan. As result, Georgia improved their budget transparency score from 67 to 78 points largely due the online budget monitoring platform tools that monitors government expenditure as noted by the International Budget Partnership, an independent American NGO. Although a decentralization process was not directly targeted by the project, its objectives were considered as part of a broader public governance reform. In this sense, some evidence was observed to conclude that the GGI supported and even, at times, revitalized the local governance and decentralization reform. The GGI was especially important in scaling up the OGP process to the local level. As per the M&E data, the GGI supported about 30 government 3 Independent Reporting Mechanism 2016, 2017, 2018 4 See Figure 3, Results Chain. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 18 agencies and local authorities in carrying out evidence-based participatory budgeting and which supported the implementation of the Local Self-Government Code. According to CSOs meter 20205, participatory budgeting has been gradually gaining interest at both local and national level. In fact, some principles and standards of civic budgeting were developed by CSOs and the Ministry of Regional Development and Infrastructures. It also strengthened the relations between citizens and local public authorities, which is typically almost nonexistent in a highly centralized context such as Georgia. This was achieved through supporting civic consultations, participatory budgeting processes and increasing access to information by creating websites for the local governments. Although fiscal competencies have not been fully transferred to the municipalities yet, PFM assessments are considered key to supporting it and making this information available for accountability purposes. PFM assessments through PEFA indicators were key to address the gaps observed in the PFM assessment, including the development of specific recommendations and activities. The GGI further tried to support top-down and bottom-up processes to enhance decentralization by supporting the development of evidence-based policies and strategies at central and local levels. Besides scaling up the implementation of the OGP at the local level, the project provided expertise for the development of the MRDI Anti-Corruption Policy. It further supported 11 local authorities in developing policy documents and plans in different sectors, from integrity and transparency to urban development and the introduction of participatory budget mechanisms. Nevertheless, it was noted that the contribution of the GGI to decentralization reform in terms of oversight and civil service reform was rather limited, most probably because support to these areas mainly targeted central government agencies. The overall contribution of the GGI to the civil service reform area was in terms of organizational and human capacity. According to the M&E data, the percentage of GoG agencies that demonstrated improved efficiency during the implementation period was more than 80 percent in 2017 and 2018, reaching 100 percent in 2019. About 25 agencies demonstrated improved organizational and human capacities. In terms of reforms, the GGI contributed to the review of a number of laws and policies related to civil service reform, such as a law on civil service, a law on the remuneration of civil servants, and amendments to the labor code “on the elimination of all forms of discrimination.” TABLE 2: GGI PERFORMANCE MONITORING TABLE WITH ANNUAL PROGRESS RESULTS PERFORMANCE INDICATOR UNIT BASE￾LINE RESULT Y1 RESULT Y2 RESULT Y3 RESULT Y4 RESULT Y5 1.1 Percent of GoG agencies that demonstrated improved efficiency at the organizational and human capacity levels (Outcome/Custom) Percent 25.90% 70% 82.80% 86.40% 100% 86.40% GoG Agency (cumulative) SOURCE: GGI ANNUAL REPORT 2020 Public oversight could be considered the weakest area of the GGI project, as little progress in terms of reforms could be tracked. The support provided was mainly focused on strengthening processes and the overall institutional performance of internal oversight mechanisms such public internal control departments, which report to the Ministry of Finance and focused less on external oversight institutions, such as the SAO or CSOs.6 The project approach in this area is confirmed by the M&E framework, which tracked the delivery of an important number of oversight tools and mechanisms delivered by the GGI to oversight institutions, like the SAO's ARIS software, which monitors the number of audit recommendations implemented. 5 ECNL (2020) ‘CSO Meter. Assessing the civil society environment in the Eastern Partnership countries. Georgia Country update’, Tbilisi, 2020 6 That was an institution targeted by R3 of the project. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 19 Despite this monitoring an improvement of the government in implementing audit recommendations from 43% in 2019 to 47% in 2020, there has not been improved accountability or related specific reforms. This could be attributed two main issues. On one hand, the main partner and, thus, focus of the GGI project was the Government of Georgia and, therefore, this was much more active in requesting support, leaving less time and resources to the rest of the institutions. On the other hand, it has to do with the capacity of the institutions targeted to absorb the support provided by the GGI, being the reasons behind the lack of capacity, different levels of approval or lack of ownership over the ongoing reforms. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 20 FIGURE 2: THE RESULT CHAIN Type of activities delivered Expected outcomes Expected results Impact: Contribution to legal and public reforms ● Technical assistance/Provisio n of expertise ● AD hoc studies ● Capacity development activities: trainings, mentoring, etc. ● Facilitation of public awareness activities ● Civic participation and engagement in reforms ● Provision of funding and administrative assistance to a nation-wide public awareness and educational program Outcome 1.1: Internal systems and processes for long-term strategic planning and performance management improved. Outcome 1.2: Strengthened local and national-level PFM systems, including performance-based budgeting. Outcome 1.3: Functional civil service and human resource management systems introduced. Result One: Improved administrative and financial management of public institutions at all levels Direct contribution: ● Law on Civil Service ● Civil service reform and secondary law related to performance appraisal in civil service, ranking system, leading to a more fair and independent selection of the staff ● The Law on Remuneration of Civil Servants ● Amendments to the labor code “on the elimination of all forms of discrimination.” ● Batumi Public Service Delivery Strategy and Action Plan Complementary contribution: ● Public Administration Reform Roadmap ● Draft Law on Civil Service Outcome 2.1: Functional government mechanisms for ensuring transparency and access to public information strengthened. Outcome 2.2: Improved national and local government mechanisms and processes for citizen consultation. Result Two: Increased openness of government Direct contribution: ● OGP Action Plans implemented ● Anti-Corruption Policy of the MRDI ● Anti-Corruptions Strategy (2017– 2020) and an Action Plan ● Building Integrity and Transparency Strategy and Action Plan ● Communication Strategy and Communication Action Plan ● Expansion of the OGP to local level, including the realization of PFM and PEFA assessments at local level and the transparency at local level ● National Anti-Corruption Strategy and Action plan of Georgia for 2019–2020 ● Building Integrity and Transparency Strategy and Action Plan at local level: Tbilisi, Telavi, Lagodekhi, Senaki, Zugdidi, Ozurgeti ● Increasing interest and use of participatory budgeting Outcome 3.1: Internal and inter-agency systems for evidence-based policy development and lawmaking will be established, including mechanisms for civic participation. Outcome 3.2: Development of select policies and laws supported. Result Three: Strengthened policy development and lawmaking processes Direct contribution: ● Water Policy Development and Reform ● State Policy on Population Aging ● Continuing Education Policy Development in Healthcare Sector ● Rules of Regulation on Use and Development of the Kutaisi City Municipality Territory ● Spatial urban development in Kutaisi ● Housing Policy ● Law on Excise Fees and Law on Traffic Regulation ● Decentralization Strategy ● The adoption of RIA, which requires the design of evidence based public policies. Complementary contribution: ● Law to the Government of Autonomous Republic of Adjara ● Organic Law “On Normative Acts” ● Law on Medicine and Pharmaceutical Activities Outcome 4.1: Strengthened capacity of key parliamentary committees to provide oversight of line ministries’ and state agencies’ performance and spending, including related to the Result Four: Strengthened institutional oversight of government Direct contribution: ● SAO Communications Policy Outline to the Local Authorities ● Rules of Procedure, which introduced significant changes to the legislative USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 21 decentralization and public health sectors. Outcome 4.2: Strengthened government internal control mechanisms and systems. Outcome 4.3: Improved systems, tools, and capacity of the central government to monitor the performance of local governments based on objective and measurable criteria. Parliament Research Center in the form of a legal entity of public law ● Certifiedauditors ● An increase number of addressed audit recommendations by the government (by the SAO's ARIS software monitoring system) USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 22 In line with the high levels of OGP implementation, the GoG improved their budget transparency per the Open Budget Index, from scoring 55 (with “limited information available”) in 2014, before the project started, to 81 in 2019 (with “extensive information available"). Improvements in macro indicators in terms of open government were also observed, growing from 0.48 before the intervention to 0.57 by 2019. However, more worldwide governance indices (e.g., Freedom House’s Nations in Transit Local and National Democratic Governance Ratings and the Governance Index and Governance Performance from Bertelsmann Transformation Index) remained steady across all the GGI implementation years. TABLE 3: GLOBAL GOVERNANCE INDICATORS GLOBAL GOVERNANCE INDICATOR UNIT BASELINE Y1 Y2 Y3 Y4 Y5 Y6 1.Freedom House’s Nations in Transit Local and National Democratic Governance Ratings (Impact/Context) Score, Composite Local 5.50 5.25 5.25 5.25 5.25 5.25 5.25 Score, Composite National 5.50 5.5 5.5 5.5 5.5 5.5 5.5 2. Governance Index Score, Country Level 5.8 n/a 5.7 n/a 5.9 n/a 5.9 3.The Open Budget Index (Impact/Context) Score, Country Level 55 n/a 66 n/a 82 n/a 81 4.The Open Government Index (Impact/Context) Score, Country Level 0.48 0.61 0.63 0.57 0.57 0.59 0.57 SOURCES: BERTELSMANN TRANSFORMATION INDEX GOVERNANCE INDEX; OPEN BUDGET INDEX; OPEN GOVERNMENT INDEX; GGI ANNUAL REPORT 2020 The mixed impact of the GGI at the macro level can be due to different factors that cannot be totally attributed to the GGI performance. Besides the contextual factors (e.g., political instability and COVID-19), the evaluation experts may have acted too early to assess the contribution of the GGI to the overall governance performance indicators. On the one hand, the policies, laws, and mechanisms supported by the GGI were enforced and implemented at the time of the present evaluation which made it difficult to observe any impact over those macro indicators. On the other hand, ensuring the implementation of these policies and laws was out of the scope of the GGI. Rather, they depended on the government’s ownership and the capacity of targeted institutions to do so (e.g. the MRDI to implement the Anti-Corruption Policy). Finally, it is important to highlight that the political context of Georgia considerably deteriorated in the last three years, leading to a backlash around the failure to pass the amendments necessary to introduce a proportional electoral system. Political instability might highly influence these indicators. 2.3 TO WHAT EXTENT WAS THE LEVEL OF EFFORT COMMENSURATE WITH THE RESULT? Overall, the project was considered by interviewees to be efficient in mobilizing resources and delivering activities in response to requests from beneficiaries. About one third of the interviewees indicated that the GGI was a “reliable partner” as it could accommodate the different government requests and deliver them on short notice. This resulted in the performance of an extensive number of activities and the production of outputs in terms of tools, mechanisms, studies, and training. These activities were considered by the interviewed organizations to be necessary to push for reforms. In the absence of the GGI project, these activities may have been undertaken by the government or by another development partner. But, most likely, they would not have taken place. The government would not have been able to afford them, and the programmatic approaches from the other development partners would have made it difficult to fit them into USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 23 ongoing projects. Therefore, the GGI, with a variety of activities, was able to address gaps and immediate needs that otherwise would have remained uncovered or delayed. The project also showed good outreach. The training sessions involved about 3,512 people by the end of 2020 who utilized new or improved GGI-assisted online citizen consultation platforms (local and national ones), i-change web portals, web pages developed for local authorities (e.g., Kutaisi or Akhaltsikhe), and citizen consultation tools. Nonetheless, it was difficult to assess the level of economic and human efforts compared to the results, due to the general description of the results and the diversity of reforms supported. Delivering a wide range of short-term and one-off activities targeting so many institutions and sectors required much effort and large investments in terms of time and human and economic resources, often leading to high transaction costs. Organizing so many activities (preparing multiple contracts, logistics, etc.), dealing with different organizational settings of the different targeted organizations (Parliament, local authorities, line ministries, internal audit services, etc.), and the levels of intervention (e.g., local and national), while necessary to keep aligning with government priorities, might have added additional costs to the value of the support provided, increasing the costs of these activities but not necessarily increasing their impact. Based on the available documentation, it is very difficult to calculate the costs devoted to each of the results areas targeted by the project. As previously assessed, the flexible and demand-driven approach might have created some trade-offs in terms of effectiveness. While the project implementation approach may have allowed the GGI to have great outreach and satisfactorily address immediate needs, complementarity among the different types of activities delivered within the same organization or sector was often not ensured, resulting in an atomized impact of the activities, with the exception of results in terms of open governance, which was a key priority for the government. As a result, the government developed great ownership and devoted human, political, and economic resources to implementing it. In fact, the overall achievements in terms of the openness of government, as a result of supporting the implementation of the OGP, would confirm the importance of the focused concentration of activities in leveraging the impact and where the government has more ownership. According to some of the interviewees, the limitations of the GGI project in providing grants to the target institutions also affected the impact. Some of the institutions, such as the CSB, lacked the economic resources, for example, to hire additional staff rather than experts to perform some activities. In these cases, the GGI responded to the needs by providing on-site short-term expertise using external consultants, which was considered substituting capacities rather than giving them the opportunity to develop their existing capacities. Within this context, the beneficiaries did not always appreciate the support received leading to weak ownership over project outputs. Others indicated the lack of quality control of the activities. An impression among limited number of interviewees was that while a lot of efforts were made in responding to the requests from the organizations efficiently and effectively and applying managerial controls to the delivery of activities, quality control over the products was not applied in a systematic manner, which led to studies of low quality. This was especially the case for studies carried out by CSOs, but similar trends were also noticed in other areas with outputs delivered by single experts. At a more strategic level, the strong focus (as per the design) on supporting the supply side of accountability and the limited attention paid to the demand side of accountability led to an unbalanced approach toward democratic governance. Based on both results and impacts, the GGI, as planned, intensively engaged with the national and local GoG, compared to the engagement with Parliament, and other oversight institutions, such as CSOs or the SAO, when their purpose for engaging them was not to promote the supply side of accountability. While the project invested in enhancing institutions and mechanisms to ensure the effectiveness of USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 24 the supply side of accountability, the project was not originally designed to provide incentives to the demand side of governance to hold the government accountable; however, the project did pivot to include grants to CSOs to enhance the demand side. It was also challenging for the GGI to push the institutions involved in the supply side of governance to fulfill their commitments and implement their policies, mainly due to limited capacities on the government side to implement which undermined further impact of GGI. Besides the GGI, some reforms pushed and supported by the GGI lacked further support to be implemented, which most likely undermined the impact of achieving wider reforms in the short and medium term such as the sector policies (e.g. water policy, housing policy or the oversight capacity of parliamentary committees were limitedly used to hold government accountable during Covid).During the Covid crisis, parliament ceded much of its authority to the executive citing the national health emergency. Similarly, the project's investment in the parliamentary budget office (PBO) suffered from a lack of political will/interest from the chair of the budget committee to use the PBO's products or to appoint a full time PBO director. Nonetheless, the project was able to help the PBO to create more user-friendly products (analytical reports written in basic language) and this seems to be generating interest in the PBO again.2.4 Are there specific areas recommended for future programming? The GGI project is characterized by its large scope in addressing a wide range of governance issues, from public sector reform to PFM, open government, decision-making and lawmaking processes, and oversight of government institutions. This involved targeting many central and local governmental institutions and oversight organizations. Nevertheless, support was not balanced across the results areas, having had a strong focus on R2 and R3 but much less in R1 and R4. The large scope of the project did not allow either deepening in certain areas or in certain institutions, addressing more structural gaps and weaknesses. In fact, each of the results areas could constitute a GGI project for itself. Within this context, GGI-targeted areas remain relevant for future programming. Among the recommendations provided by the stakeholders consulted, future programming should put stronger attention on providing support to the decentralization process and self-governance, concretely helping the government in implementing the Decentralization Strategy 2020–2025. In line with this, local authorities should have a more prominent place in future programming as target groups in order to enhance the oversight, transparency, and accountability of the self-governing bodies. Furthermore, it is suggested that participatory budgeting be more widely expanded and consistently supported to ensure its embeddedness within local administration systems. Other areas indicated for further support included continuing with the support to the Parliament, especially in strengthening the institutional apparatus and committees and shifting some aid from policymaking to policy implementation as one of the main bottlenecks for impact was weak policy reforms implementation. Seemingly, support to SAO should continue and even increase the involvement of other independent oversight institutions such as the ombudsman and CSOs, limited or not targeted by the GGI. Indeed, the new interventions aimed at enhancing democratic governance should include a component to strengthen both the demand and supply sides of governance. This requires not only working on governmental oversight institutions but also engaging with CSOs in a more strategic and meaningful manner as oversight organizations, policy implementers, and demanders of accountability. In line with the areas for further work, the engagement of CSOs is seen as of special importance at the local level. CSOs can push the decentralization process acting at the national and local levels. They are key in involving citizens in participatory budgeting and decision-making processes at the local level as well as in holding local governments accountable. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 25 RECOMMENDATIONS ● Increase the focus of the project by reducing the scope (e.g., fewer institutions targeted, or areas of demand-driven support—the support provided to OGP is a good example of focus) while keeping a demand-driven approach (e.g., support to decentralization or oversight institutions). This would reduce the atomization of activities and increase the possibility to leverage efforts and resources for better results. ● Always integrate a component of the supply and demand side of governance and their actors within the focus chosen (e.g., decentralization or specific sectors). It is of particular importance to include in the demand side of accountability the role and participation of CSOs as actors of accountability and civic participation, rather than service providers to the government. ● When involving oversight institutions, open the scope for working with other institutions beyond SAO and target other actors such as the ombudsman and other organizations channeling citizen complaints about the state’s action. EQ 3. CAPACITY-BUILDING As per the SoW, the ET was asked to identify the tools and approaches used to support capacity-building in the GGI project and its contribution to the sustainability of the results. This also involves assessing whether tools and approaches were effective and sustainable and whether they responded to the context needs. FINDINGS 3.1 TO WHAT EXTENT HAS GGI IDENTIFIED AND APPLIED THE MOST EFFECTIVE CAPACITY-BUILDING APPROACHES/TOOLS TO DELIVER AND SUSTAIN RESULTS IN THE KEY REFORM AREAS? One of the most important elements of the GGI was the focus on capacity-building toward strengthening institutional development that many interviewees highly appreciated. Capacity￾building was at the core of the GGI as more than half of the activities were devoted to developing the skills and knowledge of GoG staff. The GGI project carried out a capacity needs assessment at the beginning of the project. Training activities were then planned year by year in collaboration with the beneficiary institutions that, according to interviewees, helped develop ownership of these activities. GGI contracted Georgian experts to provide capacity￾building activities, which conveyed the material in the local language and made the engagement of targeted institutions more effective. As per the information collected during the present evaluation, about 138 training activities targeting 4,073 staff from the GoG, including local authorities and oversight institutions, were delivered up to April 2021. It can be noted that the outreach capacity of each of the training activities also varied. For example, in the municipalities, 21 trainings were attended by 1,181 people, while 24 trainings in Parliament were attended by 637 people. This could be attributed to the different needs of each of the institutions and the number of staff they have. TABLE 4: SUMMARY OF TRAINING ACTIVITIES AND OUTREACH INSTITUTION NATIONAL/LOCAL GOVERNMENT NUMBER OF TRAINING ACTIVITIES NUMBER OF PEOPLE TRAINED MRDI National 14 382 Administration of the GoG/CSB National 28 774 Ministry of Finance National 6 143 USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 26 Municipalities Local 21 1181 Data Exchange Agency National 6 263 SIS National 1 28 Parliament National 34 637 SAO/Public Audit Institute National 16 304 PICD/Institutional Analysis Units National 12 325 Total 138 4037 Nevertheless, in terms of capacity-building, the levels of engagement with beneficiary institutions followed the general trend of the GGI project: more focus on addressing the needs of the central services of the GoG (Administration of the GoG/CSB), the Parliament, and the municipalities, and much less focus on those of line ministries and other oversight institutions. This confirms the deliberate targeting made by the project to prioritize particular stakeholders to achieve the expected results. Again, strengthening the capacities of oversight institutions, besides the Parliament, remains the weakest leg of the project and, therefore, the area with less impact (R4). The ET also noted that the result areas that received more training (e.g., R1) have not necessarily delivered more results or showed more impact in terms of reforms. In terms of tools, the GGI made use of a wide range of capacity-building tools such as coaching, traditional training workshops, on-the-job training, experience sharing, and fellowship programs. More than half of the interviewees who received trainings highly appreciated the trainings because they supported the development of capacities and also incentivized beneficiaries to participate. GGI combined trainings with other technical assistance activities such as those addressing IT issues, which contributed to institutional strengthening. In terms of approach, the project used a “learning by doing” approach despite this not being foreseen in the document's original design. This approach involves combining knowledge with practice. For example, the approach was used to develop the parliamentary oversight capacity following the adoption of the new Rules and Procedures. This involved staff doing research and presentations on their own, which built their skills, confidence, and preparedness. Many of them confirmed they applied the knowledge acquired during those trainings and felt prepared to be involved in distance learning. In the case of the SAO, applying the “learning by doing” approach also resulted in an impact at the institutional level. The GGI supported the development and implementation of a certification to qualify auditors to carry out public audits for the institution in line with international standards. The result was that the SAO can now sign two-year contracts with the auditors who benefitted from the certification courses. However, many interviewees stated it would be good to have had more combined “learning by doing” activities. The “learning by doing” approach was not consistently applied across the project implementation and not in all areas targeted by the GGI. As per the document review, this approach was used more in the last three years of the project and mainly for parliamentary oversight and evidence-based policymaking. The capacity development activities were predominantly based on the delivery of traditional training, often of one-off type and on ad hoc bases. Very few were linked to technical assistance activities (e.g., specific research studies) or a follow-up of technical studies, combining different learning tools for more effective learning (e.g., using research studies as case studies for trainings). Last but not least, the GGI lacked adequate performance measurement to capitalize on the learning process. The use of tools to measure the learning outputs of participants was not systematic; it largely depended on the trainer or consultant. Apart from tracking the number of trainings and participants, the performance indicators did not yield useful information on the efficacy of the type of training, modality, and impact of all the capacity-building activities. A lack of adequate measurement made it very difficult to assess the contribution of these activities to the results area, both at the individual and institutional levels. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 27 3.2 WHICH CAPACITY-BUILDING APPROACHES/TOOLS HAVE RESULTED IN SUSTAINABLE CAPACITY INCREASES FOR KEY ACTORS AND/OR PROCESSES, WHICH HAVE NOT, AND WHY? The GGI project did not specifically identify measures of sustainability in the design of the action. Nevertheless, the ET noted some practices that were integrated during the implementation of the project, which aimed at ensuring the sustainability of capacity-building activities. Some training activities utilized ToT. The ToT approach was mainly applied to training activities targeting CSB staff on the implementation of public sector reforms (e.g., civil service law implementation); representatives of civic engagement centers; and the Public Audit Institute’s policy. Documents were provided to the ET, along with ToT and a manual for ToT; and the capacity built in the SAO auditors and Internal auditors from the GoG Ministries and municipalities. Nevertheless, ToT was not a common practice applied to all the areas of capacity development, which lessened the sustainability of the vast majority of learning outputs. The approach to capacity development activities for the SAO was completely different compared to other targeted institutions. For SAO staff, GGI proposed a series of trainings with an incremental and consistent approach. The trainings were tailor-made—not only to the needs of its staff but also the needs of the institution—to ensure an impact at the institutional level. This allowed for developing knowledge and skills where staff were exposed to a progressive learning process that led to changes at the institutional level in terms of the quality of the audits. Indeed, the SAO was able to certify auditors to carry out public audits for the institution. The training course offered by GGI was also integrated into the training curriculum of the institution, which indicates a greater likelihood of sustainability in the short/medium term. Given the length of the project, this type of approach would have also been suitable for government institutions, including the Parliament and CSB. The GGI relied on existing training institutions to organize and conduct the trainings and several training modules were handed over to them so that trainings could continue after the project implementation. The use of local experts such as CSOs that are characterized for their strong expertise and experience in providing capacity development to the GoG (including municipalities) was also a way to ensure the sustainability of capacity development activities. Pre-evaluation of selected participants was considered particularly important and a step that might have helped to produce better and more sustainable results than short-term training of civil servants that were not screened for their qualifications. Nonetheless, these practices were not applied consistently and coherently. Even though capacity needs assessments were done at the beginning of the project and trainings planned on an annual basis, the project failed to structure the learning activities more systematically to ensure the embeddedness of GGI learning activities within the government’s system. 3.3 WERE THERE COMMON CONTEXTUAL FACTORS THAT AFFECTED THE SUSTAINABILITY OF CAPACITY-BUILDING RESULTS? There was a general impression that the GGI placed a great emphasis on ensuring the sustainability of its activities, especially in recent years. One of the key elements to ensure sustainability was the ownership developed among various institutions. According to interviewees, GGI fomented ownership among counterparts by engaging consultants in “learning by doing” activities, trainings, and coaching, which ensured that the beneficiary acquired sufficient capacity and skills as well as infrastructure (in case of e-interventions) to further sustain processes and services. The project also built new activities upon previously supported activities where strong commitment already existed from the side of the beneficiary. The reputation of USAID, coupled with the ‘extraordinary professionalism’ USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 28 referred by a couple of interviewees, of the GGI team, was also considered a huge factor when exerting pressure over authorities to fulfill their commitments. In other cases, GGI supported the fulfillment of the GoG’s international commitments, such as in the area of OGP and anti-corruption. GGI ensured the institutionalization of new products or services. For instance, GGI allocated annual funding for municipalities where they supported participatory budgeting to ensure continuity in the short/medium term, with the aim to embed the practices within the local institutions. Yet, other contextual factors played against the sustainability of capacity-building efforts. High turnover of staff is a well-known issue within Georgian institutions that undermines the sustainability of any capacity development activity. The permanent and frequent staff changes and consequent brain-drain weakens institutional capacities. This was especially the case for the local authorities, who have limited capacity and financial resources to implement strategies and plans supported by the GGI in line with their limited power/competencies over implementing policies. Decision-making competency is also limited as political decentralization is not completed and the political directions are still controlled and given from the central government. The project often encountered a lack of political will because of staff and leadership issues. With every election, development partners and stakeholders are required to start an engagement process with the new cabinet or new staff who are often skeptical of the project and tend to view it as linked to the previous government. Political will is a critical factor in the case of capacities developed in the Parliament and among parliamentarians. A couple of participants indicated that in the future GGI could pay some more attention to the influence of political parties and raise awareness among political parties to ensure the consistency and sustainability of capacity development, although it is outside the current scope. Initiatives and interventions are still donor-driven. The GGI limitedly aligned the learning approaches within frames of Georgian research institutions or supported the adaptation of the terminology pertaining to the project. Moreover, respondents also indicated that the project tended to rely on the same service providers, so that knowledge hardly went beyond those people. According to few interviewees, the problem would also be that local academic institutions have little experience in being involved in projects like GGI and they have a low capacity in policy brief writing skills or evidence-based research. Finally, some CSO interviewees mentioned that the GGI project may have allowed government institutions to have fewer responsibilities for capacity-building during the last six years. Many respondents felt that cost-sharing could have galvanized the government's sense of ownership. RECOMMENDATIONS ● It is strongly recommended to integrate a capacity-building approach in the design of the activity so that this can respond to the needs of the context and the specific institutions targeted. ● It is also advised that the design of the activity include measures of the sustainability of capacity-building activities and, if possible, an exit strategy so that capacity￾building activities are embedded within the beneficiary institutions. Partnering with existing training institutions is an option that should be considered carefully, including the consideration of developing their capacities as part of this “exit strategy.” ● In long-term projects like the GGI with a strong focus on delivering a wide range of capacity development activities, it is recommended to establish “learning paths” or “learning modules” based on the needs of the institutions and not on the individuals USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 29 in a specific point in time and make this available not only for existing staff but also for future staff. ● Establish a system to measure the participants’ learning performance and the quality of the capacity-building activities to be applied more consistently. The information collected in terms of quality and performance of capacity-building activities should be then analyzed and reported in the project reports and, hence, used for further capacity-building planning and design. ● The ET realizes that reporting was mainly focused on outputs in line with indicators. Learning on capacity-building practices and knowledge was rather limited, contributing further to the atomization of impact. The introduction of collaborative, learning, and adapting techniques together with a Thinking and Working Politically through applied political economy approach to implementation should be prioritized to ensure the project continues to seize political “windows of opportunity” and pivot rapidly. EQ 4. DONOR COORDINATION The sector governance in Georgia relies on an important number of players that need to be taken into account to avoid duplicating efforts and overlapping activities while ensuring complementarity and leveraging aid. This EQ is intended to determine the levels of donor coordination in the framework of the GGI project. FINDINGS 4.1 TO WHAT EXTENT HAS GGI EFFECTIVELY COORDINATED WITH OTHER DONORS ACTIVE IN CENTRAL AND LOCAL GOVERNANCE TO ENHANCE ITS PERFORMANCE? GGI’s reports provided ample evidence of donor coordination. However, the respondents interviewed had mixed opinions as to the level of coordination. While many of these respondents were specifically talking about GGI’s coordination efforts, some of their answers may have also addressed coordination within and by the government, or even specifically related to the OGP. At the beginning of the GGI project, donor coordination was viewed as a serious problem to be addressed. Despite the consultations carried out during the design of the project, some overlapping issues could not be avoided. This was mainly caused by the fact that the development partner relied on GoG leadership in donor coordination. However, the GoG showed challenges in coordinating interventions in the governance sector as well as in terms of communication with line ministries. Within this context, GGI informally established contact with main development partners supporting the governance sector in Georgia—the European Union, Swedish International Development Agency (SIDA), UNDP, GIZ, and the World Bank. These donor organizations meet once per quarter on an informal basis and there are cases of joint advocacy actions by several donor organizations. A certain informal division of labor has been established throughout the project as a result of this coordination. One respondent shared that this happens through informal meetings and ad hoc meetings when necessary but said that communication takes place on a regular basis. In addition to informal meetings, the OGP framework was mentioned by several respondents, from both donors and the government, as helpful in the coordination mechanism beyond development partners and with other key governance stakeholders such as CSOs. It is worth noting that many of the donor coordination activities took place at the national level. Going forward, there could be an increased emphasis, where appropriate, for donor coordination at the local level. However, the evaluators recognize that GGI’s efforts were often bilateral and other donors might not be operating in the same space. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 30 4.2 WERE THERE ANY SPECIFIC CASES OF DONOR COORDINATION/COOPERATION THAT DIRECTLY ASSISTED GGI IN DELIVERING RESULTS? IF SO, WHAT APPROACHES WERE USED? Given the extensive support that the governance sector receives in Georgia, donor coordination and cooperation was vitally important for GGI’s success. As a result of informal coordination efforts made by all development partners, delivery of joint or complementary activities avoiding overlapping and duplication of efforts was possible. Through the present evaluation, it was possible to identify several cases of coordination assisting GGI in delivering results. They are as follows: TECHNICAL ASSISTANCE ● GGI, the NDI, and UNDP’s Gender Program worked together to assist parliament on efforts related to GIA, which contributed to capacity-building of parliamentary committees and the completion of GIAs related to draft laws on labor safety, water resource management, small business taxation and income tax, secondary education, amendments to the law on violence against women and domestic violence, and a package of legislative amendments to the law on domestic violence and gender-based crimes. ● GGI, UNDP, and the IPU supported Parliament’s efforts to institutionalize a self￾assessment toolkit related to gender. As a result, the toolkit is now on Parliament’s website and it will serve as a guide for future strategic and institutional development planning. ● MRDI shared the draft Decentralization Action Plan with donors, including GGI, for their feedback and asked for support in the implementation of the decentralization strategy and action plan. ● GGI and GIZ supported the CSB in presenting the CLS Manual to stakeholders including civil servants, CSOs, donors, and academics. CONVENING POWER/KNOWLEDGE TRANSFER ● GGI, along with IRI, NDI, WVI, and UNDP, supported the initiative of the Regional Policy and Local Self Government Committee in an “Institutional Dialogue.” This general forum served as a platform for communication between Parliament and central and local governments to discuss their communication strategies and action plans. ● With GoG as the initiator, GGI, GIZ, the European Union, and USAID’s Governing for Growth activity all collaborated in a CoP of national RIA practitioners. The CoP aims to strengthen the capacities of public institutions to use evidence-based research in law and policy-making processes. ACTIVITIES RELATED TO CAPACITY-BUILDING: ● GGI, UNDP, NDI, and GIZ co-led trainings of parliamentary units on the amendments of the new Rules and Procedures of Parliament. The goal was to improve parliamentary staff’s and MPs’ comprehension of the new Rules and Procedures. ● MRDI—in cooperation with USAID GGI, Norwegian Mapping Agency, United Nations Economic Commission for Europe, and UN-Habitat—organized a two-day national workshop addressing Sustainable Development Goal 11. ● GGI and the World Bank organized two three-day trainings in public investment management and public-private partnerships. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 31 4.3 WERE THERE OTHER WAYS/APPROACHES FOR DONOR COOPERATION THAT COULD HAVE BROUGHT BETTER RESULTS AND WOULD BE RECOMMENDED FOR FUTURE PROGRAMMING? GoG has some donor coordination mechanisms, mainly informal meetings and OGP coordination meetings. The level of coordination led by the GoG appears to be inefficient and often politicized. This contributed to some overlapping of donor activities during the first years of project implementation. For example, one respondent shared a case when GGI wanted to do the evaluation and assessment of MRDI’s participatory practices, but MRDI stated that GIZ was scheduled to conduct this activity, which was not accurate. Most government respondents made no mention of overlap or challenges with coordination; however, donor respondents shared a different view. One donor respondent stated, “We have made efforts to make the donor coordination more operational, but this was not completely bought into by the government. I also think that donors should try to attempt to improve coordination….In order for the coordination to bring about synergy and effective use of donor resources, it requires coordination at a senior level, as well as a technical level, and these two strands have to be synchronized.” This sentiment was expressed by another donor respondent who mentioned that the ambassador-level coordination was good, but sector-level staff viewed efforts more as a competition and less about coordination. Another donor respondent expressed that there were insufficiencies. Efforts to encourage and strengthen coordination on the GoG’s side could produce better results going forward. Incentives for increased leadership on the GoG side should be explored in the future. In the meantime, informal channels of coordination among the development partners should continue. This evaluation concludes that when coordination between the donors is managed by the state authorities is not effective. Had there been stronger leadership on donor coordination from the government, it could have brought better results. RECOMMENDATIONS ● Conduct a brief mapping exercise to understand where the activity will need to ensure donor coordination for future projects/activities. ● Connect with key donors and GoG institutions to “check in” on coordination efforts and learn if the activity could increase coordination efforts in the short or long term. The EU joint programming process might be a good entry point to connect with other key donors. ● Invest in efforts to strengthen GoG donor coordination in the twilight of the activity to support sustainability in coordination post-activity. Explore incentives that may encourage coordination or barriers that may prevent it. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 32 ANNEX A. KII LIST The following table shows the full list of 60 Key Informants interviewed by the ET by stakeholder group. Stakeholder group: Government of Georgia (N=18) Name and position Organization Ekaterine Kardava; Head of the Civil Service Bureau of Georgia Civil Service Bureau of Georgia Maia Dvalishvili; Deputy Head of the Civil Service Bureau of Georgia Civil Service Bureau of Georgia Irina Aghapishvili; Head of the Analytical Department Civil Service Bureau of Georgia Giorgi Bobghiashvili; Head of Policy Planning Unit Policy Planning and Coordination Department Administration of the Government of Georgia Ketevan Tsanava; Head of the Public Administration Unit of Policy Planning and Coordination Department Administration of the Government of Georgia Nikoloz Rosebashvili; Head of the Department for Development of Local Self-Government and Policy Ministry of Regional Development and Infrastructure of Georgia Giorgi Dididze; Head of the Regional Development and the High Mountains Ministry of Regional Development and Infrastructure of Georgia Nino Kvernadze Ministry of Regional Development and Infrastructure of Georgia (Former staff) Nikoloz Gagua; Deputy Minister Ministry of Finance Tea Gvaramadze; Head of Pension and Social Assistance Division Department for Social Protection Ministry of Internally Displaced Persons from occupied territories, labor, health and social affairs of Georgia Sophia Jiadze; Deputy State Inspector Service of State Inspector Tamar Zubashvili; Head of the Department for International Relations, Analysis and Strategic Development Service of State Inspector Pelagia Makhauri; Deputy Head of Analytical Department Ministry of Justice of Georgia Lana Morgoshia; Senior Specialist; Analytical Department Ministry of Justice of Georgia Gulisa Kakhniashvili; senior specialist; Analytical Department Ministry of Justice of Georgia Nikoloz Gagnidze; Deputy Chairman of the DGA Digital Governance Agency/LEPL of the Ministry of Justice of Georgia Etuna Burduli (instead of Sesili Vardzadze, who moved to the UNDP and did not get permission from her supervisor for the interview); Public Services Development Agency, Research and Development Department Ministry of Justice of Georgia Tamar Gvaramadze; Deputy Public Defender Office of Public Defender The Parliament; The State audit; Municipalities (N=18) Name and position Organization Ketevan Kvinikadze; Secretary-General The Parliament of Georgia Nino Tsilosani; Head of Agricultural Committee The Parliament of Georgia Tamar Omiadze; Head of the Parliamentary Training Center The Parliament of Georgia USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 33 Irina Khasaia; Healthcare and Social Issues Committee Chief of Staff The Parliament of Georgia Teona Mdivani; Budget and Finance Committee Chief Specialist The Parliament of Georgia Temur Chitaia The Parliament of Georgia Maka Tsulaia The Parliament of Georgia Shorena Kakhidze The Parliament of Georgia Giorgi Arziani The Parliament of Georgia Rati Pruidze The Parliament of Georgia Ioseb Kuchava The Parliament of Georgia Tsonte Karkashadze; Head of the strategic planning and budget analyses unit State Audit Office Tinatin Sikmashvili; Head of the certification and training unit Public Audit Institute Giuli Chkuaseli; Head of Public Internal Control Department Public Internal Control Department Irakli Chincharauli; Chair of the OGP WG Tbilisi Municipality Guram Melikidze; First Deputy Mayor Akhaltsikhe Municipality Akaki Gvianidze; Head of the Municipal Services Unit/Municipal Service agency Batumi Municipality Nino Tvaltvadze; Former Deputy Mayor of Kutaisi (2018-2020) Kutaisi Municipality Stakeholder group: Implementing Partner (N=3) Name and position Organization Irina Arabidze GGI/Tetra Tech Marika Gorgadze GGI/Tetra Tech Levan Samadashvili GGI/Tetra Tech Stakeholder group: Development Partners (N=9) Name and position Organization Nana Tsiklauri, Project Manager UNDP Sophie Guruli, Project Manager UNDP Nino Kakubava, Project Manager UNDP Rusudan Abulashvili, Adviser GIZ Archil Zhorzholiani, Programme Coordinator GIZ Teona Kupunia, Acting Director (Georgia) NDI Ana Amiranashvili, Parliamentary Programme Officer NDI Tengiz Gogotishvili, Programme Manager World Bank Sophia Svanadze, National Programme Officer Swiss Cooperation Office Stakeholder group: CSOs (N=21) Name and position Organization Vazha Salamadze, Director Civil Society Institute Natia Apkhazava, Head of Batumi Branch Civil Society Institute Giorgi Gabrielashvili, Director Innovations and Reforms Center (IRC) Marika Chumburidze, Consultant Innovations and Reforms Center (IRC) Norberto Pagnatti, Project Manager (Regulatory Impact Assessment) ISET Policy Institute Nodar Kherkheulidze, National Researcher Independent Reporting Mechanism (IRM) Irina Khantadze, Director Centre for Training and Consultation David Aprasidze, Project Manager of Civil Society Sustainability Instrument Konrad Adenauer Foundation USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 34 David Melua National Association of Local Authorities of Georgia (NALAG) Levan Avalishvili, Program Director Institute for the Development of Freedom of Information (IDFI) Lika Sajaia, Parliamentary Secretary Transparency International Georgia (TIG) George Topuria, Senior Analyst Transparency International Georgia (TIG) Giorgi Oniani, Deputy Executive Director Transparency International Georgia (TIG) Tamar Chugoshvili, Former MP Egeria Solutions Irina Pruidze, Former MP Egeria Solutions Salome Sagaradze, Project Coordinator Georgian Young Lawyers Association (GYLA) Nino Tsukhishvili, consultant (worked in her individual capacity) Georgian Young Lawyers Association (GYLA) Nino Evgenidze, Executive Director Economic Policy Research Center (EPRC) Irina Guruli, Deputy Executive Director Economic Policy Research Center (EPRC) Eka Gvinjilia, Project Consultant Policy and Management Consulting Group (PMCG) David Lezhava, Public Sector Consultant Policy and Management Consulting Group (PMCG) ANNEX B. DESK REPORT The Good Governance Initiative (GGI) Evaluation is an independent midterm performance evaluation to examine the appropriateness of GGI’s design in light of Georgia’s development context, determine the efficiency and effectiveness in advancing key reforms and of select interventions, and provide recommendations on reform topics and technical approaches that the United States Agency for International Development (USAID) should continue to support in the future. The GGI activity is intended to enhance government transparency, accountability, and effectiveness through strengthening the demand and supply side of governance. Concretely, the GGI focused on: ● Result 1: Improved administrative and financial management systems of public institutions at all levels. This includes the public administration system, local and national public financial management systems, and budgeting processes. ● Result 2: Increased openness of government through the improvement of mechanisms to ensure transparency and accountability, including the participation of citizens as part of the checks and balances of government actions. ● Result 3: Strengthened policy development and lawmaking processes, engaging with government agencies and parliament for evidence-based decisions. ● Result 4: Enhanced institutional oversight of government involving not only Parliament but also other state checks and balances institutions such as the State Audit Office Good Governance: Concepts and Elements The theory of change of the GGI project is underpinned by the USAID strategy on Democracy, Human Rights, and Governance (2013). There, democratic governance is key to ensuring sustainable social and economic development and requires the existence of accountable, legitimate, and effective state institutions. In fact, democratic or good governance occurs when the state has the capacity, legitimacy, and authority to deliver public services, ensure rule of law, and act on behalf of the public good.7 There seems to be a consensus around the dimensions necessary to ensure good governance: technical and managerial competencies; 7 Teskey, G., Schnell, S. and Poole, A. (2012) Getting beyond Capacity – Addressing Authority and Legitimacy in Fragile States. See also: Ghani, A., Lockhart, C. and Carnahan, M. (2005) Closing the Sovereignty Gap: an Approach to Statebuilding, or: Fukuyama, F. (2013) “What is Governance?” Centre for Global Development Working Paper 314. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 35 organizational capacities; reliability, predictability, and the rule of law; accountability; transparency and open information systems; and participation. Finally, the quality of governance shapes gender equality. A system that systematically ignores and denies the participation of women, who represent approximately half of the population, and perpetuates the use of patriarchal norms cannot be considered to be founded on democratic governance principles.8 CAPACITY DEVELOPMENT FOR GOOD GOVERNANCE Capacity development generally refers to the process through which people, organizations, and institutions develop and strengthen their capabilities for achieving a set goal. It is an interdependent process where the enhancement of the individual’s skills and knowledge are closely related to organizational capacity and the quality of the institutional framework. This process does not take place in a vacuum but is a multi-faceted approach. Therefore, capacity development must be performed at three levels: individual, organizational, and institutional.9 While individual capacities refer to the process of changing attitudes and behaviors, imparting knowledge, and developing skills, organizational capacities focus on organizational performance and functional capacities. Finally, institutional capacities are concerned with the legal, regulatory, and accountability frameworks that enable organizations and individuals to enhance their capacity in each sphere. The relationships between organizations and their mandate are also elements to consider when assessing institutional capacity. Developing capacities and incentive structures for good governance in developing countries is a complex process. It has been largely proven that the links between democracy and economic development are not straightforward and that representative, accountable, and transparent government is not always a condition for economic development or poverty reduction.10 It is difficult to establish clear links between regime type and developmental outcomes. However, there is also evidence that greater representation and accountability contribute to building a political system that makes decisions in favor of citizens while increasing its legitimacy, reducing corruption, and increasing long-term stability.11 Accountable governments also tend to be more inclusive and equitable. INITIAL FINDINGS BASED ON THE DESK REVIEW Within the frameworks set by the above literature review and based on a desk review of the project documents, the following initial findings were observed. These are preliminary findings and, as such, will need to be triangulated with the information collected during the fieldwork phase to be confirmed. DESIGN AND SCOPE OF THE GGI While the project document provides a good assessment of most of the dimensions related to good governance, there is no specific evidence that the project was designed in a participatory manner. At the design level, the project includes a fair description of the main problems in terms of governance, referring to state institutions targeted by the project. The project also seems to 8 Browne, E. (2014) Gender in Political Economy Analysis, Governance and Social Development Resource Centre Helpdesk Research Report. 9 DAC Network on Governance (2006) The Challenge of Capacity Development: Working towards Good Practice; Lucas, B. (2013) Current Thinking on Capacity development, helpdesk Research Report, GSDRC. 10 See: Fukuyama, F. (2014) “Political Order and Political Decay.” Farrar, Strauss and Giroux. Or: Chang, H. (2002) “Kicking Away the Ladder: Development Strategy in Historical Perspective.” Anthem Press. See also: Putzel, J. and Di John, J. (2012) “Meeting the Challenges of Crisis States.” 11 Tim Kelsall (2014) “Authoritarianism, democracy and development.” Developmental Leadership Program USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 36 adopt a holistic approach toward good governance in the sense that it accounts for both the demand and supply sides of accountability. The situation analysis provides a more robust analysis of challenges facing the supply side at the national and local levels, while there is only a cursory context analysis of challenges faced on the demand side of accountability and civil society is not well considered in the project design. The project document does not include a comprehensive stakeholder analysis. References to gender equality and social inclusion (GESI) issues, including the integration of minorities, are considered cross-cutting issues, but it does not indicate any reference of how GESI will be mainstreamed across the project. At the implementation level, the annual reports state that GGI employed a demand-driven approach. Interventions were designed based on the request/need of the partners, including line ministries and municipalities. Additionally, activities were implemented with the participation of the project partners, which should be an effective manner of addressing partners’ needs. The project includes a theory of change based on a linear causal-result effect result chain (e.g. R1 = R1.1; R1.2 etc.). Result 1 is to improve the technical and managerial competencies of the good governance dimension through supporting specific administration reforms in terms of staff management and public financial management. Results 2, 3, and 4 were designed to enhance accountability, transparency, and reliability of the governmental institutions, through supporting the adoption and implementation of transparency-related policies as well as strengthening the capacity of lawmaking and oversight institutions. All the results should ultimately contribute to enhancing organizational capacities. However, as indicated in the literature review, results in terms of good governance are not straightforward and all the good governance dimensions are connected. In fact, all the expected results and sub-results of GGI’s theory of change can be linked beyond the area they pertain to and, therefore, can contribute to other result areas. It is also observed that some of the different result areas target the same actors (e.g., R3 and R4), which might also lead to some confusion within partner organizations or even some overlapping of actions within the different key result areas, unless the different and additional interlinkages were clear at the time of project implementation. Figure 1: Theory of Change USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 37 CONTRIBUTION AND EFFICIENCY The major challenge in carrying out the desk review was the quality and quantity of the documents analyzed. While some of the Government of Georgia’s (GoG’s) policy documents noted GGI contributions, the Tetra Tech project reports contained mainly descriptive information about the main milestones achieved in the implementation of the project during the period reported. Project reports focused on describing inputs and outputs of the project with limited reporting on result area outcomes achieved. Furthermore, the content of the reports was not consistent (e.g., systematically reporting the same areas). Hence, the information gap created between the project reports and the GoG policy documents made it difficult to establish causal links between GGI contribution and outcomes produced. Nonetheless, some progress in the different results areas could be tracked. Major areas of achievement were observed under Results 1 and 2, mainly in terms of providing implementation inputs at both the national and local level of the Open Government Partnership (OGP) National Action Plan (NAP). In fact, a majority of the targets for the output and outcome indicators were achieved or exceeded for the period 2017–2018. Regarding Result 3 contributions, GGI’s support for the development of the GoG’s Decentralization Strategy (2019–2025) served as the basis for the GoG and Ministry of Regional Development and Infrastructure to use evidence-based decisions and a vision for further decentralization of public services. GGI provided the expertise to develop roadmaps, guidelines, and a toolkit on the development of an inter-municipal cooperation (IMC) framework. GGI supported the MRDI to plan and implement decentralization reform. GGI provided international and local short-term assistance to do a strategic review of public services in Georgia and assess their potential for decentralization to local government authorities. In terms of impact, there seems to be some progress in general and in each result area. However, more information is needed to assess the impact at this stage of the evaluation. Nevertheless, the project has provided support to different stakeholders in different matters, depending on the request for support (i.e., health, urban development, housing, sports, etc.) Result 1: Improved Administrative and Financial Management of Public Institutions at All Levels Outcome 1.1: Internal systems and processes for long-term strategic planning and performance management improved. Outcome 1.2: Strengthened local- and national-level public financial management systems, including program budgeting. Outcome 1.3: Functional civil service and human resource management systems introduced. Result 2: Increased Openness of Government Outcome 2.1: Functional government mechanisms for ensuring transparency and access to public information strengthened. Outcome 2.2: Improved national and local government mechanisms and processes for citizen consultation and engagement. Outcome 2.3: Mechanisms to ensure better accountability at both the national and local levels supported. Result 3: Strengthened Policy Development and Lawmaking Process Outcome 3.1: Internal and inter-agency systems for evidence-based policy development and lawmaking established, including mechanisms for civic consultation. Outcome 3.2: Strengthened capacity of Parliament to produce evidence-based policies and laws. Result 4: Strengthened Institutional Oversight of Government Outcome 4.1: Strengthened capacity of key parliamentary committees to provide oversight of line ministries’ and state agencies’ performance and spending, including but not limited to the decentralization and health governance sectors. Outcome 4.2: Strengthened government external and internal audit mechanisms and systems at national and local levels. Outcome 4.3: Improved systems, tools, and capacity at the national level to conduct ff ti i ht f f f l l t USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 38 and in policy- and budget-making at the local level. For example, some local authorities received support on local public financial management and participatory budgeting, while others received support on policy design and urban development. No specific strategy in terms of sectors and/or areas of support to local authorities was identified (e.g., public finance, local policymaking process for the newly set local authorities, or specific sectors such as education and justice [which is strongly related to rule of law]—only health is targeted as a sector in the project document). The wide variety of sectors targeted corresponds with the fact that the project was designed according to partners’ needs and, therefore, is demand-driven. However, the resulting multi-sector approach and the great number of sometimes “one-shot” activities in different central and local administrations might reduce effectiveness and become one of the main factors undermining project impact. Finally, the role of civil society in holding governments accountable or contributing to evidence-based decision-making is generally overlooked and GESI issues have not been properly mainstreamed. CAPACITY-BUILDING Capacity-building activities targeted mainly individuals and organizations, and less focus was directed at institutional strengthening. The implementing partner used a mix of tools to develop capacity, including trainings, the provision of short- and long-term technical assistance, mentoring, or specific advisory services. The project documents provided limited information on the specific approaches/tools to assess and deliver capacity development activities. GGI conducted 42 trainings. As can be seen in the table below, Result 1 and Result 2 are the areas where more trainings were targeted at both the national and local levels. NUMBER OF TRAININGS (APPROXIMATELY) RESULT 1 RESULT 2 RESULT 3 RESULT 4 4 1 1 1 2 10 8 0 1 0 0 0 7 0 0 0 7 0 0 0 21 11 9 1 TOTAL: 42 GGI’s approach to capacity development emphasized sustainability. For example, offered trainings were incorporated into career development within government systems. Training of trainers was also delivered, resulting in certified trainees. USAID.GOV GEORGIA GGI MID-TERM PERFORMANCE EVALUATION | 39 DONOR COORDINATION GGI involvement in governance coordination mechanisms in good governance in Georgia was not specifically found in the reports. This is likely because USAID attended such meetings, not the GGI project. Nevertheless, references to other development partners supporting the sector are referenced and some potential trends in terms of donor coordination could be tracked. In principle, the GoG leads donor coordination, but the implementation of the GGI project highlights some important weaknesses in terms of donor coordination in Georgia, at least in the good governance sector. During the first three to four years of project implementation, overlapping issues were identified in Result 1—mainly in terms of public finance management and management systems—and in Result 4 in terms of support to parliament in its role as an oversight institution. Another example of overlap is the standardized job evaluation and ranking system for the civil service developed by both GGI and a European Union project, even though both projects were aware of delivering similar activities. Despite duplication issues with different projects implemented by a wide range of development partners, it seems that some efforts were made to reduce replication and transaction costs for GoG institutions; for instance, coordination of support to parliament has been promising. In partnership with United Nations Development Programme, National Democratic Institute, and GIZ, GGI participated in improving a training module to assist in capacity-building around the new Rules of Procedure (RoP) for Development, Monitoring, and Evaluation of the policy documents for parliamentary staff and members of Parliament. GGI also helped organize a workshop for Parliament’s leadership to hold discussions on their oversight function under the newly approved RoP. Nevertheless, evidence of coordination remains limited in the documents and should be further informed through semi-structured interviews.